(47-5) 05 * << * >> * Russian * English * Content * All Issues

Color imaging using a system based on 3 diffractive lenses
S. Stepanenko 1, V. Evdokimova 1,2, M. Petrov 1, R. Skidanov 1,2, A. Nikonorov 1,2

Samara National Research University, 443086, Samara, Russia, Moskovskoye Shosse 34;
IPSI RAS – Branch of the FSRC "Crystallography and Photonics" RAS,
443001, Samara, Russia, Molodogvardeyskaya 151

 PDF, 2561 kB

DOI: 10.18287/2412-6179-CO-1258

Pages: 716-724.

Full text of article: Russian language.

Abstract:
The possibility of essentially reducing the weight and production cost of computer vision systems has led to the publication of a large number of research works dealing with the development of new imaging systems based on diffractive optics. This study proposes a new imaging system composed of three diffractive lenses, with each forming a separate channel of the color RGB image. This approach allows us to significantly narrow the spectral range of each lens, thus significantly reducing the image distortion caused by chromatic aberration inherent in diffractive optics. It shows that this scheme allows us to perform the neural network-aided image reconstruction, providing a significantly improved resulting image quality. The study proposes a false edge level criterion (FEL) for evaluating the neural network-aided reconstruction.

Keywords:
color correction, color stabilization, global optimization.

Citation:
Stepanenko SO, Evdokimova VV, Petrov MV, Skidanov RV, Nikonorov AV. Color imaging using a system based on 3 diffractive lenses. Computer Optics 2023; 47(5): 716-724. DOI: 10.18287/2412-6179-CO-1258.

Acknowledgements:
This work was financially supported by RSF grant #22-19-00364.

References:

  1. Chen MK, Liu X, Sun Y, Tsai DP. Artificial intelligence in meta-optics. Chem Rev 2022; 122(19): 15356-15413.
  2. Genevet P, Capasso F, Aieta F, Khorasaninejad M, Devlin R. Recent advances in planar optics: from plasmonic to dielectric metasurfaces. Optica 2017; 4(1): 139-152.
  3. Banerji S, et al. Imaging with flat optics: metalenses or diffractive lenses? Optica 2019; 6(6): 805-810.
  4. Nikonorov A, Evdokimova V, Petrov M, Yakimov P, Bibikov S, Yuzifovich Y, Skidanov R, Kazanskiy N. Deep learning-based imaging using single-lens and multi-aperture diffractive optical systems. 2019 IEEE/CVF Int Conf on Computer Vision Workshop (ICCVW) 2019: 3969-3977. DOI: 10.1109/ICCVW.2019.00491.
  5. Kazanskii NL, Khonina SN, Skidanov RV, Morozov AA, Kharitonov SI, Volotovskiy SG. Formation of images using multilevel diffractive lens. Computer Optics 2014; 38(3): 425-434. DOI: 10.18287/0134-2452-2014-38-3-425-434.
  6. Nikonorov A, Skidanov R, Fursov V, Petrov M, Bibikov S, Yuzifovich Y. Fresnel lens imaging with post-capture image processing. 2015 IEEE Conf on Computer Vision and Pattern Recognition Workshops (CVPRW) 2015: 33-41. DOI: 10.1109/CVPRW.2015.7301373.
  7. Nikonorov A, Petrov M, Bibikov S, Yakimov P, Kutikova V, Yuzifovich Y, Morozov A, Skidanov R, Kazanskiy N. Toward ultralightweight remote sensing with harmonic lenses and convolutional neural networks. IEEE J Sel Top Appl Earth Obs Remote Sens 2018; 11(9): 3338-3348. DOI: 10.1109/JSTARS.2018.2856538.
  8. Nikonorov AV, Skidanov RV, Kutikova VV, Petrov MV, Alekseev AP, Bibikov SA, Kazanskiy NL. Towards multi-aperture imaging using diffractive lens. Proc SPIE 2019; 11146: 111460Y. DOI: 10.1117/12.2526923.
  9. Nikonorov A, Petrov M, Bibikov S, Yuzifovich Y, Yakimov P, Kazanskiy N, Skidanov R, Fursov V. Comparative evaluation of deblurring techniques for fresnel lens computational imaging. 2016 23rd Int Conf on Pattern Recognition (ICPR) 2016: 775-780. DOI: 10.1109/ICPR.2016.7899729.
  10. Peng Y, Fu Q, Amata H, Su Sh, Heide F, Heidrich W. Computational imaging using lightweight diffractive-refractive optics. Opt Express 2015; 23(24): 31393-31407.
  11. Skidanov R, Strelkov Y, Volotovsky S, Blank V, Ganchevskaya S, Podlipnov V, Ivliev N, Kazanskiy N. Compact imaging systems based on annular harmonic lenses. Sensors 2020; 20(14): 3914. DOI: 10.3390/s20143914.
  12. Sales TRM, Morris GM. Diffractive-refractive behavior of kinoform lenses. Appl Opt 1997; 36: 253-257.
  13. Mohammad N, Meem M, Shen B, Wang P, Menon R. Broadband imaging with one planar diffractive lens. Sci Rep 2018; 8: 2799.
  14. Kim G, Domínguez-Caballero JA, Menon R. Design and analysis of multi-wavelength difractive optics. Opt Express 2012; 20: 2814-2823.
  15. Blank V, Skidanov R, Doskolovich L, Kazanskiy N. Spectral diffractive lenses for measuring a modified red edge simple ratio index and a water band index. Sensors 2021; 21(22): 7694. DOI: 10.3390/s21227694.
  16. Blank VA, Skidanov RV, Doskolovich LL. Investigation of a spectral lens for the formation of a normalized difference vegetation index NDVI0.705. J Opt Technol 2022; 89(3): 137-141. DOI: 10.1364/JOT.89.000137.
  17. Doskolovich LL, Golub MA, Kazanskiy NL, Khramov AG, Pavelyev VS, Seraphimovich PG, Soifer VA, Volotovskiy SG. Software on diffractive optics and computer generated holograms. Proc. SPIE 1995; 2363: 278-284. DOI: 10.1117/12.199645.
  18. Kazanskiy NL. Modeling diffractive optics elements and devices. Proc SPIE 2018; 10774: 107740O. DOI: 10.1117/12.2319264.
  19. Kazanskiy NL. Research & Education Center of Diffractive Optics. Proc SPIE 2012; 8410: 84100R. DOI: 10.1117/12.923233.
  20. Kazanskiy NL, Skidanov RV. Technological line for creation and research of diffractive optical elements. Proc. SPIE 2019; 11146: 111460W. DOI: 10.1117/12.2527274.
  21. Zhang Y, Li Ku, Li K, Wang L, Zhong B, Fu Y. Image super-resolution using very deep residual channel attention networks. In Book: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Computer Vision – ECCV 2018. Cham: Springer Nature Switzerland AG; 2018: 286-301.
  22. Kim J, Lee JK, Lee KM. Accurate image super-resolution using very deep convolutional networks. 2016 IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2016: 1646-1654.
  23. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y. Residual dense network for image super-resolution. 2018 IEEE/CVF Conf on Computer Vision and Pattern Recognition 2018: 2472-2481.
  24. Wang L, Wang Y, Dong X, Xu Q, Yang J, An W, Guo Y. Unsupervised degradation representation learning for blind superresolution. 2021 IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR) 2021: 10576-10585. DOI: 10.1109/CVPR46437.2021.01044.
  25. Wang Z, Chen J, Hoi SCH. Deep learning for image super-resolution: A survey. IEEE Trans Pattern Anal Mach Intell 2020: 43(10): 3365-3387. DOI: 10.1109/TPAMI.2020.2982166.
  26. Dong C, Loy CC, He K, Tang X. Learning a deep convolutional network for image super-resolution. In Book: Fleet D, Pajdla T, Schiele B, Tuytelaars T, eds. Computer Vision – ECCV 2014. Cham: Springer International Publishing Switzerland; 2014: 184-199.
  27. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y. Image super-resolution using very deep residual channel attention networks. In Book: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Computer Vision – ECCV 2018. Cham: Springer Nature Switzerland AG; 2018: 294-310.
  28. Zhang K, Zuo W, Zhang L. Learning a single convolutional super-resolution network for multiple degradations. 2018 IEEE/CVF Conf on Computer Vision and Pattern Recognition 2018: 3262-3271.
  29. Wang X, Xie L, Dong C, Shan Y. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data. IEEE/CVF Int Conf on Computer Vision Workshops (ICCVW) 2021: 1904-1914.
  30. Soh JW, Cho S, Cho NI. Meta-transfer learning for zero-shot super-resolution. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020: 3513-3522. DOI: 10.1109/CVPR42600.2020.00357.
  31. Evdokimova VV, Petrov MV, Klyueva MA, Zybin EY, Kosianchuk VV,Mishchenko IB, Novikov VM, Selvesiuk NI, Ershov EI, Ivliev NA, Skidanov RV, Kazanskiy NL, Nikonorov AV. Deep learning-based video stream reconstruction in mass production diffractive optical systems. Computer Optics 2021; 45(1): 130-141. DOI: 10.18287/2412-6179-CO-834.
  32. Ivliev N, Evdokimova V, Podlipnov V, Petrov M, Ganchevskaya S, Tkachenko I, Abrameshin D, Yuzifovich Y, Nikonorov A, Skidanov R, Kazanskiy N, Soifer V. First Earth-imaging CubeSat with harmonic diffractive lens. Remote Sens 2022; 14(9): 2230. DOI: 10.3390/rs14092230.
  33. Ronneberger O. U-Net: Convolutional networks for biomedical image segmentation. In Book: Navab N, Hornegger J, Wells WM, Frangi AF, eds. Medical image computing and computerassisted intervention – MICCAI. New York, Dordrecht, London: Springer, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
  34. Dun X, Ikoma H, Wetzstein G, Wang Z, Cheng X, Peng Y. Learned rotationally symmetric diffractive achromat for full-spectrum computational imaging. Optica 2020; 7: 913-922.
  35. Peng Y, Sun Q, Dun X, Wetzstein G, Heidrich W, Heide F. Learned large field-of-view imaging with thin-plate optics. ACM Trans Graph 2019; 38(6): 219. DOI: 10.1145/3355089.3356526.
  36. Foi A, Trimeche M, Katkovnik V, Egiazarian K. Practical Poissonian-Gaussian noise modeling and fitting for single-image raw-data. IEEE Trans Image Process 2008; 17(10): 1737-1754. DOI: 10.1109/TIP.2008.2001399.
  37. Guo S, Yan Z, Zhang K, Zuo W, Zhang L. Toward convolutional blind denoising of real photographs. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2019: 1712-1722. DOI: 10.1109/CVPR.2019.00181.
  38. Do more with less data. 2023. Source: <https://albumentations.ai/>.
  39. Ying Z, Li G, Ren Y, Wang R, Wang W. A new low-light image enhancement algorithm using camera response model. IEEE Int Conf on Computer Vision Workshops (ICCVW) 2017: 3015-3022. DOI: 10.1109/ICCVW.2017.356.
  40. Karpeev SV, Alferov SV, Khonina SN, Kudryashov SI. Study of the broadband radiation intensity distribution formed by diffractive optical elements. Computer Optics 2014; 38(4): 689-694. DOI: 10.18287/0134-2452-2014-38-4-689-694.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20