(48-4) 11 * << * >> * Русский * English * Содержание * Все выпуски

Applied aspects of modern non-blind image deconvolution methods
O.B. Chaganova 1,3, A.S. Grigoryev 1,2, D.P. Nikolaev 1,4, I.P. Nikolaev 1

1 Institute for Information Transmission Problems, RAS,
127051, Russia, Moscow, Bolshoy Karetny per. 19;
2 Evocargo LLC, 129085, Moscow, Russia,Godovikova st., 9, b. 4;
3 Moscow Institute of Physics and Technology (National Research University),
141701, Russia, Dolgoprudny, Institutskiy per. 9;
4 LLC “Smart Engines Service”, 117312, Russia, Moscow, prospect 60-letiya Oktyabrya 9

  PDF, 4547 kB

DOI: 10.18287/2412-6179-CO-1409

Страницы: 562-572.

Язык статьи: English.

Аннотация:
The focus of this paper is the study of modern non-blind image deconvolution methods and their application to practical tasks. The aim of the study is to determine the current state-of-the-art in non-blind image deconvolution and to identify the limitations of current approaches, with a focus on practical application details. The paper proposes approaches to examine the influence of various effects on the quality of restoration, the robustness of models to errors in blur kernel estimation, and the violation of the commonly assumed uniform blur model. We developed a benchmark for validating non-blind deconvolution methods, which includes datasets of ground truth images and blur kernels, as well as a test scheme for assessing restoration quality and error robustness. Our experimental results show that those neural network models lacking any pre-optimization, such as quantization or knowledge distillation, fall short of classical methods in several key properties, such as inference speed or the ability to handle different types of blur. Nevertheless, neural network models have made notable progress in their robustness to noise and distortions. Based on the results of the study, we provided recommendations for more effective use of modern non-blind image deconvolution methods. We also developed suggestions for improving the robustness, versatility and performance quality of the models by incorporating additional practices into the training pipeline.

Ключевые слова:
non-blind image deconvolution, image deblurring, state-of-the-art methods, method robustness, non-blind deconvolution benchmarking.

Благодарности
This work was supported by the Russian Science Foundation (Project No. 20-61-47089).

Citation:
Chaganova OB, Grigoryev AS, Nikolaev DP, Nikolaev IP. Applied aspects of modern non-blind image deconvolution methods. Computer Optics 2024; 48(4): 562-572. DOI: 10.18287/2412-6179-CO-1409.

References:
  1. Wang H, Sreejith S, Lin Y, Ramachandra N, Slosar A, Yoo S. Neural network based point spread function deconvolution for astronomical applications. arXiv Preprint. 2022. Source: <https://arxiv.org/abs/2210.01666>. DOI: 10.48550/arXiv.2210.01666.
  2. Strohl F, Kaminski C. A joint Richardson-Lucy deconvolution algorithm for the reconstruction of multifocal structured illumination microscopy data. Methods Appl Fluoresc 2015; 3(1): 014002. DOI: 10.1088/2050-6120/3/1/014002.
  3. Agarwal S, Singh O.P., Nagaria D. Deblurring of MRI image using blind and non-blind deconvolution methods. Biomed Pharmacol J 2017; 9(10): 1409-1413. DOI: 10.13005/bpj/1246.
  4. Chakrabarti A. A neural approach to blind motion deblurring. In Book: Leibe S, Matas J, Sebe N, Welling M, eds. Computer Vision – ECCV 2016. Cham: Springer International Publishing AG; 2016: 221-235. DOI: 10.1007/978-3-319-46487-9_14.
  5. Zhang K, Gool L, Radu T. Deep unfolding network for image super-resolution. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition 2020: 3214-3223. DOI: 10.1109/CVPR42600.2020.00328.
  6. Dong J, Roth S, Schiele B. Deep Wiener deconvolution: Wiener meets deep learning for image deblurring. 34th Conf on Neural Information Processing Systems (NeurIPS 2020) 2020. DOI: 10.48550/arXiv.2103.09962.
  7. Nan Y, Ji H. Deep learning for handling kernel/model uncertainty in image deconvolution. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition 2020: 2385-2394. DOI: 10.1109/CVPR42600.2020.00246.
  8. Levin A, Weiss Y, Durand F, Freeman WT. Understanding and evaluating blind deconvolution algorithms. IEEE Conf on Computer Vision and Pattern Recognition 2009: 1964-1971. DOI: 10.1109/CVPR.2009.5206815.
  9. Sun L, Cho S, Wang J, Hays J. Edge-based blur kernel estimation using patch priors. IEEE Int Conf on Computational Photography (ICCP) 2013: 1-8. DOI: 10.1109/ICCPhot.2013.6528301.
  10. Bhandari A, Kadambi A, Raskar R. Computational Imaging. MIT Press; 2022. ISBN: 9780262046473.
  11. Tai Y, Chen X, Kim S, et al. Nonlinear camera response functions and image deblurring: Theoretical analysis and practice. IEEE Trans Pattern Anal Mach Intell 2013; 35(10): 2498-2512. DOI: 10.1109/TPAMI.2013.40.
  12. Fergus R, Singh B, Hertzmann A, et al. Removing camera shake from a single photograph. ACM Trans Graph 2006; 25(3): 787-794. DOI: 10.1145/1141911.1141956.
  13. Anger J, Facciolo G, Delbracio M. Modeling realistic degradations in non-blind deconvolution. 2018 25th IEEE Int Conf on Image Processing (ICIP) 2018: 978-982. DOI: 10.1109/ICIP.2018.8451115.
  14. Whyte O, Sivic J, Zisserman A. Deblurring shaken and partially saturated images. Int J Comput Vis 2014; 110(2): 185-201. DOI: 10.1007/s11263-014-0727-3.
  15. Murli A, D’Amore L, De Simone V. The Wiener filter and regularization methods for image restoration problems. Proc 10th Int Conf on Image Analysis and Processing 1999: 394-399. DOI: 10.1109/ICIAP.1999.797627.
  16. Lee J, Ho Y. High-quality non-blind image deconvolution with adaptive regularization. J Vis Commun Image Represent 2011; 22(7): 653-663. DOI: 10.1016/j.jvcir.2011.07.010.
  17. Bioucas-Dias JM, Figueiredo MAT, Oliveira JP. Total variation-based image deconvolution: A majorization-minimization approach. 2006 IEEE Int Conf on Acoustics Speech and Signal Processing Proceedings 2006; 2: II-II. DOI: 10.1109/ICASSP.2006.1660479.
  18. Lucy LB. An iterative technique for the rectification of observed distributions. Astron J 1974; 79: 745-754. DOI: 10.1086/111605.
  19. Richardson WH. Bayesian-based iterative method of image restoration. J Opt Soc Am 1972; 62(1): 55-59. DOI: 10.1364/JOSA.62.000055.
  20. Huang H, Ma S. Gradient-based image deconvolution. J Electron Imaging 2013; 01(22): 013006. DOI: 10.1117/1.JEI.22.1.013006.
  21. Karnaukhov VN, Mozerov MG. Restoration of multispectral images by the gradient reconstruction method and estimation of the blur parameters on the basis of the multipurpose matching model. J Commun Technol Electron 2016; 61(12): 1426-1431. DOI: 10.1134/S106422691612010X.
  22. Cascarano P, Sebastiani A, Comes MC, Franchini G, Porta F. Combining weighted total variation and deep image prior for natural and medical image restoration via ADMM. 2021 21st Int Conf on Computational Science and its Applications (ICCSA) 2021: 39-46. DOI: 10.1109/ICCSA54496.2021.00016.
  23. Schuler C, Burger H, Harmeling S, Scholkopf B. A machine learning approach for non-blind image deconvolution. IEEE Computer Society Conf on Computer Vision and Pattern Recognition (CVPR) 2013: 1067-1074. DOI: 10.1109/CVPR.2013.142.
  24. Xu L, Ren J, Liu C, Jia J. Deep convolutional neural network for image deconvolution. Adv Neural Inf Process Syst 2014; 27: 1790-1798.
  25. Gong D, Zhang Z, Shi Q, Hengel A, Shen C, Zhang Y. Learning deep gradient descent optimization for image deconvolution. IEEE Trans Neural Netw Learn Syst 2020; 31(12): 5468-5482. DOI: 10.1109/TNNLS.2020.2968289.
  26. Agarwal C, Khobahi S, Bose A, et al. DEEP-URL: A model-aware approach to blind deconvolution based on deep unfolded Richardson-Lucy network. 2020 IEEE International Conference on Image Processing (ICIP) 2020: 3299-3303. DOI: 10.1109/ICIP40778.2020.9190825.
  27. Mou C, Wang Q, Zhang J. Deep generalized unfolding networks for image restoration. arXiv Preprint. 2022. Source: <https://arxiv.org/abs/2204.13348>. DOI: 10.48550/arXiv.2204.13348.
  28. Lai W-S, Huang J-B, Hu Z, et al. A comparative study for single image blind deblurring. 29th IEEE Conf on Computer Vision and Pattern Recognition (CVPR 2016) 2016: 1701-1709. DOI: 10.1109/CVPR.2016.188.
  29. Alkzir NB, Nikolaev IP, Nikolaev DP. SCA-2023: a two-part dataset for benchmarking the methods of image precompensation for users with refractive errors. 37th ECMS Int Conf on Modelling and Simulation 2023: 298-305. DOI: 10.7148/2023-0298.
  30. Horé A, Ziou D. Image quality metrics: PSNR vs. SSIM. 2010 20th Int Conf on Pattern Recognition 2010: 2366-2369. DOI: 10.1109/ICPR.2010.579.
  31. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004; 13(4): 600-612. DOI: 10.1109/tip.2003.819861.
  32. Wang Z, Bovik AC. Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures. IEEE Signal Process Mag 2009; 26(1): 98-117. DOI: 10.1109/MSP.2008.930649.
  33. Kotevski Z, Mitrevski P. Experimental comparison of PSNR and SSIM metrics for video quality estimation. In Book: Davcev D, Gómez JM, eds. ICT Innovations 2009. Berlin, Heidelberg: Springer-Verlag; 2010: 357-366. DOI: 10.1007/978-3-642-10781-8_37.
  34. Kober VI, Karnaukhov VN. Restoration of multispectral images distorted by spatially nonuniform camera motion. J Commun Technol Electron 2015; 60(12): 1366-1371. DOI: 10.1134/S1064226915120153.
  35. Andriyanov NA, Dementiev VE, Vasiliev KK. Developing a filtering algorithm for doubly stochastic images based on models with multiple roots of characteristic equations. Pattern Recognit Image Anal 2019; 29(1): 10-20. DOI: 10.1134/S1054661819010048.
  36. Sher A, Trusov A, Limonova E, Nikolaev D, Arlazarov VV. Neuron-by-neuron quantization for efficient low-bit QNN training. Mathematics 2023: 11(9): 2112. DOI: 10.3390/math11092112.

© 2009, IPSI RAS
Россия, 443001, Самара, ул. Молодогвардейская, 151; электронная почта: journal@computeroptics.ru; тел: +7 (846) 242-41-24 (ответственный секретарь), +7 (846) 332-56-22 (технический редактор), факс: +7 (846) 332-56-20