(50-5) 12 * << * >> * Russian * English * Content * All Issues

Application of Deep Learning Technologies for Brightness Peak Detection in Bispiral Images of Fluorescent Point Emitters
A.V. Naumov1, A.A. Rozhentsov2, A.A. Baev2, K.O. Ivanov2

1 Institute of Spectroscopy of the Russian Academy of Sciences (ISAN),
Fizicheskaya St. 5, Troitsk, Moscow, 108840, Russia;
2 Volga State University of Technology, Lenin Square 3, Yoshkar-Ola, 424000, Russia

  Full text (PDF)

DOI: 10.18287/COJ1919

Article ID: 1919

Language: Russian

Abstract:
The article examines approaches to forming training datasets, features of input data representation, and neural network architectures for solving the problem of localizing diffraction peaks in bi-spiral images of fluorescent point emitters (molecules, quantum dots, and others) in three-dimensional localization microscopy with ultra-high spatial resolution using rotating point spread function optics. The advantage of calculating sub-pixel peak coordinates from heatmaps using the soft-argmax operation and the use of pre-trained encoders with limited data volume is demonstrated. In particular, the mean absolute error (MAE) of localization for the model with a pre-trained ConvNeXt-Tiny encoder and soft-argmax was 0.60 pixels, approximately 5 times lower than with direct coordinate regression using the same encoder (3.08 pixels) and 10 times lower than that of a convolutional neural network with randomly initialized weights (6.03 pixels). For the EfficientNet-B0 and ResNet-18 encoders with soft-argmax, the MAE was 0.92 and 2.66 pixels, respectively.

Keywords:
bispiral images, deep learning, artificial intelligence, computer vision, machine learning, nanoscopy, convolutional neural networks, digital microscopy.

Acknowledgements:
This work was financially supported by the Analytical Center for the Government of the Russian Federation under grant No. 70-2025-000738.

Citation:
Naumov AV, Rozhentsov AA, Baev AA, Ivanov KO. Application of Deep Learning Technologies for Brightness Peak Detection in Bispiral Images of Fluorescent Point Emitters. Computer Optics 2026; 50(5): 1919. doi: 10.18287/COJ1919.

References:

  1. Pavani SRP, Thompson MA, Biteen JS, Lord SJ, Liu N, Twieg RJ, Piestun R, Moerner WE. Three-dimensional single-molecule fluorescence imaging beyond the diffraction limit using a double-helix point spread function. Proceedings of the National Academy of Sciences of the United States of America 2009; 106(9): 2995-2999. DOI: 10.1073/pnas.0900245106.
  2. Pavani SRP, Piestun R. 3D microscopy with a double-helix point spread function. In: Proceedings of SPIE 7184, Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XVI. SPIE; 2009: 71840Z. DOI: 10.1117/12.816585.
  3. Shechtman Y, Sahl SJ, Backer AS, Moerner WE. Optimal point spread function design for 3D imaging. Physical Review Letters 2014; 113(13): 133902. DOI: 10.1103/PhysRevLett.113.133902.
  4. Martens KJA, Jabermoradi A, Yang S, Hohlbein J. Integrating engineered point spread functions into the phasor-based single-molecule localization microscopy framework. Methods 2021; 193: 107-115. DOI: 10.1016/j.ymeth.2020.07.010.
  5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521(7553): 436-444. DOI: 10.1038/nature14539.
  6. Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2016. ISBN: 978-0-262-03561-3.
  7. Weigert M, Schmidt U, Haase R, Sugawara K, Myers G. Star-convex polyhedra for 3D object detection and segmentation in microscopy. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. IEEE; 2020: 3655-3662. DOI: 10.1109/WACV45572.2020.9093435.
  8. Hendrycks D, Dietterich T. Benchmarking neural network robustness to common corruptions and perturbations. In: Proceedings of the International Conference on Learning Representations; 2019. arXiv:1903.12261.
  9. Toshev A, Szegedy C. DeepPose: Human pose estimation via deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2014: 1653-1660. DOI: 10.1109/CVPR.2014.214.
  10. Luvizon DC, Tabia H, Picard D. Human pose regression by combining indirect part detection and contextual information. Computers & Graphics 2019; 85: 15-22. DOI: 10.1016/j.cag.2019.09.002.
  11. He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. IEEE; 2017: 2961-2969. DOI: 10.1109/ICCV.2017.322.
  12. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
  13. Tan M, Le QV. EfficientNet: Rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research. Vol 97. PMLR; 2019: 6105-6114.
  14. Liu Z, Mao H, Wu CY, Feichtenhofer C, Darrell T, Xie S. A ConvNet for the 2020s. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2022: 11976-11986. DOI: 10.1109/CVPR52688.2022.01167.
  15. Bayas C, von Diezmann A, Gustavsson AK, Moerner WE. Easy-DHPSF 2.0: Open-source software for three-dimensional localization and two-color registration of single molecules with nanoscale accuracy. Protocol Exchange; 2019. DOI: 10.21203/rs.2.9151/v2.
  16. Lew MD, von Diezmann ARS, Moerner WE. Easy-DHPSF: Open-source software for three-dimensional localization of single molecules with precision beyond the optical diffraction limit. Protocol Exchange; 2013. DOI: 10.1038/protex.2013.026.
  17. Zhang Z, Ochirova A, Liu S, Herbert AD, Wu Y, Boucher W, Lee SF, Laue ED, Klenerman D, Ponjavic A. DHPSFU: A Fiji plugin for fast and accurate double helix-PSF 3D single-molecule localisation microscopy. Scientific Reports 2025; 15: 30551. DOI: 10.1038/s41598-025-15623-3.
  18. Gao J, Wang P, Li W, Zhang X, Song C, Liu Z, Han S, Liu H. Imaging and positioning through scattering media with double-helix point spread function engineering. Journal of Biomedical Optics 2023; 28(4): 046008. DOI: 10.1117/1.JBO.28.4.046008.
  19. Carreira J, Agrawal P, Fragkiadaki K, Malik J. Human pose estimation with iterative error feedback. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2016: 4733-4742. DOI: 10.1109/CVPR.2016.512.
  20. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2016: 779-788. DOI: 10.1109/CVPR.2016.91.
  21. Sun X, Xiao B, Wei F, Liang S, Wei Y. Integral human pose regression. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Computer Vision – ECCV 2018. Lecture Notes in Computer Science. Vol 11210. Springer; 2018: 529-545. DOI: 10.1007/978-3-030-01231-1_33.
  22. Nibali A, He Z, Morgan S, Prendergast L. Numerical coordinate regression with convolutional neural networks. arXiv preprint arXiv:1801.07372; 2018. DOI: 10.48550/arXiv.1801.07372.
  23. Zhou X, Wang D, Krähenbühl P. Objects as points. arXiv preprint arXiv:1904.07850; 2019. DOI: 10.48550/arXiv.1904.07850.
  24. Papandreou G, Zhu T, Chen LC, Gidaris S, Tompson J, Murphy K. PersonLab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. In: Computer Vision – ECCV 2018. Lecture Notes in Computer Science. Vol 11218. Springer; 2018: 282-299. DOI: 10.1007/978-3-030-01264-9_17.
  25. Cheezum MK, Walker WF, Guilford WH. Quantitative comparison of algorithms for tracking single fluorescent particles. Biophysical Journal 2001; 81(4): 2378-2388. DOI: 10.1016/S0006-3495(01)75884-5.
  26. Pallotta L, Giunta G, Clemente C. Sub-pixel SAR image registration through parabolic interpolation of the 2D cross-correlation. IEEE Transactions on Geoscience and Remote Sensing 2020; 58(6): 4132-4144. DOI: 10.1109/TGRS.2019.2961245.
  27. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009: 248-255. DOI: 10.1109/CVPR.2009.5206848.

151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20