(47-5) 12 * << * >> * Russian * English * Content * All Issues

Generation and study of the synthetic brain electron microscopy dataset for segmentation purpose
N.A. Sokolov 1, E.P. Vasiliev 1, A.A. Getmanskaya 1

1 Department of Mathematical Software and Supercomputing Technologies, Lobachevsky University,
603950, Nizhny Novgorod, Russia, Gagarina st. 23

 PDF, 10 MB

DOI: 10.18287/-6179-CO-1273

Pages: 778-787.

Full text of article: English language.

Abstract:
Advanced microscopy technologies such as electron microscopy have opened up a new field of vision for biomedical researchers. The use of artificial intelligence methods for processing EM data is largely difficult due to the small amount of annotated data at the training stage. Therefore, we add synthetic images to an annotated real EM dataset or use a fully synthetic training dataset. In this work, we present an algorithm for the synthesis of 6 types of organelles. Based on the EPFL dataset, a training set of 1161 real fragments 256×256 (ORG) and 2000 synthetic ones (SYN), as well as their combination (MIX), were generated. The experiment of training models for 6, 5-classes and binary segmentation showed that, despite the imperfections of synthetics, training on a mixed (MIX) dataset gave a significant increase (about 0.1) in the Dice metric for 6 and 5 and same results at binary. The synthetic data strategy gives annotations for free, but shifts the effort to producing sufficiently realistic images.

Keywords:
multi-class segmentation, electron microscopy, neural network, image segmentation, machine learning.

Citation:
Sokolov NA, Vasiliev EP, Getmanskaya AA. Generation and study of the synthetic brain electron microscopy dataset for segmentation purpose. Computer Optics 2023; 47(5): 778-787. DOI: 10.18287/-6179-CO-1273.

Acknowledgements:
The work was carried out with the support of the Priority 2030 program.

References:
  1. Deerinck T, Bushong E, Lev-Ram V, Shu X, Tsien R, Ellisman M. Enhancing serial block-face scanning electron microscopy to enable high resolution 3-D nanohistology of cells and tissues. Microsc Microanal 2010; 16: 1138-1139.
  2. Ciresan D, Gambardella L, Giusti A, Schmidhuber J. Deep neural networks segment neuronal membranes in electron microscopy images. NIPS'12: Proc 25th Int Conf on Neural Information Processing Systems 2012; 2: 2852-2860.
  3. Lucchi A, Smith K, Achanta R, Knott G, Fua P. Supervoxel-based segmentation of mitochondria in EM image stacks with learned shape features. IEEE Trans Med Imaging 2012; 31: 474-486.
  4. Helmstaedter M, Mitra P. Computational methods and challenges for large-scale circuit mapping. Curr Opin Neurobiol 2012; 22: 162-169.
  5. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015) 2015: 234-241.
  6. Drozdzal M, Vorontsov E, Chartrand G, Kadoury S, Pal C. The importance of skip connections in biomedical image segmentation. In Book: Carneiro G, Mateus D, Peter L, Bradley A, Tavares JMRS, Belagiannis V, Papa JP, Nascimento JC, Loog M, Lu Z, Cardoso JS, Cornebise J, eds. Deep learning and data labeling for medical applications. Cham: Springer International Publishing AG; 2016: 179-187.
  7. Fakhry A, Zeng T, Ji S. Residual deconvolutional networks for brain electron microscopy image segmentation. IEEE Trans Med Imaging 2017; 36: 447-456.
  8. Xiao C, Liu J, Chen X, Han H, Shu C, Xie Q. Deep contextual residual network for electron microscopy image segmentation in connectomics. 2018 IEEE 15th Int Symp on Biomedical Imaging (ISBI) 2018: 378-381.
  9. Çiçek Ö, Abdulkadir A, Lienkamp S, Brox T, Ronneberger O. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In Book: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W, eds. Medical image computing and computer-assisted intervention – MICCAI 2016. Pt II. Cham: Springer International Publishing AG 2016: 424-432.
  10. Milletari F, Navab N, Ahmadi S. V-Net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth Int Conf on 3D Vision (3DV) 2016: 565-571.
  11. Kamnitsas K, Ledig C, Newcombe V, Simpson J, Kane A, Menon D, Rueckert D, Glocker B. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 2017; 36: 61-78.
  12. Li W, Wang G, Fidon L, Ourselin S, Cardoso M, Vercauteren T. On the compactness, efficiency, and representation of 3D convolutional networks: Brain parcellation as a pretext task. In Book: Niethammer M, Styner M, Aylward S, Zhu H, Oguz I, Yap P-T, Shen D, eds. Information processing in medical imaging. Cham: Springer International Publishing AG; 2017: 348-360.
  13. Zhang Z, Wu C, Coleman S, Kerr D. DENSE-INception U-net for medical image segmentation.  Comput Methods Programs Biomed 2020; 192: 105395.
  14. Mubashar M, Ali H, Grönlund C, Azmat S. R2U++: A multiscale recurrent residual U-Net with dense skip connections for medical image segmentation. Neural Comput Appl 2022; 6: 17723-17739.
  15. Getmanskaya AA, Sokolov NA, Turlapov VE. Multiclass U-Net segmentation of brain electron microscopy data using original and semi-synthetic training datasets. Program Comput Softw 2022; 48: 164-171.
  16. Fend C, Moghiseh A, Redenbach C, Schladitz K. Reconstruction of highly porous structures from FIB-SEM using a deep neural network trained on synthetic images. J Microsc 2021; 281(1): 16-27.
  17. Arganda-Carreras I, Turaga SC, Berger DR, Cireşan D, Giusti A, Gambardella LM, Schmidhuber J, Laptev D, Dwivedi S, Buhmann JM, Liu T, Seyedhosseini M, Tasdizen T, Kamentsky L, Burget R, Uher V, Tan X, Sun C, Pham TD, Bas E, Uzunbas MG, Cardona A, Schindelin J, Seung HS. Crowdsourcing the creation of image segmentation algorithms for connectomics. Front Neuroanat 2015; 9: 142.
  18. Lucchi A, Smith K, Achanta R, Knott G, Fua P. Electron microscopy dataset. 2023. Source: <https://www.epfl.ch/labs/cvlab/data/data-em/>.
  19. Kasthuri N, Hayworth KJ, Berger DR, Schalek RL, Conchello JA, Knowles-Barley S, Lee D, Vázquez-Reina A, Kaynig V, Jones TR, Roberts M, Morgan JL, Tapia JC, Seung HS, Roncal WG, Vogelstein JT, Burns R, Sussman DL, Priebe CE, Pfister H, Lichtman JW. Saturated reconstruction of a volume of neocortex. Cell 2015; 162(3): 648-661.
  20. Mekuč MŽ, Bohak C, Hudoklin S, Kim B, Romih R, Kim M, Marolt M. Automatic segmentation of mitochondria and endolysosomes in volumetric electron microscopy data. Comput Biol Med 2020; 119: 103693.
  21. MancaZerovnikMekuc / UroCell. The UroCell dataset. 2023. Source: <https://github.com/MancaZerovnikMekuc/UroCell>.
  22. Vulović M, Ravelli R, Van Vliet L, Koster A, Lazić I, Lücken U, Rullgård H, Öktem O, Rieger B. Image formation modeling in cryo-electron microscopy. J Struct Biol 2013; 183: 19-32.
  23. Yuan Z, Ma X, Yi J, Luo Z, Peng J. HIVE-Net: Centerline-aware hierarchical view-ensemble convolutional network for mitochondria segmentation in EM images. Comput Methods Programs Biomed 2021; 200: 105925.
  24. Casser V, Kang K, Pfister H, Haehn D. Fast mitochondria detection for connectomics. Proc Mach Learn Res 2020; 121: 111-120.
  25. Cheng H, Varshney A. Volume segmentation using convolutional neural networks with limited training data. 2017 IEEE Int Conf on Image Processing (ICIP) 2017: 590-594.
  26. Peng J, Yuan Z. Mitochondria segmentation from EM images via hierarchical structured contextual forest. IEEE J Biomed Health Inform 2020; 24: 2251-2259.
  27. Cetina K, Buenaposada J, Baumela L. Multi-class segmentation of neuronal structures in electron microscopy images. BMC Bioinformatics 2018; 19: 298.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20