(46-4) 14 * << * >> * Russian * English * Content * All Issues

Automatic segmentation of intracytoplasmic sperm injection images
V.Y. Kovalev 1, A.G. Shishkin 1

1 Lomonosov Moscow State University, 119234, Moscow, Russia, Leninskie Gory st., 1

 PDF, 987 kB

DOI: 10.18287/2412-6179-CO-1060

Pages: 628-633.

Full text of article: Russian language.

Abstract:
In this paper, a multiclass image semantic segmentation problem was solved. For analysis, images of the intracytoplasmic sperm injection process were used. For training the neural network, 656 frames were manually labelled. As a result, each pixel of the images was assigned to one of four classes: microinjector, suction micropipette, oolemma, background. An analysis of modern approaches was carried out and the best architecture, encoders, and hyperparameters of the neural network were selected experimentally: the convolutional neural network FPN (feature pyramid network) with the resnext101 encoder having a depth of 101 layers with 32 parallel separable convolutions. The developed neural network model has allowed obtaining the segmentation efficiency of IOU=0.96 at the algorithm speed of 15 frames per second.

Keywords:
intracytoplasmic sperm injection, semantic segmentation, convolutional neural networks.

Citation:
Kovalev VY, Shishkin AG. Automatic segmentation of intracytoplasmic sperm injection images. Computer Optics 2022; 46(4): 628-633. DOI: 10.18287/2412-6179-CO-1060.

References:
  1. Murid J, Essam M. Intracytoplasmic sperm injection – factors affecting fertilization. In Book: Darwish AMM, ed. Enhancing success of assisted reproduction. Rijeka: IntechOpen; 2012: 117-144.
  2. Hajiyavand AM, Saadat M, Abena A, Sadak F, Sun X. Effect of injection speed on oocyte deformation in ICSI. micromachines 2019; 10: 226.
  3. Hafiz P, Nematollahi M, Boostani R, Jahromi BN. Predicting implantation outcome of in vitro fertilization and intracytoplasmic sperm injection using data mining techniques. Int J Fertil Steril 2017; 11(3): 184-190.
  4. Mostaar A, Sattari MR, Hosseini S, Deevband MR. Use of artificial neural networks and PCA to predict results of infertility treatment in the ICSI method. J Biomed Phys Eng 2019; 9(6): 679-686.
  5. Rubino P, Viganò P, Luddi A, Piomboni P. The ICSI procedure from past to future: a systematic review of the more controversial aspects. Hum Reprod Update 2015; 22(2): 194-227.
  6. Tiegs AW, Scott RT. Evaluation of fertilization, usable blastocyst development and sustained implantation rates according to intracytoplasmic sperm injection operator experience. Reprod Biomed Online 2020; 41(1): 19-27.
  7. Daniel CE, Hickman C, Wilkinson T, Oliana O, Gwinnett D, Trew G, Lavery S. Maximising success rates by improving ICSI technique: which factors affect outcome? Fertil Steril 2015; 104(3): E95-E96.
  8. Shen S, Khabani A, Klein N, Battaglia D. Statistical analysis of factors affecting fertilization rates and clinical outcome associated with intracytoplasmic sperm injection. Fertil Steril 2003; 79(2): 355-360.
  9. Ebner T, Moser M, Sommergruber M, Jesacher K, Tews G. Complete oocyte activation failure after ICSI can be overcome by a modified injection technique. Hum Reprod 2004; 19(8): 1837-1841.
  10. Mirroshandel SA, Ghasemian F. Automated morphology detection from human sperm images. In Book: Palermo GD, Sills ES, eds. Intracytoplasmic sperm injection – Indications, techniques and applications. Springer International Publishing AG; 2018.
  11. Saadat M, Hajiyavand AM, Singh Bedi AP. Oocyte positional recognition for automatic manipulation in ICSI. Micromachines (Basel) 2018; 9(9): 429.
  12. Targosz A, Przystałka P, Wiaderkiewicz R, et al. Semantic segmentation of human oocyte images using deep neural networks. Biomed Eng Online 2021; 20: 40.
  13. Zhao M, Li H, Li R, Li Y, Luo X, Li TC, Lee TL, Wang WJ, Chan DYL. Automated and precise recognition of human zygote cytoplasm: A robust image-segmentation system based on a convolutional neural network. Biomed Signal Process Control 2021; 67: 102551.
  14. He P, et al. Semantic video segmentation for intracytoplasmic sperm injection procedures. arXiv preprint 2021. Source: <https://arxiv.org/abs/2101.01207>.
  15. Ronneberger O, Fischer P, Brox Th. U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention 2015: 234-241.
  16. Lin T, Dollár P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2017: 936-944.
  17. Li H, et al. Visualizing the loss landscape of neural nets. arXiv preprint 2017. Source: <https://arxiv.org/abs/1712.09913>.
  18. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. Proc IEEE Conf on Computer Vision and Pattern Recognition 2015: 1-9.
  19. Xie S, Girshick R, Dollar P, Tu Z, He K. Aggregated residual transformations for deep neural networks. arXiv preprint 2016. Source: <https://arxiv.org/abs/1611.05431v1>.
  20. Hu J, Shen L, Sun G. Squeeze-and-excitation networks. Proc IEEE conf on Computer Vision and Pattern Recognition 2018: 7132-7141.
  21. Tan M, Le Q. EfficientNet: Rethinking model scaling for convolutional neural networks. Proc 36th Int Conf on Machine Learning 2019: 6105-6114.
  22. Chollet F. Xception: Deep learning with depthwise separable convolutions. Proc IEEE conf on Computer Vision and Pattern Recognition 2017: 1251-1258.
  23. Tan C, et al. A survey on deep transfer learning. Int conf on Artificial Neural Networks 2018: 270-279.
  24. Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conf on Computer Vision and Pattern Recognition 2009: 248-255.
  25. Xie Q, et al. Self-training with noisy student improves imagenet classification. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition 2020: 10687-10698.
  26. Kingma DP, Ba J. Adam: A method for stochastic optimization. 3rd Int Conf for Learning Representations 2015. Source: <https://arxiv.org/abs/1412.6980>.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20