(48-5) 15 * << * >> * Russian * English * Content * All Issues

Methodology for detecting and assessing the dynamics of defects in engineering structures by processing images from an unmanned aerial vehicle
M.N. Suetin 1,2, V.E. Dementiev 2, A.G. Tashlinskii 2, R.G. Magdeev 2

1 FRPC JSC 'RPA 'Mars',
432027, Ulyanovsk, Russia, Solnechnaya 20;
2 Ulyanovsk State Technical University,
432027, Ulyanovsk, Russia, Severnyy Venets 32

 PDF, 2513 kB

DOI: 10.18287/2412-6179-CO-1438

Pages: 762-771.

Full text of article: Russian language.

Abstract:
We propose a non-invasive technique for automated detection and assessment of the dynamics of defects in engineering structures based on processing images received from unmanned aerial vehicles during periodic surveillance flights over an engineering structure. The technique includes stages of detecting defects, collating the defect images acquired during the latest and previous surveillance flights, and identifying the dynamics of defect development. An example of the implementation and testing of the methodology for detecting and assessing the dynamics of cracks in metal structures of bridge crossings is given. The technique allows you to significantly reduce the cost of monitoring the condition of defects in engineering structures while increasing the probability of detecting defects.

Keywords:
monitoring, bridge, metal structure, defect, technique, processing, neural network, detection, stochastic adaptation, image, deformation, combination.

Citation:
Suetin MN, Dementiev VE, Tashlinskii AG, Magdeev RG. Methodology for detecting and assessing the dynamics of defects in engineering structures by processing images from an unmanned aerial vehicle. Computer Optics 2024; 48(5): 762-771. DOI: 10.18287/2412-6179-CO-1438.

Acknowledgements:
This work was financially supported by the Russian Science Foundation under grants Nos. 22-21-00513 and 23-21-00249.

References:
  1. Guidelines for monitoring the condition of in-service bridge structures [In Russian]. Industry road methodological document GRM 218.4.002-2008. Moscow: "Federal'noye Dorozhnoye Agentstvo" Publisher; 2008.
  2. Guidelines for inspection and reinforcement of in-service welded spans [In Russian]. Ministry of Railways. Moscow: "NII Mostov LIIZHTa" Publisher; 1990.
  3. Arushonok YuYu. Determination of parameters of building structures operation under load in the process of their testing [In Russian]. Metallicheskiye Konstruktsii 2019; 25(2): 53-64.
  4. Gorokhov EV Evaluation of technical condition and technical diagnostics of metal structures of industrial buildings and structures [In Russian]. Stroitel' Donbassa 2019; 2(7): 15-25.
  5. Kanaeva IA, Ivanova YuA, Spitsyn VG. Deep convolutional generative adversarial network-based synthesis of datasets for road pavement distress segmentation. Computer Optics 2021; 45(6): 907-916. DOI: 10.18287/2412-6179-CO-844.
  6. Su H, Wang X, Han T, Wang Z, Zhao Z, Zhang P. Research on a U-Net bridge crack identification and feature-calculation methods based on a CBAM attention mechanism. Buildings 2022; 12(10): 1561. DOI: 10.3390/buildings12101561
  7. Pu R, Ren G, Li H, Jiang W, Zhang J, Qin H. Autonomous concrete crack semantic segmentation using deep fully convolutional encoder-decoder network in concrete structures inspection. Buildings 2022; 12(11): 2019. DOI: 10.3390/buildings12112019.
  8. Özgenel ÇF. Concrete crack images for classification. Mendeley Data. V2. 2019. Source: <https://data.mendeley.com/datasets/5y9wdsg2zt/2>. DOI: 10.17632/5y9wdsg2zt.2.
  9. Magdeev R, Safina G, Tashlinskii A. Analysis of the influence of background areas on the registration efficiency of multiple images bridge structures. Int Conf on Information Technology and Nanotechnology (ITNT-2023) 2023: 1-5. DOI: 10.1109/ITNT57377.2023.10138954.
  10. Suetin MN, Tashlinskii AG, Magdeev RG. Joint usage of neural networks and stochastic referencing of images while estimating defects in bridge structures. Int Conf on Information Technology and Nanotechnology (ITNT-2022) 2022: 1-5. DOI: 10.1109/ITNT55410.2022.9848624.
  11. Tsypkin YaZ. Information identity theory [In Russian]. Moscow: "Fizmatlit" Publisher; 1995. ISBN: 5-02-015071-1.
  12. Andriyanov NA, Dementiev VE, Tashlinskiy AG. Detection of objects in the images: from likelihood relationships towards scalable and efficient neural networks. Computer Optics 2022; 46(1): 139-159. DOI: 10.18287/2412-6179-CO-922.
  13. Huang B, Reichman D, Collins LM, Bradbury K, Malof JM. Tiling and stitching segmentation output for remoute sensing: basic challenges and recomendations. arXiv Preprint. 2024. Source: <https://arxiv.org/abs/1805.12219>. DOI: 10.48550/arXiv.1805.12219.
  14. Dementev VE, Suetin MN, Gaponova MA. Using Machine Learning Techniques to Detect Defects in Images of Metal Structures. Pattern Recognition and Image Analysis 2021; 31(3): 506-512. DOI: 10.1134/51054661821030068.
  15. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. Proc IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2016; 1: 779-788. DOI: 10.1109/CVPR.2016.91.
  16. Redmon J, Farhadi A. YOLOv3: An incremental improvement. arXiv Preprint. 2024. Source: <https://arxiv.org/abs/1804.02767>. DOI: 10.48550/arXiv.1804.02767.
  17. Kalinina MO, Nikolaev PL. Book spine recognition with the use of deep neural networks. Computer Optics 2020; 44(6): 968-977. DOI: 10.18287/2412-6179-CO-731.
  18. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: a large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), Miami, FL, USA, 2009; 1: 248-255. DOI: 10.1109/CVPR.2009.5206848.
  19. Computer Vision Annotation Tool (CVAT). Source: <https://github.com/opencv/cvat>.
  20. Ronneberger O, Fischer F, Brox T. U-Net: Convolutional networks for biomedical image segmentation. 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Munich, Germany, October 5-9, 2015; 9351: 234-241. DOI: 10.1007/978-3-319-24574-4_28
  21. Iglovicov V, Shvets A. TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. Source: <https://arxiv.org/abs/1801.05746>. DOI: 10.48550/arXiv.1801.05746.
  22. Long J, Shelhamer E, Darrel T. Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015; 1: 3431-3440. DOI: 10.1109/CVPR.2015.7298965.
  23. Chen L-C, Papandreou G, Schroff F, Adam H. Rethinking Atrous Convolution for Semantic Image Segmentation. Source: <https://arxiv.org/abs/1706.05587>. DOI: 10.48550/arXiv.1706.05587
  24. Chen L-C, Zhu Y, Papandreou G, Schroff F. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Book: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Computer vision – ECCV 2018. Cham: Nature Switzerland AG; 2018: 833-851. DOI: 10.1007/978-3-030-01234-2_49.
  25. Dementyiev VE, Andriyanov NA, Vasilyiev KK. Use of images augmentation and implementation of doubly stochastic models for improving accuracy of recognition algorithms based on convolutional neural networks. Systems on Signal Synchronization, Generating and Processing in Telecommunications (SYNCHROINFO) 2020: 1-4. DOI: 10.1109/SYNCHROINFO49631.2020.9166000.
  26. Girshick R. Fast R-CNN. IEEE Int Conf on Computer Vision (ICCV) 2015: 1440-1448. DOI: 10.1109/ICCV.2015.169.
  27. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 2017; 39(6): 1137-1149. DOI: 10.1109/TPAMI.2016.2577031.
  28. Dementev VE, Tashlinsky AG, Suetin MN, Gaponova MA. Improving the quality of video processing based on the use of an artificial YOLO neural network. VIII Int Conf on Information Technology and Nanotechnology (ITNT-2022) 2022: 1-4. DOI: 10.1109/ITNT55410.2022.9848613.
  29. Voronov SV, Tashlinskii AG. Efficiency analysis of information theoretic measures in image registration. Pattern Recogn Image Anal 2016; 26(3): 502-505. DOI: 10.1134/S1054661816030226.
  30. Minkina GL, Samoilov MYu, Tashlinskii AG. Choice of the objective function for pseudogradient measurement of image parameters. Pattern Recogn Image Anal 2007; 17(1): 136-139. DOI:10.1134/S1054661807010166
  31. Tashlinskii AG. Optimization of goal function pseudogradient in the problem of interframe geometrical deformations estimation. In Book: Yin P-Y, ed. Pattern recognition techniques, technology and applications. London, UK: "InTech" Publisher; 2008: 249-280. DOI: 10.5772/6244.
  32. Tashlinskii AG, Tikhonov VO. Methodology for analyzing the error of pseudogradient measurement of parameters of multidimensional processes [In Russian]. Izvestiya VUZov: Radioelektronika 2001, 44(9): 75-80.
  33. Tashlinskii AG, Safina GL, Voronov SV. Pseudogradient optimization of objective function in estimation of geometric interframe image deformations. Pattern Recogn Image Anal 2012; 22(2): 386-392. DOI: 10.1134/S1054661812020174.
  34. Magdeev RG, Tashlinskii AG. A comparative analysis of the efficiency of the stochastic gradient approach to the identification of objects in binary images. Pattern Recogn Image Anal 2014; 24(4): 535-541. DOI: 10.1134/S1054661814040130.
  35. Gonzalez RC, Woods E. Digital image processing. 4th ed. New York: Pearson; 2017. ISBN: 978-1-292-22304-9.
  36. Magdeev R, Tashlinskii A, Safina G. Efficiency of stochastic gradient identification of similar shape objects in binary and grayscale images. CEUR Workshop Proc 2020; 2665: 25-28.
  37. You Y, Cao J, Zhou W. A survey of change detection methods based on remote sensing images for multi-source and multi-objective scenarios. Remote Sens 2020; 12(15): 2460. DOI: 10.3390/rs12152460.
  38. Jiang H, Peng M, Zhong Y, Xie H, Hao Z, Lin J, Ma X, Hu X. A survey on deep learning-based change detection from high-resolution remote sensing images. Remote Sens 2022; 14(7): 1552. DOI: 10.3390/rs14071552.
  39. Zhou Z, Siddiquee MR, Tajbakhsh N, Liang J. UNet++: A nested U-Net architecture for medical image segmentation. 4th Int Workshop, DLMIA 2018, and 8th Int Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 2018: 3-11. DOI: 10.1007/978-3-030-00889-5_1
  40. Peng D, Zhang Y, Guan H. End-to-end change detection for high resolution satellite images using improved UNet++. Remote Sens 2019; 11(11): 1382. DOI: 10.3390/rs11111382.
  41. Zhong Z, Zheng L, Kang G, Li S, Yang Y. Random erasing data augmentation. Proc AAAI Conf on Artificial Intelligence 2020; 34(07): 13001-13008. DOI: 10.1609/aaai.v34i07.7000.
  42. Jamshidi M, El-Badry M, Nourian N. Improving concrete crack segmentation networks through CutMix data synthesis and temporal data fusion. Sensors 2023; 23(1): 504. DOI: 10.3390/s23010504..

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20