(49-4) 12 * << * >> * Русский * English * Содержание * Все выпуски

Improving the quality of building space depth maps using multi-area active-pulse television measurement systems in dynamic scenes
S.A. Zabuga 1, V.V. Kapustin 1, I.D. Musihin 1

1 Federal State Autonomous Institution of Higher Education "Tomsk State University of Control Systems and Radioelectronics",
634050, Russia, Tomsk, Lenin Ave., 40

  PDF, 2765 kB

DOI: 10.18287/2412-6179-CO-1590

Страницы: 647-653.

Язык статьи: English.

Аннотация:
The purpose of this work is software implementation of the temporal frame interpolation, the formation of selection criteria and the choice of a suitable neural network model based on the obtained practical data. And also, evaluation of its efficiency for eliminating the interframe shift effect of dynamic objects on the depth maps of multi-area active-pulse television measuring systems in order to improve the accuracy of map building. As initial data for the experiments, static frames were recorded while moving the test rig along the X and Z axes. The static frames are images of the test rig, averaged 100 times, at a distance of 13 meters, which moved along an automated linear guide with a step of 1 mm. As a result of the work, an assessment of the interframe shift effect influence on space depth maps of multi-area active-pulse television measuring systems containing dynamic objects was made. The implementation and testing of the temporal frame interpolation algorithm for suppressing the interframe shift effect of dynamic objects on depth maps was also performed. The algorithm was implemented using Python and the PyCharm IDE with SciPy, NumPy, OpenCV, PyTorch, Threading and other libraries. Numerical values of the RMSE, PSNR, and SSIM metrics were obtained before and after eliminating the effect of interframe shift of dynamic objects on depth maps. The use of the temporal frame interpolation algorithm allows more accurate measurement of distance to moving object in the field of view of multi-area active-pulse television measuring systems.

Ключевые слова:
multi-area active-pulse television measurement system, depth map, Python, neural network, video frame prediction.

Благодарности
The study was carried out with the support of the Russian Science Foundation grant No. 21-79-10200 at TUSUR.

Citation:
Zabuga SA, Kapustin VV, Musikhin ID. Improving the quality of building space depths maps using multi-area active-pulse television measuring systems in dynamic scenes. Computer Optics 2025; 49(4): 647-653. DOI: 10.18287/2412-6179-CO-1590.

References:
  1. Kapustin VV, Zahlebin AS, Movchan AK, Kuryachiy MI, Krutikov MV. Experimental assessment of the distance measurement accuracy using the active-pulse television measuring system and a digital terrain model. Computer Optics 2022; 46(6): 948-954. DOI: 10.18287/2412-6179-CO-1114.
  2. Musikhin ID, Kapustin VV, Tislenko AA, Movchan A, Zabuga SA. Building depth maps using an active-pulse television measuring system in real time domain. Scientific Visualization 2024; 16(1): 38-51. DOI: 10.26583/sv.16.1.04.
  3. Kapustin VV, Movchan AK, Tislenko AA. Experimental evaluation of the accuracy of range measurement with multiarea methods using an active-pulse television measuring system. Optoelectron Instrument Proc 2024; 60: 145-155. DOI: 10.3103/S8756699024700134.
  4. Fleet D, Weiss Y. Optical flow estimation. In Book: Paragios N, ed. Handbook of mathematical models in computer vision. Ch 15. Boston: Springer US; 2006: 237-257.
  5. Liu Z, Yeh RA, Tang X, Liu Y, Agarwala A. Video frame synthesis using deep voxel flow. Proc IEEE Int Conf on Computer Vision 2017: 4463-4471. DOI: 10.1109/ICCV.2017.478.
  6. Jiang H, Sun D, Jampani V, Yang MH, Learned-Miller E, Kautz J. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. 2018 IEEE/CVF Conf on Computer Vision and Pattern Recognition 2018: 9000-9008. DOI: 10.1109/CVPR.2018.00938.
  7. Lee H, Kim T, Chung TY, Pak D, Ban Y, Lee S. Adacof: Adaptive collaboration of flows for video frame interpolation. 2020 IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR) 2020: 5316-5325. DOI: 10.1109/CVPR42600.2020.00536.
  8. Huang Z, Zhang T, Heng W, Shi B, Zhou S. Real-time intermediate flow estimation for video frame interpolation. In Book: Avidan S, Brostow G, Cissé M, Farinella GM, Hassner T, eds. Computer vision – ECCV 2022. 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XIV. Cham, Switzerland: Springer Nature Switzerland AG; 2022: 624-642. DOI: 10.1007/978-3-031-19781-9_36.
  9. Zhou T, Tulsiani S, Sun W, Malik J, Efros AA. View synthesis by appearance flow. In Book: Leibe B, Matas J, Sebe N, Welling M, eds. Computer vision – ECCV 2016. 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV. Cham, Switzerland: Springer International Publishing AG; 2016: 286-301. DOI: 10.1007/978-3-319-46493-0_18.
  10. Siyao L, Zhao S, Yu W, Sun W, Metaxas D, Loy CC, Liu Z. Deep animation video interpolation in the wild. 2021 IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR) 2021: 6587-6595. DOI: 10.1109/CVPR46437.2021.00652.
  11. Yuan S, Stenger B, Kim TK. Rgb-based 3d hand pose estimation via privileged learning with depth images. arXiv Preprint. 2018. Source: <https://arxiv.org/abs/1811.07376>. DOI: 10.48550/arXiv.1811.07376.
  12. Ma N, Zhang X, Zheng HT, Sun J. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Book: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Computer vision – ECCV 2018. 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part XIV. Cham, Switzerland: Springer Nature Switzerland AG; 2018: 116-131. DOI: 10.1007/978-3-030-01264-9_8.
  13. Cheng X, Chen Z. Video frame interpolation via deformable separable convolution. Proceedings of the AAAI Conference on Artificial Intelligence 2020; 34(7): 10607-10614. DOI: 10.1609/aaai.v34i07.6634.
  14. Lopez-Paz D, Bottou L, Schölkopf B, Vapnik V. Unifying distillation and privileged information. arXiv Preprint. 2016. Source: <https://arxiv.org/abs/1511.03643>. DOI: 10.48550/arXiv.1511.03643.
  15. Anil R, Pereyra G, Passos A, Ormandi R, Dahl GE, Hinton GE. Large scale distributed neural network training through online distillation. Int Conf on Learning Representations 2018:10-20.
  16. Ranjan A, Black MJ. Optical flow estimation using a spatial pyramid network. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017: 4161-4170. DOI: 10.1109/CVPR.2017.291.
  17. Chen X, Zhang Y, Wang Y, Shu H, Xu C, Xu C. Optical flow distillation: Towards efficient and stable video style transfer. In Book: Vedaldi A, Bischof H, Brox T, Frahm J-M, eds. Computer vision – ECCV 2020. 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI. Cham, Switzerland: Springer Nature Switzerland AG; 2020: 614-630. DOI: 10.1007/978-3-030-58539-6_37.
  18. Meister S, Hur J, Roth S. Unflow: Unsupervised learning of optical flow with a bidirectional census loss. Proceedings of the AAAI Conference on Artificial Intelligence 2018; 32(1): 7251-7259. DOI: 10.1609/aaai.v32i1.12276.
  19. Luo K, Wang C, Liu S, Fan H, Wang J, Sun J. Upflow: Upsampling pyramid for unsupervised optical flow learning. 2021 IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR) 2021: 1045-1054. DOI: 10.1109/CVPR46437.2021.00110.
  20. Lu G, Ouyang W, Xu D, Zhang X, Cai C, Gao Z. Dvc: An end-to-end deep video compression framework. 2019 IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR) 2019: 11006-11015. DOI: 10.1109/CVPR.2019.01126.
  21. Threading – Thread-based parallelism. 2025. Source: <https://docs.python.org/3.10/library/threading.html>.
  22. NumPy. NumPy Documentation. 2025. Source: <https://numpy.org/doc/>.
  23. OpenCV. 2025. Source: <https://opencv.org/>.
  24. PyTorch – Your new deep learning framework [In Russian]. 2025. Source: <https://habr.com/ru/post/334380/>.
  25. SciPy. 2025. Source: <https://scipy.org/>.
  26. Ding C, Lin M, Zhang H, Liu J, Yu L. Video frame interpolation with stereo event and intensity cameras. IEEE Transactions on Multimedia 2024:, 26: 9187-9202. DOI: 10.1109/TMM.2024.3387690.
  27. Wu Y, Wen Q, Chen Q.  Optimizing video prediction via video frame interpolation. IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR) 2022: 17793-17802. DOI: 10.1109/CVPR52688.2022.01729.
  28. Yang B, Jiang G, Fang Y, Li W. Incorporating event information for high-quality video interpolation. Int Conf on Networking, Sensing and Control (ICNSC) 2023: 1-6. DOI: 10.1109/ICNSC58704.2023.10319006.
  29. Mahalingam V, Bhattacharya K, Ranganathan N, Chakravarthula H, Murphy RR, Pratt KS. A VLSI architecture and algorithm for lucas-kanade-based optical flow computation. IEEE Trans Very Large Scale Integr VLSI Syst 2010; 18(1): 29-38. DOI: 10.1109/TVLSI.2008.2006900.
  30. Farneback G. Two-frame motion estimation based on polynomial expansion. In Book: Bigun J, Gustavsson T, eds. Image analysis. 13th Scandinavian Conference, SCIA 2003 Halmstad, Sweden, June 29 – July 2, 2003 Proceedings. Berlin, Heidelberg: Springer-Verlag; 2003: 363-370. DOI: 10.1007/3-540-45103-X_50.

© 2009, IPSI RAS
Россия, 443001, Самара, ул. Молодогвардейская, 151; электронная почта: journal@computeroptics.ru; тел: +7 (846) 242-41-24 (ответственный секретарь), +7 (846) 332-56-22 (технический редактор), факс: +7 (846) 332-56-20