(42-5) 21 * << * >> * Русский * English * Содержание * Все выпуски

Fusion of information from multiple Kinect sensors for 3D object reconstruction
Ruchay A.N., Dorofeev K.A., Kolpakov V.I.

Federal Research Centre of Biological Systems and Agro-technologies of the Russian Academy of Sciences,
Orenburg, Russia,

Department of Mathematics, Chelyabinsk State University, Chelyabinsk, Russia

 PDF, 154kB

DOI: 10.18287/2412-6179-2018-42-5-898-903

Страницы: 898-903.

Аннотация:
In this paper, we estimate the accuracy of 3D object reconstruction using multiple Kinect sensors. First, we discuss the calibration of multiple Kinect sensors, and provide an analysis of the accuracy and resolution of the depth data. Next, the precision of coordinate mapping between sensors data for registration of depth and color images is evaluated. We test a proposed system for 3D object reconstruction with four Kinect V2 sensors and present reconstruction accuracy results. Experiments and computer simulation are carried out using Matlab and Kinect V2.

Ключевые слова:
multiple sensors, Kinect, 3D object reconstruction, fusion.

Цитирование:
Ruchay AN, Dorofeev KA, Kolpakov VI. Fusion of information from multiple Kinect sensors for 3D object reconstruction. Computer Optics 2018; 42(5): 898-903. DOI: 10.18287/2412-6179-2018-42-5-898-903.

Литература:
  1. Echeagaray-Patron, B.A. Conformal parameterization and curvature analysis for 3D facial recognition / B.A. Echeagaray-Patron, D. Miramontes-Jaramillo, V. Kober // 2015 International Conference on Computational Science and Computational Intelligence (CSCI). – 2015. – P. 843-844. – DOI: 10.1109/CSCI.2015.133.
  2. Echeagaray-Patron, B.A. 3D face recognition based on matching of facial surfaces / B.A. Echeagaray-Patron, V. Kober // Proceedings of SPIE. – 2015. – Vol. 9598. – 95980V. – DOI: 10.1117/12.2186695.
  3. Smelkina, N.A. Reconstruction of anatomical structures using statistical shape modeling [In Russian] / N.A. Smelkina, R.N. Kosarev, A.V. Nikonorov, I.M. Bairikov, K.N. Ryabov, A.V. Avdeev, N.L. Kazanskiy // Computer Optics. – 2017. – Vol. 41(6). – P. 897-904. – DOI: 10.18287/2412-6179-2017-41-6-897-904.
  4. Vokhmintsev, A. A fusion algorithm for building three-dimensional maps / A. Vokhmintsev, A. Makovetskii, V. Kober, I. Sochenkov, V. Kuznetsov // Proceedings of SPIE. – 2015. – Vol. 9599. – 959929. – DOI: 10.1117/12.2187929.
  5. Kotov, A.P. Technology for fast 3D-scene reconstruction from stereo images [In Russian] / A.P. Kotov, V.A. Fursov, Ye.V. Goshin // Computer Optics. – 2015. – Vol. 39(4). – P. 600-605. – DOI: 10.18287/0134-2452-2015-39-4-600-605.
  6. Sochenkov, I. Effective indexing for face recognition / I. Sochenkov, A. Sochenkova, A. Vokhmintsev, A. Makovetskii, A. Melnikov // Proceedings of SPIE. – 2016. – Vol. 9971. – 997124. – DOI: 10.1117/12.2238096.
  7. Picos, K. Accurate three-dimensional pose recognition from monocular images using template matched filtering / K. Picos, V. Diaz-Ramirez, V. Kober, A. Montemayor, J. Pantrigo // Optical Engineering. – 2016. – Vol. 55, Issue 6. – 063102. – DOI: 10.1117/1.OE.55.6.063102.
  8. Echeagaray-Patrón, B.A. Face recognition based on matching of local features on 3D dynamic range sequences / B.A. Echeagaray-Patrón, V. Kober // Proceedings of SPIE. – 2016. – Vol. 9971. – 997131. – DOI: 10.1117/12.2236355.
  9. Echeagaray-Patrón, B.A. A method of face recognition using 3D facial surfaces / B.A. Echeagaray-Patrón, V.I. Kober, V.N. Karnaukhov, V.V. Kuznetsov // Journal of Communications Technology and Electronics. – 2017. – Vol. 62, Issue 6. – P. 648-652. – DOI: 10.1134/S1064226917060067.
  10. Cai, Z. RGB-D datasets using microsoft kinect or similar sensors: a survey / Z. Cai, J. Han, L. Liu, L. Shao // Multimedia Tools and Applications. – 2017. – Vol. 76, Issue 3. – P. 4313-4355. – DOI: 10.1007/s11042-016-3374-6.
  11. Dou, M. 3D scanning deformable objects with a single RGBD sensor / M. Dou, J. Taylor, H. Fuchs, A. Fitzgibbon, S. Izadi // 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). – 2015. – P. 493-501. – DOI: 10.1109/CVPR.2015.7298647.
  12. Guo, K. Real-time geometry, albedo, and motion reconstruction using a single RGB-D camera / K. Guo, F. Xu, T. Yu, X. Liu, Q. Dai, Y. Liu // ACM Transactions on Graphics. – 2017. – Vol. 36, Issue 4. – 44a. – DOI: 10.1145/3072959.3083722.
  13. Namitha, N. Point cloud mapping measurements using kinect RGB-D sensor and kinect fusion for visual odometry / N. Namitha, S.M. Vaitheeswaran, V.K. Jayasree, M.K. Bharat // Procedia Computer Science. – 2016. – Vol. 89. – P. 209-212. – DOI: 10.1016/j.procs.2016.06.044.
  14. Jun, C. Towards a realistic indoor world reconstruction: Preliminary results for an object-oriented 3D RGB-D mapping / C. Jun, J. Kang, S. Yeon, H. Choi, T.-Y. Chung, N.L. Doh // Intelligent Automation & Soft Computing. – 2017. – Vol. 23, Issue 2. – P. 207-218. – DOI: 10.1080/10798587.2016.1186890.
  15. Susanto, W. 3D object detection with multiple kinects / Susanto W, Rohrbach M, Schiele B. // ECCV'12 Proceedings of the 12th international conference on Computer Vision 2012; 2: 93-102. DOI: 10.1007/978-3-642-33868-7_10.
  16. Kowalski, M. Livescan3D: A fast and inexpensive 3D data acquisition system for multiple Kinect v2 Sensors / M. Kowalski, J. Naruniec, M. Daniluk // 2015 International Conference on 3D Vision (3DV). – 2015. – P. 318-325. – DOI: 10.1109/3DV.2015.43.
  17. Córdova-Esparza, D.-M. A multiple camera calibration and point cloud fusion tool for Kinect V2 / D.-M. Córdova-Esparza, J.R. Terven, H. Jiménez-Hernández, A.-M. Herrera-Navarro // Science of Computer Programming. – 2017. – Vol. 143. – P. 1-8. – DOI: 10.1016/j.scico.2016.11.004.
  18. Aguilar-Gonzalez, P.M. Design of correlation filters for pattern recognition with disjoint reference image / P.M. Aguilar-Gonzalez, V. Kober // Optical Engineering. – 2011. – Vol. 50, Issue 11. – 117201. – DOI: 10.1117/1.3643723.
  19. Aguilar-Gonzalez, P.M. Design of correlation filters for pattern recognition using a noisy reference / P.M. Aguilar-Gonzalez, V. Kober // Optics Communications. – 2012. – Vol. 285, Issue 5. – P. 574-583. – DOI: 10.1016/j.optcom.2011.11.012.
  20. Ruchay, A. Clustered impulse noise removal from color images with spatially connected rank filtering / A. Ruchay, V. Kober // Proceedings of SPIE. – 2016. – Vol. 9971. – 99712Y. – DOI: 10.1117/12.2236785.
  21. Ruchay, A. Removal of impulse noise clusters from color images with local order statistics / A. Ruchay, V. Kober // Proceedings of SPIE. – 2017. – Vol. 10396. – 1039626. – DOI: 10.1117/12.2272718.
  22. Ruchay, A. Impulsive noise removal from color video with morphological filtering / A. Ruchay, V. Kober // Proceedings of SPIE. – 2017. – Vol. 10396. – 1039627. – DOI: 10.1117/12.2272719.
  23. Ruchay, A. Impulsive noise removal from color images with morphological filtering / A. Ruchay, V. Kober. – In: Analysis of Images, Social Networks and Texts (AIST 2017) / ed. by W. van der Aalst, D.I. Ignatov, M. Khachay, S.O. Kuznetsov, V. Lempitsky, I.A. Lomazova, N. Loukachevitch, A. Napoli, A. Panchenko, P.M. Pardalos, A.V. Savchenko, S. Wasserman. – Cham: Springer, 2018. – P. 280-291. – DOI: 10.1007/978-3-319-73013-4_26.
  24. Takimoto, R.Y. 3D reconstruction and multiple point cloud registration using a low precision RGB-D sensor / R.Y. Takimoto, M. de Sales GuerraTsuzuki, R. Vogelaar, T. de Castro Martins, A.K. Sato, Y. Iwao, T. Gotoh, S. Kagei // Mechatronics. – 2016. – Vol. 35. – P. 11-22. – DOI: 10.1016/j.mechatronics.2015.10.014.
  25. Nasrin, T. Partially occluded object reconstruction using multiple Kinect sensors / T. Nasrin, F. Yi, S. Das, I. Moon // Proceedings of SPIE. – 2014. – Vol. 9117. – 91171G. – DOI: 10.1117/12.2053938.
  26. Xiang, S. A gradient-based approach for interference cancelation in systems with multiple Kinect cameras / S. Xiang, L. Yu, Q. Liu, Z. Xiong // 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013). – 2013. – P. 13-16. – DOI: 10.1109/ISCAS.2013.6571770.
  27. Susperregi, L. Fusing multiple image transformations and a thermal sensor with kinect to improve person detection ability / L. Susperregi, A. Arruti, E. Jauregi, B. Sierra, J.M. Martinez-Otzeta, E. Lazkano, A. Ansuategui // Engineering Applications of Artificial Intelligence. – 2013. – Vol. 26, Issue 8. – P. 1980-1991. – DOI: 10.1016/j.engappai.2013.04.013.
  28. Kwon, B. Implementation of human action recognition system using multiple Kinect sensors / B. Kwon, D. Kim, J. Kim, I. Lee, J. Kim, H. Oh, H. Kim, S. Lee. – In: Advances in Multimedia Information Processing – PCM 2015 / ed. by Y.S. Ho, J. Sang, Y. Ro, J. Kim, F. Wu. – Cham: Springer, 2015. – Vol. I. – P. 334-343. – DOI: 10.1007/978-3-319-24075-6_32.
  29. Du, H. Data fusion of multiple kinect sensors for a rehabilitation system / H. Du, Y. Zhao, J. Han, Z. Wang, G. Song // 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). – 2016. – P. 4869-4872. – DOI: 10.1109/EMBC.2016.7591818.
  30. Noonan, P.J. Simultaneous multiple Kinect v2 for extended field of view motion tracking / P.J. Noonan, J. Ma, D. Cole, J. Howard, W.A. Hallett, B. Glocker, R. Gunn // 2015 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC). – 2015. – P. 1-4. – DOI: 10.1109/NSSMIC.2015.7582070.
  31. Pathirana, P.N. Robust real-time bio-kinematic movement tracking using multiple kinects for tele-rehabilitation / P.N. Pathirana, S. Li, H.M. Trinh, A. Seneviratne // IEEE Transactions on Industrial Electronics. – 2016. – Vol. 63, Issue 3. – P. 1822-1833. – DOI: 10.1109/TIE.2015.2497662.
  32. Nakazawa, M. Calibration of multiple kinects with little overlap regions / M. Nakazawa, I. Mitsugami, H. Habe, H. Yamazoe, Y. Yagi // IEEJ Transactions on Electrical and Electronic Engineering. – 2015. – Vol. 10, Issue S1. – P. S108-S115. – DOI: 10.1002/tee.22171.
  33. Córdova-Esparza, D.-M. Multiple Kinect V2 calibration / D.-M. Córdova-Esparza, J.R. Terven, H. Jiménez-Hernández, A. Vázquez-Cervantes, A.-M. Herrera-Navarro, A. Ramírez-Pedraza // Automatika. – 2016. – Vol. 57, Issue 3. – P. 810-821. – DOI: 10.7305/automatika.2017.02.1758.
  34. Tsui, K.P. Calibration of multiple Kinect depth sensors for full surface model reconstruction / K.P. Tsui, K.H. Wong, C. Wang, H.C. Kam, H.T. Yau, Y.K. Yu // Proceedings of SPIE. – 2016. – Vol. 10011. – 100111H. – DOI: 10.1117/12.2241159.
  35. Liao, Y. Simultaneous calibration: A joint optimization approach for multiple kinect and external cameras / Y. Liao, Y. Sun, G. Li, J. Kong, G. Jiang, D. Jiang, H. Cai, Z. Ju, H. Yu, H. Liu // Sensors. – 2017. – Vol. 17, Issue 7. – 1491. – DOI: 10.3390/s17071491.
  36. Li, H. Real-time RGB-D image stitching using multiple Kinects for improved field of view / H. Li, H. Liu, N. Cao, Y. Peng, S. Xie, J. Luo, Y. Sun // International Journal of Advanced Robotic Systems. – 2017. – Vol. 14, Issue 2. – P. 1-8. – DOI: 10.1177/1729881417695560.
  37. Choi, S. A large dataset of object scans / S. Choi, Q.-Y. Zhou, S. Miller, V. Koltun // arXiv:1602.02481. – 2016.
  38. Khoshelham, K. Accuracy and resolution of kinect depth data for indoor mapping applications / K. Khoshelham, S.O. Elberink // Sensors. – 2012. – Vol. 12, Issue 2. – P. 1437-1454. – DOI: 10.3390/s120201437.

© 2009, IPSI RAS
Россия, 443001, Самара, ул. Молодогвардейская, 151; электронная почта: journal@computeroptics.ru ; тел: +7 (846) 242-41-24 (ответственный секретарь), +7 (846) 332-56-22 (технический редактор), факс: +7 (846) 332-56-20