(43-6) 10 * << * >> * Русский * English * Содержание * Все выпуски

A novel switching bilateral filtering algorithm for depth map

A.N. Ruchay1,2, K.A. Dorofeev2, V.V. Kalschikov2

1 Federal Research Centre of Biological Systems and Agro-technologies of the Russian Academy of Sciences, Orenburg, Russia,
2 Department of Mathematics, Chelyabinsk State University, Chelyabinsk, Russia

 PDF, 1806 kB

DOI: 10.18287/2412-6179-2019-43-6-1001-1007

Страницы: 1001-1007.

Язык статьи: английский.

Аннотация:
In this paper, we propose a novel switching bilateral filter for depth map from a RGB-D sensor. The switching method works as follows: the bilateral filter is applied not at all pixels of the depth map, but only in those where noise and holes are possible, that is, at the boundaries and sharp changes. With the help of computer simulation we show that the proposed algorithm can effectively and fast process a depth map. The presented results show an improvement in the accuracy of 3D object reconstruction using the proposed depth filtering. The performance of the proposed algorithm is compared in terms of the accuracy of 3D object reconstruction and speed with that of common successful depth filtering algorithms.

Ключевые слова:
depth map, switching filtering, 3D reconstruction.

Цитирование:
Ruchay, A.N. A novel switching bilateral filtering algorithm for depth map / A.N. Ruchay, K.A. Dorofeev, V.V. Kalschikov // Computer Optics. – 2019. – Vol. 43(6). – P.  1001-1007. – DOI: 10.18287/2412-6179-2019-43-6-1001-1007.

Благодарности:
The Russian Science Foundation (project #17-76-20045) financially supported the work.

References:
  1. Gonzalez-Fraga JA, Kober V, Diaz-Ramirez VH, Gutierrez E, Alvarez-Xochihua O. Accurate generation of the 3D map of environment with a RGB-D camera. Proc SPIE 2017; 10396: 103962A. DOI: 10.1117/12.2273074.
  2. Echeagaray-Patrón BA, Kober VI, Karnaukhov VN, Kuznetsov VV. A method of face recognition using 3D facial surfaces. Journal of Communications Technology and Electronics 2017; 62: 648-652. DOI: 10.1134/s1064226917060067.
  3. Ruchay A, Kober V, Yavtushenko E. Fast perceptual image hash based on cascade algorithm. Proc SPIE 2017; 10396: 1039625. DOI: 10.1117/12.2272716.
  4. Ruchay A, Dorofeev K, Kober A. Accurate reconstruction of the 3D indoor environment map with a RGB-D camera based on multiple ICP. Proceedings of the International Conference Information Technology and Nanotechnology. Session Image Processing and Earth Remote Sensing 2018; 2210: 300-308. DOI: 10.18287/1613-0073-2018-2210-300-308.
  5. Ruchay A, Dorofeev K, Kober A, Kolpakov V, Kalschikov V. Accuracy analysis of 3D object shape recovery using depth filtering algorithms. Proc SPIE 2018; 10752: 1075221. DOI: 10.1117/12.2319907.
  6. Ruchay A, Kober V. Impulsive noise removal from color images with morphological filtering. In Book: van der Aalst WMP, et al, eds. Analysis of images, social networks and texts. Cham: Springer International Publishing; 2018: 280-291. DOI: 10.1007/978-3-319-73013-4_26.
  7. Ruchay A, Kober A, Kolpakov V, Makovetskaya T. Removal of impulsive noise from color images with cascade switching algorithm. Proc SPIE 2018; 10752: 1075224. DOI: 10.1117/12.2319914.
  8. Tihonkih D, Makovetskii A, Voronin A. A modified iterative closest point algorithm for noisy data. Proc SPIE 2017; 10396: 103962W. DOI: 10.1117/12.2274139.
  9. Makovetskii A, Voronin S, Kober V. An efficient algorithm for total variation denoising. In Book: Ignatov DI, et al, eds. Analysis of images, social networks and texts. Cham: Springer International Publishing; 2017: 326-337. DOI: 10.1007/978-3-319-52920-2_30.
  10. Voronin S, Makovetskii A, Voronin A, Diaz-Escobar J. A regularization algorithm for registration of deformable surfaces. Proc SPIE 2018; 10752: 107522S. DOI: 10.1117/12.2321521.
  11. Makovetskii A, Voronin S, Kober V. An efficient algorithm of 3D total variation regularization. Proc SPIE 2018; 10752: 107522V. DOI: 10.1117/12.2321646.
  12. Liu W, Chen X, Yang J, Wu Q. Robust color guided depth map restoration. IEEE Trans Image Process 2017; 26: 315-327. DOI: 10.1109/tip.2016.2612826.
  13. Milani S, Calvagno G. Correction and interpolation of depth maps from structured light infrared sensors. Signal Processing: Image Communication 2016; 41: 28-39. DOI: 10.1016/j.image.2015.11.008.
  14. Fu J, Wang S, Lu Y, Li S, Zeng W. Kinect-like depth denoising. 2012 IEEE International Symposium on Circuits and Systems 2012: 512-515. DOI: 10.1109/iscas.2012.6272078.
  15. Lin BS, Chou WR, Yu C, Cheng PH, Tseng PJ, Chen SJ. An effective spatial-temporal denoising approach for depth images. 2015 IEEE International Conference on Digital Signal Processing (DSP) 2015: 647-651. DOI: 10.1109/icdsp.2015.7251954.
  16. Zhang X, Wu R. Fast depth image denoising and enhancement using a deep convolutional network. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2016: 2499-2503. DOI: 10.1109/icassp.2016.7472127.
  17. Moser B, Bauer F, Elbau P, Heise B, Schoner H. Denoising techniques for raw 3D data of TOF cameras based on clustering and wavelets. Proc SPIE 2008; 6805: 68050E. DOI: 10.1117/12.765541.
  18. Boubou S, Narikiyo T, Kawanishi M. Adaptive filter for denoising 3D data captured by depth sensors. 2017 3DTV Conference: The True Vision – Capture, Transmission and Display of 3D Video (3DTV-CON) 2017: 1-4. DOI: 10.1109/3dtv.2017.8280401.
  19. Bhattacharya S, Venkatesh KS, Gupta S. Depth filtering using total variation based video decomposition. 2015 Third International Conference on Image Information Processing (ICIIP) 2015: 23-26. DOI: 10.1109/iciip.2015.7414733.
  20. Frank M, Plaue M, Hamprecht FA. Denoising of continuous-wave time-of-flight depth images using confidence measures. Optical Engineering 2009; 48: 077003. DOI: 10.1117/1.3159869.
  21. Kopf J, Cohen MF, Lischinski D, Uyttendaele M. Joint bilateral upsampling. ACM Trans Graph 2007; 26: 96. DOI: 10.1145/1276377.1276497.
  22. Georgiev M, Gotchev A, Hannuksela M. Real-time denoising of ToF measurements by spatio-temporal non-local mean filtering. 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2013: 1-6. DOI: 10.1109/icmew.2013.6618384.
  23. Lei J, Zhang C, Wu M, You L, Fan K, Hou C. A divide-and-conquer hole-filling method for handling disocclusion in single-view rendering. Multimedia Tools and Applications 2017; 76: 7661-7676. DOI: 10.1007/s11042-016-3413-3.
  24. Zhang Q, Chen M, Zhu H, Wang X, Gan Y. An efficient depth map filtering based on spatial and texture features for 3D video coding. Neurocomputing 2016; 188: 82-89. DOI: 10.1016/j.neucom.2014.11.103.
  25. Chen R, Liu X, Zhai D, Zhao D. Depth image denoising via collaborative graph fourier transform. In Book: Zhai G, Zhou J, Yang X, eds. Digital TV and wireless multimedia communication. Singapore: Springer; 2018: 128-137. DOI: 10.1007/978-981-10-8108-8_12.
  26. Fu M, Zhou W. Depth map super-resolution via extended weighted mode filtering. 2016 Visual Communications and Image Processing (VCIP) 2016: 1-4. DOI: 10.1109/vcip.2016.7805430.
  27. Pourazad MT, Zhou D, Lee K, Karimifard S, Ganelin I, Nasiopoulos P. Improving depth map compression using a 3-phase depth map correction approach. 2015 IEEE International Conference on Multimedia Expo Workshops (ICMEW) 2015: 1-6. DOI: 10.1109/icmew.2015.7169790.
  28. Liu S, Chen C, Kehtarnavaz N. A computationally efficient denoising and hole-filling method for depth image enhancement. Proc SPIE 2016; 9897: 98970V. DOI: 10.1117/12.2230495.
  29. Paris S, Kornprobst P, Tumblin J. Bilateral Filtering. Hanover, MA: Now Publishers Inc; 2009.
  30. Durand F, Dorsey J. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans Graph 2002; 21: 257-266. DOI: 10.1145/566570.566574.
  31. Petschnigg G, Agrawala M, Hoppe H, Szeliski R, Cohen M, Toyama K. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 2004; 23: 664-672. DOI: 10.1145/1015706.1015777.
  32. Ruchay A, Dorofeev K, Kober A. 3D object reconstruction using multiple Kinect sensors and initial estimation of sensor parameters. Proc SPIE 2018; 10752: 1075222. DOI: 10.1117/12.2319911.
  33. Ruchay A, Dorofeev K, Kober A. An efficient detection of local features in depth maps. Proc SPIE 2018; 10752: 1075223. DOI: 10.1117/12.2319913.
  34. Ruchay AN, Dorofeev KA, Kolpakov VI. Fusion of information from multiple Kinect sensors for 3D object reconstruction. Computer Optics 2018; 42(5): 898-903. DOI: 10.18287/2412-6179-2018-42-5-898-903.
  35. Chan D, Buisman H, Theobalt C, Thrun S. A noise-aware filter for real-time depth upsampling. ECCV Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications 2008: M2SFA2.
  36. Min D, Lu J, Do MN. Depth video enhancement based on weighted mode filtering. IEEE Transactions on Image Processing 2012; 21: 1176-1190. DOI: 10.1109/tip.2011.2163164.
  37. Liu J, Gong X. Guided depth enhancement via anisotropic diffusion. In Books: Huet B, Ngo C-W, Tang J, Zhou Z-H, Hauptmann AG, eds. Advances in Multimedia Information Processing – PCM 2013: 408-417. DOI: 10.1007/978-3-319-03731-8_38.
  38. Diebel J, Thrun S. An application of Markov random fields to range sensing. Proc 18th Int Conf Neural Inform Process Systems 2005: 291-298.
  39. Harrison A, Newman P. Image and sparse laser fusion for dense scene reconstruction. In Book: Howard A, Iagnemma K, Kelly A, eds. Field and service robotics. Berlin, Heidelberg: Springer; 2010: 219-228. DOI: 10.1007/978-3-642-13408-1_20.
  40. Yang Q, Yang R, Davis J, Nister D. Spatial-depth super resolution for range images. 2007 IEEE Conference on Computer Vision and Pattern Recognition 2007: 1-8. DOI: 10.1109/cvpr.2007.383211.
  41. Newcombe RA, Izadi S, Hilliges O, Kim D, Davison AJ, Kohli P, et al. KinectFusion: Real-time dense surface mapping and tracking. IEEE ISMAR 2011: 127-136. DOI: 10.1109/ismar.2011.6162880.
  42. Fuhrmann S, Goesele M. Fusion of Depth Maps with Multiple Scales. ACM Trans Graph 2011; 30: 148. DOI: 10.1145/2024156.2024182.
  43. Okada M, Ishikawa T, Ikegaya Y. A computationally efficient filter for reducing shot noise in low S/N data. PLoS ONE 2016; 11: e0157595. DOI: 10.1371/journal.pone.0157595.
  44. Lee K-R, Nguyen TQ. Realistic surface geometry reconstruction using a hand-held RGB-D camera. Mach Vis Appl 2016; 27: 377-385. DOI: 10.1007/s00138-016-0747-9.
  45. Rusu RB, Cousins S. 3D is here: Point Cloud Library (PCL). 2011 IEEE Int Conf Robot Automat 2011: 1-4. DOI: 10.1109/icra.2011.5980567.

© 2009, IPSI RAS
Россия, 443001, Самара, ул. Молодогвардейская, 151; электронная почта: ko@smr.ru ; тел: +7 (846) 242-41-24 (ответственный секретарь), +7 (846) 332-56-22 (технический редактор), факс: +7 (846) 332-56-20