(48-1) 14 * << * >> * Russian * English * Content * All Issues

Head model reconstruction and animation method using color image with depth information
Y.K. Kozlova 1, V.V. Myasnikov 1,2

1 Samara National Research University,
443086, Samara, Russia, Moskovskoye shosse 34;
2 IPSI RAS – Branch of the FSRC “Crystallography and Photonics” RAS,
443001, Samara, Russia, Molodogvardeyskaya 151

 PDF, 5558 kB

DOI: 10.18287/2412-6179-CO-1334

Pages: 118-122.

Full text of article: English language.

Abstract:
The article presents a method for reconstructing and animating a digital model of a human head from a single RGBD image, a color RGB image with depth information. An approach is proposed for optimizing the parametric FLAME model using a point cloud of a face corresponding to a single RGBD image. The results of experimental studies have shown that the proposed optimization approach makes it possible to obtain a head model with more prominent features of the original face compared to optimization approaches using RGB images or the same approaches generalized to RGBD images.

Keywords:
3D reconstruction, 3D animation, virtual reality, augmented reality, FLAME, RGB image, depth information, RGBD, point cloud, optimization.

Citation:
Kozlova YK, Myasnikov VV. Head model reconstruction and animation method using color image with depth information. Computer Optics 2024; 48(1): 118-122. DOI: 10.18287/2412-6179-CO-1334.

Acknowledgements:
The reported study was funded by the RF Ministry of Science and Higher Education within the state project of FSRC “Crystallography and Photonics” RAS (project 0026-2021-0014).

References:
  1. Goshin YeV, Fursov VA. 3D scene reconstruction from stereo images with unknown extrinsic parameters. Computer Optics 2015; 39(5): 770-775. DOI: 10.18287/0134-2452-2015-39-5-770-776.
  2. Chen L, Cao C, De la Torre F, Saragih J, Xu C, Sheikh Y. High-fidelity face tracking for AR/VR via deep lighting adaptation. arXiv Preprint. 2021. Source: <https://arxiv.org/abs/2103.15876>. DOI: 10.48550/arXiv.2103.15876.
  3. Hu L, Saito S, Wei L, Nagano K, Seo J, Fursund J, Sadeghi I, Sun C, Chen, YC, Li H. Avatar digitization from a single image for real-time rendering. ACM Trans Graph 2017; 36(6): 195. DOI: 10.1145/3130800.3130887.
  4. Feng Y, Feng H, Black MJ, Bolkart T. Learning an animatable detailed 3D face model from in-the-wild images. ACM Trans Graph 2021; 40(4): 88. DOI: 10.1145/3450626.3459936.
  5. Li T, Bolkart T, Black MJ, Li H, Romero J. Learning a model of facial shape and expression from 4D scans. ACM Trans Graph 2017; 36(6): 194. DOI: 10.1145/3130800.3130813.
  6. Dou P, Shah SK, Kakadiaris IA. End-to-end 3D face reconstruction with deep neural networks. 30th IEEE Conf on Computer Vision and Pattern Recognition 2017: 1503-1512. DOI: 10.1109/CVPR.2017.164.
  7. Jackson AS, Bulat A, Argyriou V, Tzimiropoulos G. Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression. 2017 IEEE Int Conf on Computer Vision (ICCV) 2017: 1031-1039. DOI: 10.1109/ICCV.2017.117.
  8. Grassal PW, Prinzler M, Leistner T, Rother C, Nießner M, Thies J. Neural head avatars from monocular RGB videos. arXiv Preprint. 2022. Source: <https://arxiv.org/abs/2112.01554>. DOI: 10.48550/arXiv.2112.01554.
  9. Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees. 2014 IEEE Conf on Computer Vision and Pattern Recognition2014: 1867-1874. DOI: 10.1109/CVPR.2014.241.
  10. Paysan P, Knothe R, Amberg B, Romdhani S, Vetter T. A 3D face model for pose and illumination invariant face recognition. 2009 Sixth IEEE Int Conf on Advanced Video and Signal Based Surveillance 2009: 296-301. DOI: 10.1109/AVSS.2009.58.
  11. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016 IEEE Conf on Computer Vision and Pattern Recognition (CVPR) 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
  12. Bulat A, Tzimiropoulos G, Kingdom U. How far are we from solving the 2D & 3D Face Alignment problem? 2017 IEEE Int Conf on Computer Vision (ICCV) 2017: 1021-1030. DOI: 10.1109/ICCV.2017.116.

© 2009, IPSI RAS
151, Molodogvardeiskaya str., Samara, 443001, Russia; E-mail: journal@computeroptics.ru ; Tel: +7 (846) 242-41-24 (Executive secretary), +7 (846) 332-56-22 (Issuing editor), Fax: +7 (846) 332-56-20