A study of 3D face reconstruction methods: from geometry to deformable mesh models"

Three-dimensional face reconstruction from 2D images is a complex and challenging task. To tackle this problem, researchers have employed two primary paradigms: geometrical methods and deformable-based models. Geometrical methods rely on multiple images to extract the scene's geometry, and recent advancements in deep learning techniques for Multi-View Stereo (MVS) and Neural Radiance Fields (NeRF) have shown promising results. NeRF, in particular, has demonstrated impressive fidelity in reconstruction, although its performance is sensitive to factors such as the number of training images and the accuracy of camera pose estimation which is a computationally expansive task. Conversely, deformable-based models utilize a parametric representation of the face, constraining the optimization to a more restricted space that can be learned and predicted also from a single image making these methods very efficient and appealing for many use cases. Working on a single image, of course, also hinders their reconstruction capabilities. The contributions of this thesis are threefold. First, we explore and compare the advantages and limitations of both geometrical and deformable-based models in the context of 3D face reconstruction. Second, we curate the generation of a synthetic dataset to investigate the benefits of directly supervising the training of deformable mesh models. Third, we build a new custom network to regress a curated selection of shape parameters optimized over key regions of the face, train it on the synthetic dataset and test it also on real world images. By conducting a comprehensive analysis of these techniques, this research seeks to contribute to the advancement of 3D face reconstruction methods and their practical applications in the field of virtual try-on and augmented reality.