Automatic 3D face reconstruction and tracking using consumer RGB-D camera

Abstract

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] 3D face reconstruction and tracking has become an important research topic during the past few decades in both computer graphics and computer vision. Researchers are seeking for a method to model the human face using a low cost device with high quality. Currently, face models can be captured by expensive active sensor-like laser scanning; however, this sensor technology is not affordable for everyone and experiments must be conducted under certain conditions. In our thesis, we present an automatic 3D face reconstruction and pose estimation framework using a consumer depth camera. Our method does not require any prior face model database. We acquire location of human face part using regular face detector, and in order to generate a high quality face model, we integrate and register information from multiple frames together, which allows detection of noise. In addition, by detecting 2D landmark information provided by the RGB image, we are able to find correspondence in the 3D model. Results are demonstrated by visual inspection. Future application for our research may involve game design and face avatar generation.