Abstract: Abstract: The method that uses face landmarks to adjust the 3D morphable model is widely applied in 3D face reconstruction, but the calculation of morphable model is time-consuming and often produces errors. In this paper, we improve the fitting method of general 3D morphable model using 2D landmarks of face, and propose a real-time 3D face reconstruction method with multiple angles of video frames. First of all, we recognize the location of landmarks by the CLNF algorithm with three-layer convolutional neural networks and track the landmarks. Then, the head posture is estimated from five senses of face landmarks, and the blendshape coefficients of the model is updated, which can be used to calculate the PCA shape coefficients so as to promote the deformation of the current 3D model. Finally, we employ the ISOMAP algorithm to extract the texture information of the mesh, and proceed texture fusion to form a specific face model. Experimental results demonstrate that our method has better real-time performance and accuracy in 3D face reconstruction.

Key words: Keywords: 3D morphable model, landmarks extraction, blendshape coefficients, PCA shape coefficients, texture fusion