Yao_Feng_Joint_3D_Face_ECCV_2018_paper.zip_CVPR 2018

<html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta charset="utf-8"> <meta name="generator" content="pdf2htmlEX"> <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> <link rel="stylesheet" href="https://static.pudn.com/base/css/base.min.css"> <link rel="stylesheet" href="https://static.pudn.com/base/css/fancy.min.css"> <link rel="stylesheet" href="https://static.pudn.com/prod/directory_preview_static/625cf355be9ad24cfa6f98d8/raw.css"> <script src="https://static.pudn.com/base/js/compatibility.min.js"></script> <script src="https://static.pudn.com/base/js/pdf2htmlEX.min.js"></script> <script> try{ pdf2htmlEX.defaultViewer = new pdf2htmlEX.Viewer({}); }catch(e){} </script> <title></title> </head> <body> <div id="sidebar" style="display: none"> <div id="outline"> </div> </div> <div id="pf1" class="pf w0 h0" data-page-no="1"><div class="pc pc1 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/625cf355be9ad24cfa6f98d8/bg1.jpg"><div class="t m0 x1 h2 y1 ff1 fs0 fc0 sc0 ls0 ws0">Joint 3D Face Reconstruction and Dense</div><div class="t m0 x2 h2 y2 ff1 fs0 fc0 sc0 ls0 ws0">Alignment with Position Map Regression</div><div class="t m0 x3 h2 y3 ff1 fs0 fc0 sc0 ls0 ws0">Network</div><div class="t m0 x4 h3 y4 ff2 fs1 fc0 sc0 ls0 ws0">Yao Feng</div><div class="t m0 x5 h4 y5 ff3 fs2 fc0 sc0 ls0 ws0">1[0000−0002−9481−9783]</div><div class="t m0 x6 h3 y4 ff2 fs1 fc0 sc0 ls0 ws0">, Fan Wu</div><div class="t m0 x7 h4 y5 ff3 fs2 fc0 sc0 ls0 ws0">2[0000−0003−1970−3470]</div><div class="t m0 x8 h3 y4 ff2 fs1 fc0 sc0 ls0 ws0">, Xiaohu</div><div class="t m0 x9 h3 y6 ff2 fs1 fc0 sc0 ls0 ws0">Shao</div><div class="t m0 xa h4 y7 ff3 fs2 fc0 sc0 ls0 ws0">3,4[0000−0003−1141−6020]</div><div class="t m0 xb h3 y6 ff2 fs1 fc0 sc0 ls0 ws0">, Yanfeng Wang</div><div class="t m0 xc h4 y7 ff3 fs2 fc0 sc0 ls0 ws0">1[0000−0002−3196−2347]</div><div class="t m0 xd h3 y6 ff2 fs1 fc0 sc0 ls0 ws0">, and Xi</div><div class="t m0 xe h3 y8 ff2 fs1 fc0 sc0 ls0 ws0">Zhou</div><div class="t m0 xf h4 y9 ff3 fs2 fc0 sc0 ls0 ws0">1,2[0000−0003−2917−0436]</div><div class="t m0 x10 h5 ya ff6 fs3 fc0 sc0 ls0 ws0">1</div><div class="t m0 x11 h6 yb ff7 fs4 fc0 sc0 ls0 ws0">Cooperative Medianet In n ovation Center, Shanghai Jiao Tong University</div><div class="t m0 x12 h5 yc ff6 fs3 fc0 sc0 ls0 ws0">2</div><div class="t m0 x13 h6 yd ff7 fs4 fc0 sc0 ls0 ws0">CloudWalk Technology</div><div class="t m0 x14 h5 ye ff6 fs3 fc0 sc0 ls0 ws0">3</div><div class="t m0 x15 h6 yf ff7 fs4 fc0 sc0 ls0 ws0">CIGIT, Chinese Academy of Sciences</div><div class="t m0 x16 h5 y10 ff6 fs3 fc0 sc0 ls0 ws0">4</div><div class="t m0 x17 h6 y11 ff7 fs4 fc0 sc0 ls0 ws0">University of Chinese Academy of Sciences</div><div class="t m0 x1 h6 y12 ff8 fs4 fc0 sc0 ls0 ws0">Abstract. We propose a straightforward method that simultaneously</div><div class="t m0 x1 h6 y13 ff7 fs4 fc0 sc0 ls0 ws0">reconstructs the 3D facial structure and provides dense alignment. To</div><div class="t m0 x1 h6 y14 ff7 fs4 fc0 sc0 ls0 ws0">achieve this, we design a 2D representation called UV position map which</div><div class="t m0 x1 h6 y15 ff7 fs4 fc0 sc0 ls0 ws0">records the 3D shape of a complete face in UV spac e, then train a sim-</div><div class="t m0 x1 h6 y16 ff7 fs4 fc0 sc0 ls0 ws0">ple Convolutional Neu ra l Network to regress it from a single 2D image.</div><div class="t m0 x1 h6 y17 ff7 fs4 fc0 sc0 ls0 ws0">We also integrate a weight mask into the loss function during train i n g</div><div class="t m0 x1 h6 y18 ff7 fs4 fc0 sc0 ls0 ws0">to i mp rove the performance of the network. Our method does not rely</div><div class="t m0 x1 h6 y19 ff7 fs4 fc0 sc0 ls0 ws0">on any prior face model, and can reconstruct full facial geometry along</div><div class="t m0 x1 h6 y1a ff7 fs4 fc0 sc0 ls0 ws0">with semantic meaning. Meanwhile, our network is very light-weighted</div><div class="t m0 x1 h6 y1b ff7 fs4 fc0 sc0 ls0 ws0">and spends only 9.8ms to process an image, which is extremely faster</div><div class="t m0 x1 h6 y1c ff7 fs4 fc0 sc0 ls0 ws0">than previous works. Experiments on multiple challenging datasets show</div><div class="t m0 x1 h6 y1d ff7 fs4 fc0 sc0 ls0 ws0">that our method surpasses other state-of-the-a rt methods on both re-</div><div class="t m0 x1 h6 y1e ff7 fs4 fc0 sc0 ls0 ws0">construction and alignment tasks by a large margin. Code is available at</div><div class="t m0 x1 h6 y1f ff7 fs4 fc0 sc0 ls0 ws0">https://github.com/YadiraF/PRNet.</div><div class="t m0 x1 h6 y20 ff8 fs4 fc0 sc0 ls0 ws0">Keywords: 3D Face R eco n st ru c ti o n · Dense Face Al ig n m ent</div><div class="t m0 x18 h7 y21 ff1 fs5 fc0 sc0 ls0 ws0">1 Introduction</div><div class="t m0 x18 h3 y22 ff2 fs1 fc0 sc0 ls0 ws0">3D face reconstruction and face alignment are two fundamental and highly re-</div><div class="t m0 x18 h3 y23 ff2 fs1 fc0 sc0 ls0 ws0">lated topics i n computer vision. In the last decades, resear ches in these two ﬁelds</div><div class="t m0 x18 h3 y24 ff2 fs1 fc0 sc0 ls0 ws0">beneﬁt each other. In the beginning, face alignment that aims at detecting a spe-</div><div class="t m0 x18 h3 y25 ff2 fs1 fc0 sc0 ls0 ws0">cial 2D ﬁducial points [66, 64, 38, 46] is commonly used as a prerequisite for other</div><div class="t m0 x18 h3 y26 ff2 fs1 fc0 sc0 ls0 ws0">facial tasks such as face recognition [59] and assists 3D face rec ons tr u ct i on [68,</div><div class="t m0 x18 h3 y27 ff2 fs1 fc0 sc0 ls0 ws0">27] to a great extent. However, researchers ﬁnd that 2D alignment has diﬃculties</div><div class="t m0 x18 h3 y28 ff2 fs1 fc0 sc0 ls0 ws0">[65, 30] in dealing with problems of l arge poses or occ lu si on s. With the devel-</div><div class="t m0 x18 h3 y29 ff2 fs1 fc0 sc0 ls0 ws0">opment of deep learning, many comput er vision problems have been well solved</div><div class="t m0 x18 h3 y2a ff2 fs1 fc0 sc0 ls0 ws0">by utilizi ng Convolution Neural Networks (CNNs). Thus, some works start to</div><div class="t m0 x18 h3 y2b ff2 fs1 fc0 sc0 ls0 ws0">use CNNs to estimate the 3D Morphable M odel (3DMM) coeﬃcients [32, 67,</div><div class="t m0 x18 h3 y2c ff2 fs1 fc0 sc0 ls0 ws0">47, 39, 48, 40] or 3D model warping functions [4, 53] to restore the corres ponding</div><div class="t m0 x18 h3 y2d ff2 fs1 fc0 sc0 ls0 ws0">3D informat i on from a single 2D f aci al image, which provides both dense face</div></div><div class="pi" data-data='{"ctm":[1.568627,0.000000,0.000000,1.568627,0.000000,0.000000]}'></div></div> </body> </html>

<div id="pf2" class="pf w0 h0" data-page-no="2"><div class="pc pc2 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/625cf355be9ad24cfa6f98d8/bg2.jpg"><div class="t m0 x18 h6 y2e ff7 fs4 fc0 sc0 ls0 ws0">2 Y. Feng et al.</div><div class="t m0 x18 h3 y2f ff2 fs1 fc0 sc0 ls0 ws0">Fig. 1: The qualit at i ve results of our method. Odd row: alignment results (on ly</div><div class="t m0 x18 h3 y30 ff2 fs1 fc0 sc0 ls0 ws0">68 key points are plotted for di sp l ay). Even row: 3D reconstruction result s (re-</div><div class="t m0 x18 h3 y31 ff2 fs1 fc0 sc0 ls0 ws0">constructed shapes are rendered with head light for better view).</div><div class="t m0 x18 h3 y32 ff2 fs1 fc0 sc0 ls0 ws0">alignment and 3D face reconstruction resu l t s. However, the performance of these</div><div class="t m0 x18 h3 y33 ff2 fs1 fc0 sc0 ls0 ws0">methods is restricted due to the limitation of the 3D sp ace deﬁned by face model</div><div class="t m0 x18 h3 y34 ff2 fs1 fc0 sc0 ls0 ws0">basis or templates. The required operations including perspective projection or</div><div class="t m0 x18 h3 y35 ff2 fs1 fc0 sc0 ls0 ws0">3D Thin Plate Spline (TPS) transformation also add complexity to the overall</div><div class="t m0 x18 h3 y36 ff2 fs1 fc0 sc0 ls0 ws0">process.</div><div class="t m0 x9 h3 y37 ff2 fs1 fc0 sc0 ls0 ws0">Recently, two end-to-end works [28] [9], which bypass the limitation of model</div><div class="t m0 x18 h3 y38 ff2 fs1 fc0 sc0 ls0 ws0">space, achieve the state-of-the-art performances on their respective tasks. [9]</div><div class="t m0 x18 h3 y39 ff2 fs1 fc0 sc0 ls0 ws0">trains a complex network to regress 68 facial landmarks with 2D coordinates</div><div class="t m0 x18 h3 y3a ff2 fs1 fc0 sc0 ls0 ws0">from a single image, but needs an extra network to estimate t he depth value.</div><div class="t m0 x18 h3 y3b ff2 fs1 fc0 sc0 ls0 ws0">Besides, den se alignment is not provided by this method. [28] develops a vol-</div><div class="t m0 x18 h3 y3c ff2 fs1 fc0 sc0 ls0 ws0">umetric representation of 3D face and uses a network to regress it from a 2D</div><div class="t m0 x18 h3 y3d ff2 fs1 fc0 sc0 ls0 ws0">image. However, t hi s representation discards the semantic meaning of points,</div><div class="t m0 x18 h3 y3e ff2 fs1 fc0 sc0 ls0 ws0">thus the n etwork needs to regress the whole volume in order to restore the fa-</div><div class="t m0 x18 h3 y3f ff2 fs1 fc0 sc0 ls0 ws0">cial shape, which is only part of the volume. So this r e pr es entation limits the</div><div class="t m0 x18 h3 y40 ff2 fs1 fc0 sc0 ls0 ws0">resolution of the recovered shape, and need a complex network to regress it. To</div><div class="t m0 x18 h3 y41 ff2 fs1 fc0 sc0 ls0 ws0">sum up, model-based methods keep semantic meaning of points well but are re-</div><div class="t m0 x18 h3 y42 ff2 fs1 fc0 sc0 ls0 ws0">stricted in model space, recent model-free methods are unrestricted and achieve</div><div class="t m0 x18 h3 y43 ff2 fs1 fc0 sc0 ls0 ws0">state-of-the-art performance b u t discard the semantic meaning, which motivate</div><div class="t m0 x18 h3 y44 ff2 fs1 fc0 sc0 ls0 ws0">us to ﬁnd a new approach to reconstruct 3D face with alignment information in</div><div class="t m0 x18 h3 y45 ff2 fs1 fc0 sc0 ls0 ws0">a model-free manner.</div><div class="t m0 x9 h3 y29 ff2 fs1 fc0 sc0 ls0 ws0">In this paper, we propose an end-to-end method called Position map Regres-</div><div class="t m0 x18 h3 y46 ff2 fs1 fc0 sc0 ls0 ws0">sion Network (PRN) to jointly predict dense alignment and reconstruct 3D face</div><div class="t m0 x18 h3 y47 ff2 fs1 fc0 sc0 ls0 ws0">shape. Our method surpasses all other previous works on both 3D face alignment</div><div class="t m0 x18 h3 y2c ff2 fs1 fc0 sc0 ls0 ws0">and reconstruction on multiple datasets. Meanwhile, our method is straightfor-</div><div class="t m0 x18 h3 y48 ff2 fs1 fc0 sc0 ls0 ws0">ward with a very light-weighted model which provides the result in one pass with</div></div><div class="pi" data-data='{"ctm":[1.568627,0.000000,0.000000,1.568627,0.000000,0.000000]}'></div></div>

<div id="pf3" class="pf w0 h0" data-page-no="3"><div class="pc pc3 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/625cf355be9ad24cfa6f98d8/bg3.jpg"><div class="t m0 x19 h6 y2e ff7 fs4 fc0 sc0 ls0 ws0">Joint 3D Face Reconstruction and Dense Alignment 3</div><div class="t m0 x18 h3 y49 ff2 fs1 fc0 sc0 ls0 ws0">9.8ms. Al l of these are achieved by the elaborate design of the 2D representation</div><div class="t m0 x18 h3 y4a ff2 fs1 fc0 sc0 ls0 ws0">of 3D facial structure and the corresponding loss function. Speciﬁcally, we design</div><div class="t m0 x18 h3 y4b ff2 fs1 fc0 sc0 ls0 ws0">a UV position map, which is a 2D image recording the 3D coordinates of a com-</div><div class="t m0 x18 h3 y4c ff2 fs1 fc0 sc0 ls0 ws0">plete facial point cloud, and at t h e same time keeping the s emantic meaning at</div><div class="t m0 x18 h3 y4d ff2 fs1 fc0 sc0 ls0 ws0">each UV place. We then train a simple encoder-decoder network with a weighted</div><div class="t m0 x18 h3 y4e ff2 fs1 fc0 sc0 ls0 ws0">loss that focuses more on discriminative region to regress the UV position map</div><div class="t m0 x18 h3 y4f ff2 fs1 fc0 sc0 ls0 ws0">from a single 2D facial image. Figure 1 s hows our method is robust to poses,</div><div class="t m0 x18 h3 y50 ff2 fs1 fc0 sc0 ls0 ws0">illuminations and occlusions.</div><div class="t m0 x9 h3 y51 ff2 fs1 fc0 sc0 ls0 ws0">In summary, our main contributions are:</div><div class="t m0 x1a h3 y52 ffa fs1 fc0 sc0 ls0 ws0">– For the ﬁrst time, we solve the problems of face alignment and 3D face</div><div class="t m0 x1b h3 y53 ff2 fs1 fc0 sc0 ls0 ws0">reconstruction togethe r in an end-to-end fashion without the restriction of</div><div class="t m0 x1b h3 y54 ff2 fs1 fc0 sc0 ls0 ws0">low-dimensional solut i on s pac e.</div><div class="t m0 x1a h3 y55 ffa fs1 fc0 sc0 ls0 ws0">– To directly regress the 3D facial structure and dense alignment, we develop</div><div class="t m0 x1b h3 y56 ff2 fs1 fc0 sc0 ls0 ws0">a novel r ep r es entation called UV position map, which records the position</div><div class="t m0 x1b h3 y57 ff2 fs1 fc0 sc0 ls0 ws0">information of 3D face and provides dense correspondence to the semantic</div><div class="t m0 x1b h3 y58 ff2 fs1 fc0 sc0 ls0 ws0">meaning of each point on UV space.</div><div class="t m0 x1a h3 y59 ffa fs1 fc0 sc0 ls0 ws0">– For training, we proposed a weight mask whi ch assigns diﬀerent weight to</div><div class="t m0 x1b h3 y5a ff2 fs1 fc0 sc0 ls0 ws0">each point on position map and compute a weighted loss. We show that this</div><div class="t m0 x1b h3 y5b ff2 fs1 fc0 sc0 ls0 ws0">design helps improving the performance of our network.</div><div class="t m0 x1a h3 y5c ffa fs1 fc0 sc0 ls0 ws0">– We ﬁnally pr ovide a light-weighted framework th at runs at over 100FPS to</div><div class="t m0 x1b h3 y5d ff2 fs1 fc0 sc0 ls0 ws0">directly obtain 3D face reconstruction and alignment result from a single 2D</div><div class="t m0 x1b h3 y5e ff2 fs1 fc0 sc0 ls0 ws0">facial image.</div><div class="t m0 x1a h3 y5f ffa fs1 fc0 sc0 ls0 ws0">– Comparison on the AFLW2000-3D and Florence datasets shows that our</div><div class="t m0 x1b h3 y60 ff2 fs1 fc0 sc0 ls0 ws0">method achieves more than 25% relative improvements over other state-</div><div class="t m0 x1b h3 y61 ff2 fs1 fc0 sc0 ls0 ws0">of-the-art methods on both tasks of 3D face reconstruction and dense face</div><div class="t m0 x1b h3 y62 ff2 fs1 fc0 sc0 ls0 ws0">alignment.</div><div class="t m0 x18 h7 y63 ff1 fs5 fc0 sc0 ls0 ws0">2 Related Works</div><div class="t m0 x18 h3 y64 ffa fs1 fc0 sc0 ls0 ws0">2.1 3D Face Reconstruction</div><div class="t m0 x18 h3 y65 ff2 fs1 fc0 sc0 ls0 ws0">Since Blanz and Vetter proposed 3D Morphable Model(3DMM) in 1999[6], meth-</div><div class="t m0 x18 h3 y66 ff2 fs1 fc0 sc0 ls0 ws0">ods based on 3DMM are pop ul ar in completing the task of monocular 3D face</div><div class="t m0 x18 h3 y67 ff2 fs1 fc0 sc0 ls0 ws0">reconstruction. Most of earlier methods are to establish the correspondences of</div><div class="t m0 x18 h3 y68 ff2 fs1 fc0 sc0 ls0 ws0">the special points between input images and the 3D template including land-</div><div class="t m0 x18 h3 y69 ff2 fs1 fc0 sc0 ls0 ws0">marks[37, 68, 56, 27, 10, 29, 19] and local features[26, 49, 19], then solve the non-</div><div class="t m0 x18 h3 y6a ff2 fs1 fc0 sc0 ls0 ws0">linear optimization function to regress the 3DMM c oeﬃcients. However, these</div><div class="t m0 x18 h3 y6b ff2 fs1 fc0 sc0 ls0 ws0">methods heavily rely on the accuracy of landmarks or other feature points de-</div><div class="t m0 x18 h3 y6c ff2 fs1 fc0 sc0 ls0 ws0">tector. Thus, some methods[22, 63] ﬁrstly use CNNs to learn the dense cor-</div><div class="t m0 x18 h3 y6d ff2 fs1 fc0 sc0 ls0 ws0">respondence between input image and 3D template, then calculate the 3DMM</div><div class="t m0 x18 h3 y6e ff2 fs1 fc0 sc0 ls0 ws0">parameters with pr ed ic t ed dense constrains. Recent works also explore the usage</div><div class="t m0 x18 h3 y6f ff2 fs1 fc0 sc0 ls0 ws0">of CNN to predict 3DMM parameters directly. [32, 67, 47, 39, 48] use cascaded</div><div class="t m0 x18 h3 y70 ff2 fs1 fc0 sc0 ls0 ws0">CNN structure to regress the accur ate 3DMM coeﬃcients, which take a lot of</div><div class="t m0 x18 h3 y71 ff2 fs1 fc0 sc0 ls0 ws0">time due to i t erations. [15, 57, 31, 36] propose end-to- en d CNN architectures to</div><div class="t m0 x18 h3 y72 ff2 fs1 fc0 sc0 ls0 ws0">directly estimate the 3DMM shape parameters. Unsupervised methods have been</div></div><div class="pi" data-data='{"ctm":[1.568627,0.000000,0.000000,1.568627,0.000000,0.000000]}'></div></div>

<div id="pf4" class="pf w0 h0" data-page-no="4"><div class="pc pc4 w0 h0"><img class="bi x0 y0 w1 h1" alt="" src="https://static.pudn.com/prod/directory_preview_static/625cf355be9ad24cfa6f98d8/bg4.jpg"><div class="t m0 x18 h6 y2e ff7 fs4 fc0 sc0 ls0 ws0">4 Y. Feng et al.</div><div class="t m0 x18 h3 y49 ff2 fs1 fc0 sc0 ls0 ws0">also researched recently, [55, 3] can regress the 3DMM coeﬃcients without the</div><div class="t m0 x18 h3 y4a ff2 fs1 fc0 sc0 ls0 ws0">help of training data, which performs badly in faces with large poses and strong</div><div class="t m0 x18 h3 y4b ff2 fs1 fc0 sc0 ls0 ws0">occlusions. However, the main defect of those methods is model-based, resulting</div><div class="t m0 x18 h3 y4c ff2 fs1 fc0 sc0 ls0 ws0">in a limited geometry which is constrained in model space. Some other methods</div><div class="t m0 x18 h3 y4d ff2 fs1 fc0 sc0 ls0 ws0">can rec ons t ru ct 3D faces without 3D shape basis, [24, 33, 20, 53, 51] can produce</div><div class="t m0 x18 h3 y4e ff2 fs1 fc0 sc0 ls0 ws0">a 3D structure by warping the shape of a reference 3D model. [4] also reconstruct</div><div class="t m0 x18 h3 y4f ff2 fs1 fc0 sc0 ls0 ws0">the 3D shape of faces by learning a 3D Thin Plate Spline( TP S ) warping func-</div><div class="t m0 x18 h3 y50 ff2 fs1 fc0 sc0 ls0 ws0">tion vi a a deep network which warps a generic 3D model t o a subject speciﬁc 3D</div><div class="t m0 x18 h3 y51 ff2 fs1 fc0 sc0 ls0 ws0">shape. Obviously, the reconstructed face geometry from these methods are also</div><div class="t m0 x18 h3 y73 ff2 fs1 fc0 sc0 ls0 ws0">restricted by the ref er en ce model, which means the structure diﬀers when the</div><div class="t m0 x18 h3 y74 ff2 fs1 fc0 sc0 ls0 ws0">template changes. R ece ntly, [28] propose to straightforwardly map the image pix-</div><div class="t m0 x18 h3 y75 ff2 fs1 fc0 sc0 ls0 ws0">els to full 3D facial structure via volumetric CNN regression. This method is not</div><div class="t m0 x18 h3 y76 ff2 fs1 fc0 sc0 ls0 ws0">restricted in the m odel space any more, while needs a compl e x network structure</div><div class="t m0 x18 h3 y77 ff2 fs1 fc0 sc0 ls0 ws0">and a lot of time to predict the voxel data. Diﬀerent from above methods, Our</div><div class="t m0 x18 h3 y78 ff2 fs1 fc0 sc0 ls0 ws0">framework is model-free and light-weighted, can r u n at real t i me and directly</div><div class="t m0 x18 h3 y79 ff2 fs1 fc0 sc0 ls0 ws0">obtain the full 3D facial geometry along with its corr es pondence information.</div><div class="t m0 x18 h3 y7a ffa fs1 fc0 sc0 ls0 ws0">2.2 Face Alignment</div><div class="t m0 x18 h3 y7b ff2 fs1 fc0 sc0 ls0 ws0">In the ﬁeld of computer vision, face alignment is a long-standing prob l em which</div><div class="t m0 x18 h3 y7c ff2 fs1 fc0 sc0 ls0 ws0">attracts lots of attention. In the beginnin g, there are a number of 2D facial align-</div><div class="t m0 x18 h3 y7d ff2 fs1 fc0 sc0 ls0 ws0">ment approaches which aim at locating a set of ﬁducial 2D facial landmarks,</div><div class="t m0 x18 h3 y7e ff2 fs1 fc0 sc0 ls0 ws0">such as classic Active Appearance Model(AMM)[43, 52, 58] and Constrain ed Lo-</div><div class="t m0 x18 h3 y7f ff2 fs1 fc0 sc0 ls0 ws0">cal Models(CLM)[34, 1]. Then cascaded regression[14, 60] and CNN-based meth-</div><div class="t m0 x18 h3 y80 ff2 fs1 fc0 sc0 ls0 ws0">ods[38, 46, 9] are l ar gel y used to achieve state-of-the-art performance in 2D land-</div><div class="t m0 x18 h3 y81 ff2 fs1 fc0 sc0 ls0 ws0">marks location. However, 2D landmarks location only regresses visible points on</div><div class="t m0 x18 h3 y82 ff2 fs1 fc0 sc0 ls0 ws0">faces, which is limited to describe face shape when the pose is large. Recent works</div><div class="t m0 x18 h3 y83 ff2 fs1 fc0 sc0 ls0 ws0">then research the 3D facial alignment, whi ch begins with ﬁtting a 3DMM[44, 67,</div><div class="t m0 x18 h3 y84 ff2 fs1 fc0 sc0 ls0 ws0">18] or registering a 3D facial template[51, 5] w it h a 2D facial image. Obviously,</div><div class="t m0 x18 h3 y85 ff2 fs1 fc0 sc0 ls0 ws0">3D r econstruction methods based on model can easily complete the task of 3D</div><div class="t m0 x18 h3 y86 ff2 fs1 fc0 sc0 ls0 ws0">face alignment. Actually, [67, 63, 31] are specially designated methods to achieve</div><div class="t m0 x18 h3 y87 ff2 fs1 fc0 sc0 ls0 ws0">3D face alignment by means of 3DMM ﬁtting. Recently [8, 9] use a deep net-</div><div class="t m0 x18 h3 y88 ff2 fs1 fc0 sc0 ls0 ws0">work to directly predict the heat map to obtain the 3D facial landmarks and</div><div class="t m0 x18 h3 y89 ff2 fs1 fc0 sc0 ls0 ws0">achieves state-of-the-art performance. Thus, as sparse face alignment tasks are</div><div class="t m0 x18 h3 y8a ff2 fs1 fc0 sc0 ls0 ws0">highly completed by aforementioned met h ods, the task of dense face alignment</div><div class="t m0 x18 h3 y8b ff2 fs1 fc0 sc0 ls0 ws0">begins to develop. Notice that, the dense face alignment means the me t hods</div><div class="t m0 x18 h3 y8c ff2 fs1 fc0 sc0 ls0 ws0">should oﬀer the correspondence between two face images as well as between a</div><div class="t m0 x18 h3 y8d ff2 fs1 fc0 sc0 ls0 ws0">2D facial image and a 3D facial reference geometr y. [40] use multi-constraints to</div><div class="t m0 x18 h3 y8e ff2 fs1 fc0 sc0 ls0 ws0">train a CNN which estimates the 3DMM parameters and then provides a very</div><div class="t m0 x18 h3 y8f ff2 fs1 fc0 sc0 ls0 ws0">dense 3D alignment. [22, 63] directly learn th e correspondence between 2D input</div><div class="t m0 x18 h3 y90 ff2 fs1 fc0 sc0 ls0 ws0">image and 3D template via a deep network, while those correspondence is not</div><div class="t m0 x18 h3 y91 ff2 fs1 fc0 sc0 ls0 ws0">complete, onl y visible face region is con si d er ed . Compared to prior works, our</div><div class="t m0 x18 h3 y92 ff2 fs1 fc0 sc0 ls0 ws0">method can di r ec t ly establish the dense correspondence of all regions once the</div><div class="t m0 x18 h3 y93 ff2 fs1 fc0 sc0 ls0 ws0">position map is regressed. No intermediate parameters such as 3DMM coeﬃ-</div><div class="t m0 x18 h3 y94 ff2 fs1 fc0 sc0 ls0 ws0">cients and TPS warping parameters are needed in our method, which means our</div><div class="t m0 x18 h3 y95 ff2 fs1 fc0 sc0 ls0 ws0">network can run very fast.</div></div><div class="pi" data-data='{"ctm":[1.568627,0.000000,0.000000,1.568627,0.000000,0.000000]}'></div></div>