Abstract
This paper addresses the recovery of face models from stereo pairs of images in the presence of foreign-body occlusions. In the proposed approach, a 3D morphable model (3DMM) for faces is augmented by an occlusion map defined on the model shape, and occlusion is detected with minimal computational overhead by incorporating robust estimators in the fitting process. Additionally, the method uses an explicit model for texture (or reflectance) in addition to shape, which is in contrast to most existing multi-view methods that use a shape model alone. We argue that both model components are required to handle certain classes of occluders, and we present empirical results to support this claim. In fact, the empirical results in this paper suggest that even in the absence of occlusions, stereo reconstruction using existing shape-only face models can perform poorly by some measures, and that the inclusion of an explicit texture model may be worth its computational expense.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Georghiades, A., Kriegman, D., Belhumeur, P.: Illumination cones for recognition under variable lighting: Faces. In: CVPR, pp. 52–59 (1998)
Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE TPAMI 19(7), 711–720 (1997)
Basri, R., Jacobs, D.: Lambertian reflectance and linear subspaces. IEEE TPAMI 25(2), 218–233 (2003)
Lee, K.C., Ho, J., Kriegman, D.J.: Nine points of light: acquiring subspaces for face recognition under variable lighting. CVPR 1, 519–526
Fransens, R., Strecha, C., Van Gool, L.: Parametric Stereo for Multi-Pose Face Recognition and 3D-Face Modeling. In: Zhao, W., Gong, S., Tang, X. (eds.) AMFG 2005. LNCS, vol. 3723, pp. 109–124. Springer, Heidelberg (2005)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Proceedings of ACM SIGGRAPH, pp. 187–194. ACM Press, New York (1999)
Blanz, V., Vetter, T.: Face recognition based on fitting a 3D morphable model. IEEE TPAMI 25(9), 1063–1074 (2003)
Blanz, V., Grother, P., Phillips, P., Vetter, T.: Face recognition based on frontal views generated from non-frontal images. In: CVPR 2 (2005)
De Smet, M., Fransens, R., Van Gool, L., Esat-Psi, K.: A Generalized EM Approach for 3D Model Based Face Recognition under Occlusions. CVPR 2, 1423–1430 (2006)
Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE TPAMI 23(6), 681–685 (2001)
Baker, S., Matthews, I.: Lucas-Kanade 20 Years On: A Unifying Framework. IJCV 56(3), 221–255 (2004)
Xiao, J., Baker, S., Matthews, I., Kanade, T.: Real-time combined 2D+ 3D active appearance models. CVPR 2, 535–542 (2004)
Hu, C., Xiao, J., Matthews, I., Baker, S., Cohn, J., Kanade, T.: Fitting a single active appearance model simultaneously to multiple images. In: Proc. British Machine Vision Conference (2004)
Koterba, S.C., Baker, S., Matthews, I., Hu, C., Xiao, J., Cohn, J., Kanade, T.: Multi-View AAM Fitting and Camera Calibration. In: ICCV, vol. 1, pp. 511–518 (2005)
Gross, R., Matthews, I., Baker, S.: Active Appearance Models with Occlusion. Image and Vision Computing 24(6), 593–604 (2006)
Basso, C., Vetter, T., Blanz, V.: Regularized 3D morphable models. In: IEEE Int. Workshop on Higher-Level Knowledge in 3D Modeling and Motion Analysis, pp. 3–10. IEEE Computer Society Press, Los Alamitos (2003)
Blanz, V., Basso, C., Poggio, T., Vetter, T.: Reanimating Faces in Images and Video. Computer Graphics Forum 22(3), 641–650 (2003)
Basso, C., Paysan, P., Vetter, T.: Registration of Expressions Data using a 3D Morphable Model. FGR, 205–210 (2006)
Dimitrijevic, M., Ilic, S., Fua, P.: Accurate face models from uncalibrated and ill-lit video sequences. CVPR 2 (2004)
Romdhani, S., Vetter, T.: Estimating 3D Shape and Texture Using Pixel Intensity, Edges, Specular Highlights, Texture Constraints and a Prior. CVPR (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Romeiro, F., Zickler, T. (2007). Model-Based Stereo with Occlusions. In: Zhou, S.K., Zhao, W., Tang, X., Gong, S. (eds) Analysis and Modeling of Faces and Gestures. AMFG 2007. Lecture Notes in Computer Science, vol 4778. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75690-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-75690-3_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75689-7
Online ISBN: 978-3-540-75690-3
eBook Packages: Computer ScienceComputer Science (R0)