Skip to main content
SpringerLink
Log in

Quasi-perspective Projection Model: Theory and Application to Structure and Motion Factorization from Uncalibrated Image Sequences

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper addresses the problem of factorization-based 3D reconstruction from uncalibrated image sequences. Previous studies on structure and motion factorization are either based on simplified affine assumption or general perspective projection. The affine approximation is widely adopted due to its simplicity, whereas the extension to perspective model suffers from recovering projective depths. To fill the gap between simplicity of affine and accuracy of perspective model, we propose a quasi-perspective projection model for structure and motion recovery of rigid and nonrigid objects based on factorization framework. The novelty and contribution of this paper are as follows. Firstly, under the assumption that the camera is far away from the object with small lateral rotations, we prove that the imaging process can be modeled by quasi-perspective projection, which is more accurate than affine model from both geometrical error analysis and experimental studies. Secondly, we apply the model to establish a framework of rigid and nonrigid factorization under quasi-perspective assumption. Finally, we propose an Extended Cholesky Decomposition to recover the rotation part of the Euclidean upgrading matrix. We also prove that the last column of the upgrading matrix corresponds to a global scale and translation of the camera thus may be set freely. The proposed method is validated and evaluated extensively on synthetic and real image sequences and improved results over existing schemes are observed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bascle, B., & Blake, A. (1998). Separability of pose and expression in facial tracing and animation. In Proceedings of the international conference on computer vision (pp. 323–328) 1998.

  • Brand, M. (2001). Morphable 3D models from video. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 456–463) 2001.

  • Brand, M. (2005). A direct method for 3D factorization of nonrigid motion observed in 2d. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 122–128) 2005.

  • Bregler, C., Hertzmann, A., & Biermann, H. (2000). Recovering non-rigid 3D shape from image streams. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 690–696) 2000.

  • Buchanan, A. M., & Fitzgibbon, A. W. (2005). Damped newton algorithms for matrix factorization with missing data. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 316–322) 2005.

  • Chen, P. (2008). Optimization algorithms on subspaces: revisiting missing data problem in low-rank matrix. International Journal of Computer Vision, 80(1), 125–142.

    Article  Google Scholar 

  • Christy, S., & Horaud, R. (1996). Euclidean shape and motion from multiple perspective views by affine iterations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(11), 1098–1104.

    Article  Google Scholar 

  • Costeira, J., & Kanade, T. (1998). A multibody factorization method for independent moving objects. International Journal of Computer Vision, 29(3), 159–179.

    Article  Google Scholar 

  • Del Bue, A., Smeraldi, F., & de Agapito, L. (2004). Non-rigid structure from motion using nonparametric tracking and non-linear optimization. In IEEE workshop in articulated and nonrigid motion ANM04, held in conjunction with CVPR2004 (pp. 8–15), June 2004.

  • Del Bue, A., Lladó, X., & de Agapito, L. (2006). Non-rigid metric shape and motion recovery from uncalibrated images using priors. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 1191–1198) 2006.

  • Han, M., & Kanade, T. (2000). Creating 3D models with uncalibrated cameras. In Proceedings of IEEE computer society workshop on the application of computer vision (WACV2000), December 2000.

  • Hartley, R. (1997). Kruppa’s equations derived from the fundamental matrix. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(2), 133–135.

    Article  MathSciNet  Google Scholar 

  • Hartley, R., & Schaffalizky, F. (2003). Powerfactorization: 3D reconstruction with missing or uncertain data. In Australia-Japan advanced workshop on computer vision, 2003.

  • Hartley, R., & Vidal, R. (2008). Perspective nonrigid shape and motion recovery. In ECCV (1), Lecture notes in computer science : Vol. 5302 (pp. 276–289). Berlin: Springer.

    Google Scholar 

  • Hartley, R., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd edn.). Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Heyden, A., & Åström, K. (1997) Euclidean reconstruction from image sequences with varying and unknown focal length and principal point. In IEEE conference on computer vision and pattern recognition (pp. 438–443) 1997.

  • Heyden, A., Berthilsson, R., & Sparr, G. (1999). An iterative factorization method for projective structure and motion from image sequences. Image and Vision Computing, 17(13), 981–991.

    Article  Google Scholar 

  • Li, T., Kallem, V., Singaraju, D., & Vidal, R. (2007). Projective factorization of multiple rigid-body motions. In IEEE conference on computer vision and pattern recognition, 2007.

  • Luong, Q., & Faugeras, O. (1997). Self-calibration of a moving camera from point correspondences and fundamental matrices. International Journal of Computer Vision, 22(3), 261–289.

    Article  Google Scholar 

  • Mahamud, S., & Hebert, M. (2000). Iterative projective reconstruction from multiple views. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 430–437) 2000.

  • Maybank, S., & Faugeras, O. (1992). A theory of self-calibration of a moving camera. International Journal of Computer Vision, 8(2), 123–151.

    Article  Google Scholar 

  • Oliensis, J., & Hartley, R. (2007). Iterative extensions of the Sturm/Triggs algorithm: convergence and nonconvergence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(12), 2217–2233.

    Article  Google Scholar 

  • Poelman, C., & Kanade, T. (1997). A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern and Analysis and Machine Intelligence, 19(3), 206–218.

    Article  Google Scholar 

  • Pollefeys, M., Koch, R., & Van Gool, L. (1999). Self-calibration and metric reconstruction in spite of varying and unknown intrinsic camera parameters. International Journal of Computer Vision, 32(1), 7–25.

    Article  Google Scholar 

  • Quan, L. (1996). Self-calibration of an affine camera from multiple views. International Journal of Computer Vision, 19(1), 93–105.

    Article  Google Scholar 

  • Rabaud, V., & Belongie, S. (2008). Re-thinking non-rigid structure from motion. In IEEE conference on computer vision and pattern recognition, 2008.

  • Sturm, P. F., & Triggs, B. (1996). A factorization based algorithm for multi-image projective structure and motion. In European conference on computer vision (2) (pp. 709–720) 1996.

  • Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9(2), 137–154.

    Article  Google Scholar 

  • Torr, P. H. S., Zisserman, A., & Maybank, S. J. (1998). Robust detection of degenerate configurations while estimating the fundamental matrix. Computer Vision and Image Understanding, 71(3), 312–333.

    Article  Google Scholar 

  • Torresani, L., Yang, D. B., Alexander, E. J., & Bregler, C. (2001). Tracking and modeling non-rigid objects with rank constraints. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 493–500) 2001.

  • Torresani, L., Hertzmann, A., & Bregler, C. (2008). Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5), 878–892.

    Article  Google Scholar 

  • Triggs, B. (1996). Factorization methods for projective structure and motion. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 845–851). San Francisco, California, USA, 1996.

  • Vidal, R., & Abretske, D. (2006). Nonrigid shape and motion from multiple perspective views. In European conference on computer vision (2). Lecture notes in computer science : Vol. 3952 (pp. 205–218). Berlin: Springer.

    Google Scholar 

  • Vidal, R., Tron, R., & Hartley, R. (2008). Multiframe motion segmentation with missing data using powerfactorization and GPCA. International Journal of Computer Vision, 79(1), 85–105.

    Article  Google Scholar 

  • Wang, G. (2006). A hybrid system for feature matching based on SIFT and epipolar constraints. (Tech. Rep.). Department of Electrical and Computer Engineering, University of Windsor.

  • Wang, G., Tsui, H.-T., & Wu, J. (2008). Rotation constrained power factorization for structure from motion of nonrigid objects. Pattern Recognition Letters, 29(1), 72–80.

    Article  Google Scholar 

  • Wang, G., & Wu, Q. J. (2008a). Stratification approach for 3D euclidean reconstruction of nonrigid objects from uncalibrated image sequences. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 38(1), 90–101.

    Article  Google Scholar 

  • Wang, G., & Wu, J. (2008b). Quasi-perspective projection with applications to 3D factorization from uncalibrated image sequences. In IEEE conference on computer vision and pattern recognition, 2008.

  • Wang, G., Wu, J., & Zhang, W. (2008). Camera self-calibration and three dimensional reconstruction under quasi-perspective projection. In Proceedings Canadian conference on computer and robot vision (pp. 129–136) 2008.

  • Xiao, J., & Kanade, T. (2005). Uncalibrated perspective reconstruction of deformable structures. In Proceedings of the international conference on computer vision (Vol. 2, pp. 1075–1082) 2005.

  • Xiao, J., Chai, J., & Kanade, T. (2006). A closed-form solution to non-rigid shape and motion recovery. International Journal of Computer Vision, 67(2), 233–246.

    Article  Google Scholar 

  • Yan, J., & Pollefeys, M. (2005). A factorization-based approach to articulated motion recovery. In IEEE conference on computer vision and pattern recognition (2) (pp. 815–821) 2005.

  • Yan, J., & Pollefeys, M. (2008). A factorization-based approach for articulated nonrigid shape, motion and kinematic chain recovery from video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(5), 865–877.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Windsor, 401 Sunset, Windsor, N9B 3P4, Ontario, Canada

    Guanghui Wang & Q. M. Jonathan Wu

  2. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100080, China

    Guanghui Wang

Authors
  1. Guanghui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Q. M. Jonathan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanghui Wang.

Additional information

The work is supported in part by Natural Sciences and Engineering Research Council of Canada, and the National Natural Science Foundation of China under Grant No. 60575015.

Electronic Supplementary Material

Below is the link to the electronic supplementary material. (AVI 4.79 MB)

Below is the link to the electronic supplementary material. (AVI 4.42 MB)

Below is the link to the electronic supplementary material. (AVI 4.73 MB)

Below is the link to the electronic supplementary material. (AVI 4.70 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, G., Wu, Q.M.J. Quasi-perspective Projection Model: Theory and Application to Structure and Motion Factorization from Uncalibrated Image Sequences. Int J Comput Vis 87, 213–234 (2010). https://doi.org/10.1007/s11263-009-0267-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0267-4

Keywords

Search

Navigation