Skip to main content
SpringerLink
Log in

Model-Based Multiple Rigid Object Detection and Registration in Unstructured Range Data

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

We present a two-stage approach to the simultaneous detection and registration of multiple instances of industrial 3D objects in unstructured noisy range data. The first non-local processing stage takes all data into account and computes in parallel multiple localizations of the object along with rough pose estimates. The second stage computes accurate registrations for all detected object instances individually by using local optimization.

Both stages are designed using advanced numerical techniques, large-scale sparse convex programming, and second-order geometric optimization on the Euclidean manifold, respectively. They complement each other in that conflicting interpretations are resolved through non-local convex processing, followed by accurate non-convex local optimization based on sufficiently good initializations.

As input data a sparse point sample of the object’s surface is required exclusively. Our experiments focus on industrial applications where multiple 3D object instances are randomly assembled in a bin, occlude each other, and unstructured noisy range data is acquired by a laser scanning device.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adler, R. L., Dedieu, J.-P., Margulies, J. Y., Martens, M., & Shub, M. (2002). Newton’s method on Riemannian manifolds and a geometric model for the human spine. IMA Journal of Numerical Analysis, 22(3), 359–390.

    Article  MathSciNet  MATH  Google Scholar 

  • Ballard, D. H. (1981). Generalizing the Hough transform to detect arbitrary shapes. Pattern Recognition, 13, 111–122.

    Article  MATH  Google Scholar 

  • Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 509–522.

    Article  Google Scholar 

  • Benhimane, S., & Malis, E. (2006). A new approach to vision-based robot control with omni-directional cameras. In Proc. of IEEE int. conf. on robotics and automation (pp. 526–531).

    Google Scholar 

  • Bennett, K., & Parrado-Hernández, E. (2006). The interplay of optimization and machine learning research. Journal of Machine Learning Research, 7, 1265–1281.

    Google Scholar 

  • Bertsimas, D., & Weismantel, R. (2005). Optimization over Integers. Dynamic Ideas. ISBN-13: 978-0975914625

  • Besl, P. J., & Jain, R. C. (1985). Three-dimensional object recognition. ACM Computing Surveys, 17(1), 75–145.

    Article  Google Scholar 

  • Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 239–256.

    Article  Google Scholar 

  • Birgin, E. G., Martínez, J. M., & Raydan, M. (2000). Nonmonotone spectral projected gradient methods on convex sets. SIAM Journal of Optimization, 10, 1196–1211.

    Article  MATH  Google Scholar 

  • Boykov, Y., & Funka-Lea, G. (2006). Graph cuts and efficient n-d image segmentation. International Journal of Computer Vision, 70(2), 109–131.

    Article  Google Scholar 

  • Breitenreicher, D., & Schnörr, C. (2009). Intrinsic second-order geometric optimization for robust point set registration without correspondence. In 7th int. workshop on energy minimization methods in comp. vision and pattern recogn.

    Google Scholar 

  • Breitenreicher, D., & Schnörr, C. (2009). Robust 3D object registration without explicit correspondence using geometric integration. Machine Vision and Applications. doi:10.1007/s00138-009-0227-6

    Google Scholar 

  • Chan, T., Esedoglu, S., & Nikolova, M. (2006). Algorithms for finding global minimizers of image segmentation and denoising models. SIAM Journal on Applied Mathematics, 66(5), 1632–1648.

    Article  MathSciNet  MATH  Google Scholar 

  • Chen, S., Donoho, D., & Saunders, M. (2001). Atomic decomposition by basis pursuit. SIAM Review, 43(1), 129–159.

    Article  MathSciNet  MATH  Google Scholar 

  • Chin, R. T., & Dyer, C. R. (1986). Model-based recognition in robot vision. ACM Computing Surveys, 18(1), 67–108.

    Article  Google Scholar 

  • Chua, C. S., & Jarvis, R. (1997). Point signatures: A new representation for 3D object recognition. International Journal of Computer Vision, 25(1), 63–85.

    Article  Google Scholar 

  • Chui, H., & Rangarajan, A. (2000). A feature registration framework using mixture models. In IEEE workshop math. methods in biomed. image anal. (pp. 190–197).

    Google Scholar 

  • Cover, T., & Thomas, J. (1991). Elements of information theory. New York: Wiley.

    Book  MATH  Google Scholar 

  • do Carmo, M. P. (1992). Riemannian geometry. Cambridge: Birkhäuser Boston.

    MATH  Google Scholar 

  • Donoho, D. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52, 1289–1306.

    Article  MathSciNet  Google Scholar 

  • Donoho, D. L., Elad, M., & Temlyakov, V. N. (2006). Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transactions on Information Theory, 52, 6–18.

    Article  MathSciNet  Google Scholar 

  • Drummond, T., & Cipolla, R. (2002). Real-time tracking of complex structures with on-line camera calibration. Image Vision Computing, 20(5–6), 427–433.

    Article  Google Scholar 

  • Edelman, A., Arias, T. A., & Smith, S. T. (1999). The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications, 20, 303–353.

    Article  MathSciNet  Google Scholar 

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Fitzgibbon, A. W. (2003). Robust registration of 2D and 3D point sets. Image Vision Computing, 21(13–14), 1145–1153.

    Article  Google Scholar 

  • Frome, A., Huber, D., Kolluri, R., Bulow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In Proc. Europ. conf. comp. vision.

    Google Scholar 

  • Gelfand, N., Mitra, N. J., Guibas, L. J., & Pottmann, H. (2005). Robust global registration. In Proc. symp. geom. processing.

    Google Scholar 

  • Golub, G., & Van Loan, C. (1996). Matrix computations (3rd edn.). Baltimore: The John Hopkins University Press.

    MATH  Google Scholar 

  • Greenspan, M. (2002). Geometric probing of dense range data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 495–508.

    Article  Google Scholar 

  • Hartley, R. I., & Kahl, F. (2009). Global optimization through rotation space search. International Journal of Computer Vision, 82(1), 64–79.

    Article  Google Scholar 

  • Jian, B., & Vemuri, B. C. (2005). A robust algorithm for point set registration using mixture of Gaussians. In Proc. int. conf. comp. vision.

    Google Scholar 

  • Johnson, A., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 433–449.

    Article  Google Scholar 

  • Krishnan, S., Lee, P. Y., Moore, J. B., & Venkatasubramanian, S. (2007). Optimisation-on-a-manifold for global registration of multiple 3D point sets. International Journal of Intelligent Systems Technologies and Applications, 3(3/4), 319–340.

    Article  Google Scholar 

  • Lavva, I., Hameiri, E., & Shimshoni, I. (2008). Robust methods for geometric primitive recovery and estimation from range images. IEEE Transactions on Systems, Man and Cybernetics, Part B, Cybernetics, 38(3), 826–845.

    Article  Google Scholar 

  • Li, H., & Hartley, R. (2007). The 3D-3D registration problem revisited. In Proc. int. conf. comp. vision.

    Google Scholar 

  • Matsushima, Y. (1972). Differentiable manifolds. New York: Dekker.

    MATH  Google Scholar 

  • Mitra, N. J., Gelfand, N., Pottmann, H., & Guibas, L. (2004). Registration of point cloud data from a geometric optimization perspective. In Proc. sym. geom. process.

    Google Scholar 

  • Natarajan, K. (1995). Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24, 227–234.

    Article  MathSciNet  MATH  Google Scholar 

  • Nesterov, Y. (2005). Smooth minimization of non-smooth functions. Mathematical Programming, 103(1), 127–152.

    Article  MathSciNet  MATH  Google Scholar 

  • Olson, C. (1997). Efficient pose clustering using a randomized algorithm. International Journal of Computer Vision, 23(2), 131–147.

    Article  Google Scholar 

  • Olsson, C., Kahl, F., & Oskarsson, M. (2009). Branch-and-bound methods for Euclidean registration problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(5), 783–794.

    Article  Google Scholar 

  • Pottmann, H., Huang, Q.-X., Yang, Y.-L., & Hu, S.-M. (2006). Geometry and convergence analysis of algorithms for registration of 3D shapes. International Journal of Computer Vision, 67(3), 277–296.

    Article  Google Scholar 

  • Rangarajan, A., Chui, H., & Bookstein, F. L. (1997). The softassign procrustes matching algorithm. In Proc. int. conf. inf. process. med. imaging.

    Google Scholar 

  • Reyes, L., Medioni, G., & Bayro, E. (2007). Registration of 3D points using geometric algebra and tensor voting. International Journal of Computer Vision, 75(3), 351–369.

    Article  Google Scholar 

  • Rockafellar, R., & Wets, R.-B. (1998). Grundlehren der math. Wissenschaften : Vol. 317. Variational analysis. Berlin: Springer.

    Book  MATH  Google Scholar 

  • Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In Proc. 3rd int. conf. on 3D digital imaging and modeling (pp. 145–152).

    Chapter  Google Scholar 

  • Salvi, J., Matabosch, C., Fofi, D., & Forest, J. (2007). A review of recent range image registration methods with accuracy evaluation. Image and Vision Computing, 25, 578–596.

    Article  Google Scholar 

  • Shang, L., & Greenspan, M. (2007). Pose determination by potential well space embedding. In Proc. 6th int. conf. 3-D digital imaging and modeling (pp. 297–304). Los Alamitos: IEEE Comput. Soc..

    Google Scholar 

  • Shi, Q., Xi, N., Chen, Y., & Sheng, W. (2006). Registration of point clouds for 3D shape inspection. In Int. conf. intell. robots syst.

    Google Scholar 

  • Subbarao, R., & Meer, P. (2009). Nonlinear mean shift over Riemannian manifolds. International Journal of Computer Vision, 84, 1–20.

    Article  Google Scholar 

  • Taylor, C. J., & Kriegman, D. J. (1994). Minimization on the Lie group SO(3) and related manifolds. Technical Report 9405, Center for Systems Science, Dept. of Electrical Engineering, Yale University.

  • Teboulle, M. (2007). A unified continuous optimization framework for center-based clustering methods. The Journal of Machine Learning Research, 8, 65–102.

    MathSciNet  Google Scholar 

  • Tropp, J. A. (2006). Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Transactions on Information Theory, 52, 1030–1051.

    Article  MathSciNet  Google Scholar 

  • Tsin, Y., & Kanade, T. (2004). A correlation-based approach to robust point set registration. In Proc. Europ. conf. comp. vision (Vol. III, pp. 558–569).

    Google Scholar 

  • Wainwright, M., & Jordan, M. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305.

    MATH  Google Scholar 

  • Wang, F., Vemuri, B. C., Rangarajan, A., Schmalfuss, I. M., & Eisenschenk, S. J. (2006). Simultaneous nonrigid registration of multiple point sets and atlas construction. In Proc. Europ. conf. comp. vision.

    Google Scholar 

  • Wiedemann, C., Ulrich, M., & Steger, C. (2008). Recognition and tracking of 3D objects. In Pattern recogn.

    Google Scholar 

  • Wolfson, H. J., & Rigoutsos, I. (1997). Geometric hashing: An overview. Computing in Science and Engineering, 4, 10–21.

    Google Scholar 

  • Zhu, L., Barhak, J., Shrivatsan, V., & Katz, R. (2007). Efficient registration for precision inspection of free-form surfaces. International Journal of Advanced Manufacturing Technology, 32, 505–515.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Image & Pattern Analysis Group (IPA), Heidelberg Collaboratory for Image Processing (HCI), University of Heidelberg, Speyerer Strasse 6, 69115, Heidelberg, Germany

    Dirk Breitenreicher & Christoph Schnörr

Authors
  1. Dirk Breitenreicher
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christoph Schnörr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dirk Breitenreicher.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Breitenreicher, D., Schnörr, C. Model-Based Multiple Rigid Object Detection and Registration in Unstructured Range Data. Int J Comput Vis 92, 32–52 (2011). https://doi.org/10.1007/s11263-010-0401-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-010-0401-3

Keywords

Search

Navigation