Skip to main content
SpringerLink
Log in

Fast 3D Scene Segmentation and Partial Object Retrieval Using Local Geometric Surface Features

  • Chapter
  • First Online:
Transactions on Computational Science XXXVI

Part of the book series: Lecture Notes in Computer Science ((TCOMPUTATSCIE,volume 12060))

Abstract

Robotic vision and in particular 3D understanding has attracted intense research efforts the last few years due to its wide range of applications, such as robot-human interaction, augmented and virtual reality etc, and the introduction of low-cost 3D sensing devices. In this paper we explore one of the most popular problems encountered in 3D perception applications, namely the segmentation of a 3D scene and the retrieval of similar objects from a model database. We use a geometric approach for both the segmentation and the retrieval modules that enables us to develop a fast, low-memory footprint system without the use of large-scale annotated datasets. The system is based on the fast computation of surface normals and the encoding power of local geometric features. Our experiments demonstrate that such a complete 3D understanding framework is possible and advantages over other approaches as well as weaknesses are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Our source code will be available on Github upon publication.

References

  1. https://www.acin.tuwien.ac.at/en/vision-for-robotics/software-tools/osd/

  2. http://hampson.cast.uark.edu/

  3. Bentley, J.L.: Multidimensional binary search trees used for associative searching. Commun. ACM 18(9), 509–517 (1975). https://doi.org/10.1145/361002.361007

    Article  MathSciNet  MATH  Google Scholar 

  4. Boulch, A., Saux, B.L., Audebert, N.: Unstructured point cloud semantic labeling using deep segmentation networks. In: Pratikakis, I., Dupont, F., Ovsjanikov, M. (eds.) Eurographics Workshop on 3D Object Retrieval. The Eurographics Association (2017). https://doi.org/10.2312/3dor.20171047

  5. Bronstein, M.M., Kokkinos, I.: Scale-invariant heat kernel signatures for non-rigid shape recognition. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1704–1711, June 2010. https://doi.org/10.1109/CVPR.2010.5539838

  6. Dimou, D., Moustakas, K.: A framework for 3D object segmentation and retrieval using local geometric surface features. In: International Conference CyberWorlds, October 2018

    Google Scholar 

  7. Ecins, A., Fermüller, C., Aloimonos, Y.: Cluttered scene segmentation using the symmetry constraint. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2271–2278, May 2016. https://doi.org/10.1109/ICRA.2016.7487376

  8. Engelmann, F., Kontogianni, T., Hermans, A., Leibe, B.: Exploring spatial context for 3D semantic segmentation of point clouds. CoRR abs/1802.01500 (2018). http://arxiv.org/abs/1802.01500

  9. Firman, M.: RGBD datasets: past, present and future. CoRR abs/1604.00999 (2016). http://arxiv.org/abs/1604.00999

  10. Fleishman, S., Drori, I., Cohen-Or, D.: Bilateral mesh denoising. ACM Trans. Graph. 22(3), 950–953 (2003). https://doi.org/10.1145/882262.882368

    Article  Google Scholar 

  11. Grard, M., Brégier, R., Sella, F., Dellandréa, E., Chen, L.: Object segmentation in depth maps with one user click and a synthetically trained fully convolutional network. CoRR abs/1801.01281 (2018). http://arxiv.org/abs/1801.01281

  12. Hackel, T., Wegner, J., Schindler, K.: Fast semantic segmentation of 3D point clouds with strongly varying density. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. III–3, 177–184 (2016)

    Article  Google Scholar 

  13. He, X., Zhou, Y., Zhou, Z., Bai, S., Bai, X.: Triplet-center loss for multi-view 3D object retrieval (2018)

    Google Scholar 

  14. Holz, D., Behnke, S.: Approximate triangulation and region growing for efficient segmentation and smoothing of range images. Robot. Auton. Syst. 62(9), 1282–1293 (2014). https://doi.org/10.1016/j.robot.2014.03.013

    Article  Google Scholar 

  15. Huang, J., You, S.: Point cloud labeling using 3D convolutional neural network. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2670–2675, December 2016. https://doi.org/10.1109/ICPR.2016.7900038

  16. Kim, E., Medioni, G.: 3D object recognition in range images using visibility context. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3800–3807, September 2011. https://doi.org/10.1109/IROS.2011.6094527

  17. Moustakas, K., Stavropoulos, G., Tzovaras, D.: Protrusion fields for 3D model search and retrieval based on range image queries. In: Bebis, G., et al. (eds.) ISVC 2012. LNCS, vol. 7431, pp. 610–619. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33179-4_58

    Chapter  Google Scholar 

  18. Newcombe, R.A.: KinectFusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 127–136, October 2011. https://doi.org/10.1109/ISMAR.2011.6092378

  19. Papadakis, P., Pratikakis, I., Theoharis, T., Perantonis, S.: PANORAMA: a 3D shape descriptor based on panoramic views for unsupervised 3D object retrieval. Int. J. Comput. Vis. 89(2), 177–192 (2010). https://doi.org/10.1007/s11263-009-0281-6

    Article  Google Scholar 

  20. Pratikakis, I., et al.: SHREC’16 track: partial shape queries for 3D object retrieval (2016)

    Google Scholar 

  21. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. CoRR abs/1612.00593 (2016). http://arxiv.org/abs/1612.00593

  22. Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (FPFH) for 3D registration. In: 2009 IEEE International Conference on Robotics and Automation, pp. 3212–3217, May 2009. https://doi.org/10.1109/ROBOT.2009.5152473

  23. Rusu, R., Marton, Z., Blodow, N., Beetz, M.: Persistent point feature histograms for 3D point clouds, vol. 16, January 2008

    Google Scholar 

  24. Rusu, R.B., Marton, Z.C., Blodow, N., Dolha, M., Beetz, M.: Towards 3D point cloud based object maps for household environments. Robot. Auton. Syst. 56(11), 927–941 (2008). https://doi.org/10.1016/j.robot.2008.08.005

    Article  Google Scholar 

  25. Savelonas, M.A., Pratikakis, I., Sfikas, K.: An overview of partial 3D object retrieval methodologies. Multimedia Tools Appl. 74(24), 11783–11808 (2014). https://doi.org/10.1007/s11042-014-2267-9

    Article  Google Scholar 

  26. Savelonas, M.A., Pratikakis, I., Sfikas, K.: Fisher encoding of differential fast point feature histograms for partial 3D object retrieval. Pattern Recogn. 55, 114–124 (2016). https://doi.org/10.1016/j.patcog.2016.02.003. http://www.sciencedirect.com/science/article/pii/S0031320316000595

    Article  Google Scholar 

  27. Sfikas, K., Pratikakis, I., Koutsoudis, A., Savelonas, M., Theoharis, T.: Partial matching of 3D cultural heritage objects using panoramic views. Multimedia Tools Appl. 75(7), 3693–3707 (2014). https://doi.org/10.1007/s11042-014-2069-0

    Article  Google Scholar 

  28. Sfikas, K., Theoharis, T., Pratikakis, I.: Exploiting the PANORAMA representation for convolutional neural network classification and retrieval, April 2017

    Google Scholar 

  29. Shi, B., Bai, S., Zhou, Z., Bai, X.: DeepPano: deep panoramic representation for 3-D shape recognition. IEEE Signal Process. Lett. 22, 2339–2343 (2015)

    Article  Google Scholar 

  30. Sinha, A., Bai, J., Ramani, K.: Deep learning 3D shape surfaces using geometry images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 223–240. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_14

    Chapter  Google Scholar 

  31. Stavropoulos, G., Moschonas, P., Moustakas, K., Tzovaras, D., Strintzis, M.G.: 3-D model search and retrieval from range images using salient features. IEEE Trans. Multimedia 12(7), 692–704 (2010). https://doi.org/10.1109/TMM.2010.2053023

    Article  Google Scholar 

  32. Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.G.: Multi-view convolutional neural networks for 3D shape recognition. CoRR abs/1505.00880 (2015). http://arxiv.org/abs/1505.00880

  33. Sun, J., Ovsjanikov, M., Guibas, L.: A concise and provably informative multi-scale signature based on heat diffusion. In: Proceedings of the Symposium on Geometry Processing, SGP 2009, pp. 1383–1392. Eurographics Association, Aire-la-Ville (2009). http://dl.acm.org/citation.cfm?id=1735603.1735621

  34. Tateno, K., Tombari, F., Navab, N.: Real-time and scalable incremental segmentation on dense slam. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4465–4472, September 2015. https://doi.org/10.1109/IROS.2015.7354011

  35. Vosselman, G.: Point cloud segmentation for urban scene classification. Int. Soc. Photogramm. Remote Sens. (ISPRS) 1, 257–262 (2013). https://doi.org/10.5194/isprsarchives-XL-7-W2-257-2013

    Article  Google Scholar 

  36. Wang, W., Yu, R., Huang, Q., Neumann, U.: SGPN: similarity group proposal network for 3D point cloud instance segmentation. CoRR abs/1711.08588 (2017). http://arxiv.org/abs/1711.08588

  37. Wang, Y., Shi, T., Yun, P., Tai, L., Liu, M.: PointSeg: real-time semantic segmentation based on 3D LiDAR point cloud. ArXiv e-prints, July 2018

    Google Scholar 

  38. Whelan, T., Kaess, M., Johannsson, H., Fallon, M., Leonard, J.J., McDonald, J.: Real-time large-scale dense RGB-D SLAM with volumetric fusion. Int. J. Robot. Res. 34(4–5), 598–626 (2015). https://doi.org/10.1177/0278364914551008

    Article  Google Scholar 

  39. Xiong, X., Munoz, D., Bagnell, J.A., Hebert, M.: 3-D scene analysis via sequenced predictions over points and regions. In: 2011 IEEE International Conference on Robotics and Automation, pp. 2609–2616, May 2011. https://doi.org/10.1109/ICRA.2011.5980125

  40. Yücer, K., Sorkine-Hornung, A., Wang, O., Sorkine-Hornung, O.: Efficient 3D object segmentation from densely sampled light fields with applications to 3D reconstruction. ACM Trans. Graph. 35(3), 22:1–22:15 (2016). https://doi.org/10.1145/2876504

    Article  Google Scholar 

  41. Ückermann, A., Haschke, R., Ritter, H.: Real-time 3D segmentation of cluttered scenes for robot grasping. In: 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pp. 198–203 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Systems and Robotics, Instituto Superior Tecnico, Lisbon, Portugal

    Dimitrios Dimou

  2. Department of Electrical and Computer Engineering, University of Patras, Patras, Greece

    Konstantinos Moustakas

Authors
  1. Dimitrios Dimou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantinos Moustakas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dimitrios Dimou .

Editor information

Editors and Affiliations

  1. University of Calgary, Calgary, AB, Canada

    Marina L. Gavrilova

  2. Sardina Systems OÜ, Tallinn, Estonia

    C.J. Kenneth Tan

  3. Nanyang Technological University, Singapore, Singapore

    Alexei Sourin

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer-Verlag GmbH Germany, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Dimou, D., Moustakas, K. (2020). Fast 3D Scene Segmentation and Partial Object Retrieval Using Local Geometric Surface Features. In: Gavrilova, M., Tan, C., Sourin, A. (eds) Transactions on Computational Science XXXVI. Lecture Notes in Computer Science(), vol 12060. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-61364-1_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-61364-1_5

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-61363-4

  • Online ISBN: 978-3-662-61364-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation