Skip to main content
SpringerLink
Log in

Extended Patch Prioritization for Depth Filling Within Constrained Exemplar-Based RGB-D Image Completion

  • Conference paper
  • First Online:
Image Analysis and Recognition (ICIAR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10882))

Included in the following conference series:

Abstract

We address the problem of hole filling in depth images, obtained from either active or stereo sensing, for the purposes of depth image completion in an exemplar-based framework. Most existing exemplar-based inpainting techniques, designed for color image completion, do not perform well on depth information with object boundaries obstructed or surrounded by missing regions. In the proposed method, using both color (RGB) and depth (D) information available from a common-place RGB-D image, we explicitly modify the patch prioritization term utilized for target patch ordering to facilitate improved propagation of complex texture and linear structures within depth completion. Furthermore, the query space in the source region is constrained to increase the efficiency of the approach compared to other exemplar-driven methods. Evaluations demonstrate the efficacy of the proposed method compared to other contemporary completion techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  2. Arias, P., Facciolo, G., Caselles, V., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vis. 93(3), 319–347 (2011)

    Article  MathSciNet  Google Scholar 

  3. Telea, A.: An image inpainting technique based on the fast marching method. Graph. Tools 9(1), 23–24 (2004)

    Article  Google Scholar 

  4. Zhang, L., Shen, P., Zhang, S., Song, J., Zhu, G.: Depth enhancement with improved exemplar-based inpainting and joint trilateral guided filtering. In: International Conference on Image Processing, pp. 4102–4106 (2016)

    Google Scholar 

  5. Hervieu, A., Papadakis, N., Bugeau, A., Gargallo, P., Caselles, V.: Stereoscopic image inpainting: distinct depth maps and images inpainting. In: International Conference on Pattern Recognition, pp. 4101–4104 (2010)

    Google Scholar 

  6. Cheng, W., Hsieh, C., Lin, S., Wang, C., Wu, J.: Robust algorithm for exemplar-based image inpainting. In: International Conference on Computer Graphics, Imaging and Visualization, pp. 64–69 (2005)

    Google Scholar 

  7. Liu, J., Gong, X., Liu, J.: Guided inpainting and filtering for kinect depth maps. In: International Conference on Pattern Recognition, pp. 2055–2058 (2012)

    Google Scholar 

  8. Goyal, P., Diwakar, S., et al.: Fast and enhanced algorithm for exemplar-based image inpainting. In: Symposium on Image and Video Technology, pp. 325–330 (2010)

    Google Scholar 

  9. Atapour-Abarghouei, A., de La Garanderie, G.P., Breckon, T.P.: Back to Butterworth - a Fourier basis for 3D surface relief hole filling within RGB-D Imagery. In: International Conference on Pattern Recognition, pp. 2813–2818 (2016)

    Google Scholar 

  10. Herrera, D., Kannala, J., Heikkilä, J.: Depth map inpainting under a second-order smoothness prior. In: Scandinavian Conference on Image Analysis, pp. 555–566 (2013)

    Google Scholar 

  11. Kumar, V., Mukhopadhyay, J., Kumar Das Mandal, S.: Modified exemplar-based image inpainting via primal-dual optimization. In: Kryszkiewicz, M., Bandyopadhyay, S., Rybinski, H., Pal, S.K. (eds.) PReMI 2015. LNCS, vol. 9124, pp. 116–125. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19941-2_12

    Chapter  Google Scholar 

  12. Breckon, T.P., Fisher, R.B.: Amodal volume completion: 3D visual completion. Comput. Vis. Image Underst. 99(3), 499–526 (2005)

    Article  Google Scholar 

  13. Camplani, M., Salgado, L.: Efficient spatio-temporal hole filling strategy for kinect depth maps. In: IS&T/SPIE Electronic Imaging, p. 82900E (2012)

    Google Scholar 

  14. Breckon, T.P., Fisher, R.B.: A hierarchical extension to 3D non-parametric surface relief completion. Pattern Recognit. 45, 172–185 (2012)

    Article  Google Scholar 

  15. Wang, L., Jin, H., Yang, R., Gong, M.: Stereoscopic inpainting: joint color and depth completion from stereo images. In: International Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)

    Google Scholar 

  16. Camplani, M., Salgado, L.: Adaptive Spatio-Temporal Filter for Low-Cost Camera Depth Maps. In: International Conference on Emerging Signal Processing Applications, pp. 33–36 (2012)

    Google Scholar 

  17. Matyunin, S., Vatolin, D., Berdnikov, Y., Smirnov, M.: Temporal filtering for depth maps generated by kinect depth camera. In: 3DTV Conference, pp. 1–4 (2011)

    Google Scholar 

  18. Berdnikov, Y., Vatolin, D.: Real-time depth map occlusion filling and scene background restoration for projected-pattern based depth cameras. In: Graphic Conference IETP (2011)

    Google Scholar 

  19. Chen, C., Cai, J., Zheng, J., Cham, T., Shi, G.: Kinect depth recovery using a color-guided, region-adaptive, and depth-selective framework. ACM Trans. Intell. Syst. Technol. 6(2), 12 (2015)

    Article  Google Scholar 

  20. Wang, Z., Hu, J., Wang, S., Lu, T.: Trilateral constrained sparse representation for kinect depth hole filling. Pattern Recognit. Lett. 65, 95–102 (2015)

    Article  Google Scholar 

  21. Qi, F., Han, J., Wang, P., Shi, G., Li, F.: Structure guided fusion for depth map inpainting. Pattern Recognit. Lett. 34(1), 70–76 (2013)

    Article  Google Scholar 

  22. Bugeau, A., Bertalmío, M., Caselles, V., Sapiro, G.: A comprehensive framework for image inpainting. IEEE Trans. Image Process. 19(10), 2634–2645 (2010)

    Article  MathSciNet  Google Scholar 

  23. Bevilacqua, M., Aujol, J., Brédif, M., Bugeau, A.: Visibility estimation and joint inpainting of lidar depth maps. In: International Conference on Image Processing, pp. 3503–3507 (2016)

    Google Scholar 

  24. Zuo, Y., Wu, Q., An, P., Zhang, J.: Explicit measurement on depth-color inconsistency for depth completion. In: International Conference on Image Processing, pp. 4037–4041 (2016)

    Google Scholar 

  25. Geiger A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. In: Robotics Research (2013)

    Article  Google Scholar 

  26. Yamaguchi, K., McAllester, D., Urtasun, R.: Efficient joint segmentation, occlusion labeling, stereo and flow estimation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 756–771. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_49

    Chapter  Google Scholar 

  27. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)

    Article  Google Scholar 

  28. Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

    Google Scholar 

  29. Atapour-Abarghouei, A., Breckon, T.P.: A comparative review of plausible hole filling strategies in the context of scene depth image completion. J. Comput. Graph. 72, 39–58 (2018)

    Article  Google Scholar 

  30. Atapour-Abarghouei, A., Breckon, T.P.: DepthComp: real-time depth image completion based on prior semantic scene segmentation. In: British Machine Vision Conference, pp. 208.1–208.13 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Durham University, Durham, UK

    Amir Atapour-Abarghouei & Toby P. Breckon

Authors
  1. Amir Atapour-Abarghouei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toby P. Breckon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amir Atapour-Abarghouei .

Editor information

Editors and Affiliations

  1. University of Porto, Porto, Portugal

    Aurélio Campilho

  2. University of Waterloo, Waterloo, Ontario, Canada

    Fakhri Karray

  3. Biomedical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands

    Bart ter Haar Romeny

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Atapour-Abarghouei, A., Breckon, T.P. (2018). Extended Patch Prioritization for Depth Filling Within Constrained Exemplar-Based RGB-D Image Completion. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds) Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science(), vol 10882. Springer, Cham. https://doi.org/10.1007/978-3-319-93000-8_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-93000-8_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-92999-6

  • Online ISBN: 978-3-319-93000-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation