Skip to main content
SpringerLink
Log in

Light Field and Plenoptic Point Cloud Compression

  • Chapter
  • First Online:
Handbook of Dynamic Data Driven Applications Systems

Abstract

Light field, or plenoptic images are richer representation of the visual information in 3D. It contains not only the color intensity of a particular view angle at the time of capture, but also much richer photometric and geometric information for re-rending and viewing from new view angles. This is the key enabling technology for the future immersive visual communication with many applications. There are mainly three types of light field images based on the capturing technology, lenslet camera based, dense camera array based, and non-regularly sampled light field or plenotpic point cloud. In this chapter we will review and discuss the current state of the art in compression for these light field images.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Dynamic Data Driven Applications Systems. http://www.1dddas.org/.

  2. Light Field Tool box 0.4. http://www.mathworks.com/matlabcentral/fileexchange/49683-light-field-toolbox-v0-4.

  3. A. Aggoun. A 3D DCT compression algorithm for omnidirectional integral images. In 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, volume 2, pages II–II, May 2006.

    Google Scholar 

  4. A. Aggoun. Compression of 3D integral images using 3D wavelet transform. Journal of Display Technology, 7(11):586–592, Nov 2011.

    Article  Google Scholar 

  5. H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. Speeded-up robust features (surf). Computer Vision and Image Understanding, 110(3):346 – 359, 2008. Similarity Matching in Computer Vision and Multimedia.

    Google Scholar 

  6. O. Chabiron, F. Malgouyres, J.-Y. Tourneret, and N. Dobigeon. Toward fast transform learning. International Journal of Computer Vision, 114(2-3):195–216, 2015.

    Article  MathSciNet  Google Scholar 

  7. F. Chen, H. Li, L. Li, D. Liu, and F. Wu. Block-composed background reference for high efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology, PP(99):1–1, 2016.

    Google Scholar 

  8. H.-M. Chen, E. Blasch, K. Pham, Z. Wang, and G. Chen. An investigation of image compression on niirs rating degradation through automated image analysis. In Sensors and Systems for Space Applications IX, volume 9838, page 983811. International Society for Optics and Photonics, 2016.

    Google Scholar 

  9. C. Conti, J. Lino, P. Nunes, L. D. Soares, and P. L. Correia. Spatial prediction based on self-similarity compensation for 3D holoscopic image and video coding. In 2011 18th IEEE International Conference on Image Processing, pages 961–964, Sept 2011.

    Google Scholar 

  10. C. Conti, P. Nunes, and L. D. Soares. HEVC-based light field image coding with bi-predicted self-similarity compensation. In IEEE International Conf. on Multimedia and Expo - ICME, pages 1– 4, July 2016.

    Google Scholar 

  11. C. Conti, L. D. Soares, and P. Nunes. HEVC-based 3D holoscopic video coding using self-similarity compensated prediction. Signal Processing: Image Communication, 42:59–78, March 2016.

    Google Scholar 

  12. E. Cornwell, L. Li, Z. Li, and Y. Sun. An efficient compression scheme for the multi-camera light field image. In 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), pages 1–6, Oct 2017.

    Google Scholar 

  13. P. Helle, S. Oudin, B. Bross, D. Marpe, M. O. Bici, K. Ugur, J. Jung, G. Clare, and T. Wiegand. Block merging for quadtree-based partitioning in HEVC. IEEE Transactions on Circuits and Systems for Video Technology, 22(12):1720–1731, Dec 2012.

    Article  Google Scholar 

  14. H. Huang, J. Woods, Y. Zhao, and H. Bai. Control-point representation and differential coding affine-motion compensation. Circuits and Systems for Video Technology, IEEE Transactions on, 23(10):1651–1660, Oct. 2013.

    Article  Google Scholar 

  15. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec. Rendering for an interactive 360 light field display. In ACM Transactions on Graphics (TOG), volume 26, page 40. ACM, 2007.

    Google Scholar 

  16. M. Krivokuća and C. Guillemot. Colour compression of plenoptic point clouds Using RAHT-KLT with prior colour clustering and specular/diffuse component separation. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1978–1982, 2020.

    Google Scholar 

  17. D. Lanman, G. Wetzstein, M. Hirsch, W. Heidrich, and R. Raskar. Polarization fields: dynamic light field display using multi-layer lcds. ACM Transactions on Graphics (TOG), 30(6):186, 2011.

    Google Scholar 

  18. M. Levoy and P. Hanrahan. Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96, pages 31–42, New York, NY, USA, 1996. ACM.

    Google Scholar 

  19. L. Li, H. Li, D. Liu, Z. Li, H. Yang, L. Sixin, H. Chen, and F. Wu. An efficient four-parameter affine motion model for video coding. IEEE Transactions on Circuits and Systems for Video Technology, PP(99):1–1, 2017.

    Google Scholar 

  20. L. Li, Z. Li, B. Li, D. Liu, and H. Li. Pseudo sequence based 2-d hierarchical coding structure for light-field image compression. In Data Compression Conference (DCC), 2017, pages 131–140. IEEE, 2017.

    Google Scholar 

  21. L. Li, Z. Li, B. Li, D. Liu, and H. Li. Pseudo-sequence-based 2-d hierarchical coding structure for light-field image compression. IEEE Journal of Selected Topics in Signal Processing, 11(7):1107–1119, Oct 2017.

    Article  Google Scholar 

  22. L. Li, Z. Li, S. Liu, and H. Li. Video-based compression for plenoptic point clouds. In arxiv 1911.01355, 2019.

    Google Scholar 

  23. L. Li, Z. Li, S. Liu, and H. Li. Occupancy-map-based rate distortion optimization and partition for video-based point cloud compression. IEEE Transactions on Circuits and Systems for Video Technology, pages 1–1, 2020.

    Google Scholar 

  24. Y. Li, D. Liu, H. Li, L. Li, F. Wu, H. Zhang, and H. Yang. Convolutional neural network-based block up-sampling for intra frame coding. IEEE Transactions on Circuits and Systems for Video Technology, PP(99):1–1, 2017.

    Google Scholar 

  25. Y. Li, R. Olsson, and M. Sjöström. Compression of unfocused plenoptic images using a displacement intra prediction. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pages 1–4, July 2016.

    Google Scholar 

  26. Y. Li, M. Sjöström, R. Olsson, and U. Jennehag. Coding of focused plenoptic contents by displacement intra prediction. IEEE Transactions on Circuits and Systems for Video Technology, 26(7):1308–1319, July 2016.

    Article  Google Scholar 

  27. D. Liu, L. Wang, L. Li, Z. Xiong, F. Wu, and W. Zeng. Pseudo-sequence-based light field image compression. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pages 1–4, July 2016.

    Google Scholar 

  28. M. Magnor and B. Girod. Data compression for light-field rendering. IEEE Transactions on Circuits and Systems for Video Technology, 10(3):338–343, 2000.

    Article  Google Scholar 

  29. J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online dictionary learning for sparse coding. In Proceedings of the 26th annual international conference on machine learning, pages 689–696. ACM, 2009.

    Google Scholar 

  30. K. Marwah, G. Wetzstein, Y. Bando, and R. Raskar. Compressive light field photography using overcomplete dictionaries and optimized projections. ACM Transactions on Graphics (TOG), 32(4):46, 2013.

    Google Scholar 

  31. R. Monteiro, L. Lucas, C. Conti, P. Nunes, N. M. M. Rodrigues, S. Faria, C. Pagliari, E. Silva, and L. D. Soares. Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction. In IEEE International Conf. on Multimedia and Expo - ICME, pages 1– 4, July 2016.

    Google Scholar 

  32. D. Naik, M. Pesonen, and S. Schwarz. [V-PCC] On surface light field support for TMC2. Document ISO/IEC JTC1/SC29/WG11 MPEG2019/M46057, Marrakesh, MA, Jan. 2019.

    Google Scholar 

  33. C. Perra and P. Assuncao. High efficiency coding of light field images based on tiling and pseudo-temporal data arrangement. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pages 1–4, July 2016.

    Google Scholar 

  34. M. Rerabek, T. Bruylants, T. Ebrahimi, F. Pereira, and P. Schelkens. ICME 2016 Grand Challenges: Light-Field Image Compression. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW), July 2016.

    Google Scholar 

  35. Y. Romano, M. Protter, and M. Elad. Single image interpolation via adaptive nonlocal sparsity-based modeling. IEEE Transactions on Image Processing, 23(7):3085–3098, 2014.

    Article  MathSciNet  Google Scholar 

  36. R. Rubinstein, A. M. Bruckstein, and M. Elad. Dictionaries for sparse representation modeling. Proceedings of the IEEE, 98(6):1045–1057, 2010.

    Article  Google Scholar 

  37. G. Sandri, R. De Queiroz, and P. A. Chou. Compression of plenoptic point clouds using the region-adaptive hierarchical transform. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 1153–1157, Oct 2018.

    Google Scholar 

  38. G. Sandri, R. L. de Queiroz, and P. A. Chou. Compression of plenoptic point clouds. IEEE Transactions on Image Processing, 28(3):1419–1427, March 2019.

    Article  MathSciNet  Google Scholar 

  39. H. Schwarz, D. Marpe, and T. Wiegand. Analysis of hierarchical B pictures and MCTF. In 2006 IEEE International Conference on Multimedia and Expo, pages 1929–1932, July 2006.

    Google Scholar 

  40. S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Cohen, M. Krivokuća, S. Lasserre, Z. Li, et al. Emerging MPEG standards for point cloud compression. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(1):133–148, 2018.

    Article  Google Scholar 

  41. R. Sjoberg, Y. Chen, A. Fujibayashi, M. M. Hannuksela, J. Samuelsson, T. K. Tan, Y. K. Wang, and S. Wenger. Overview of HEVC high-level syntax and reference picture management. IEEE Transactions on Circuits and Systems for Video Technology, 22(12):1858–1870, Dec 2012.

    Article  Google Scholar 

  42. J. Sulam, B. Ophir, M. Zibulevsky, and M. Elad. Trainlets: Dictionary learning in high dimensions. IEEE Transactions on Signal Processing, 64(12):3180–3193, 2016.

    Article  MathSciNet  Google Scholar 

  43. G. J. Sullivan, J. R. Ohm, W. J. Han, and T. Wiegand. Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits and Systems for Video Technology, 22(12):1649–1668, Dec 2012.

    Article  Google Scholar 

  44. I. Viola, M. Rerabek, T. Bruylants, P. Schelkens, F. Pereira, and T. Ebrahimi. Objective and subjective evaluation of light field image compression algorithms. In 2016 32nd Picture Coding Symposium, Dec. 2016.

    Google Scholar 

  45. G. K. Wallace. The JPEG still picture compression standard. IEEE Transactions on Consumer Electronics, 38(1):xviii–xxxiv, Feb 1992.

    Google Scholar 

  46. T. Wiegand, E. Steinbach, and B. Girod. Affine multipicture motion-compensated prediction. Circuits and Systems for Video Technology, IEEE Transactions on, 15(2):197–209, Feb. 2005.

    Article  Google Scholar 

  47. T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the H.264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 13(7):560–576, July 2003.

    Google Scholar 

  48. J. Xu, R. Joshi, and R. A. Cohen. Overview of the emerging HEVC screen content coding extension. IEEE Transactions on Circuits and Systems for Video Technology, 26(1):50–62, Jan 2016.

    Article  Google Scholar 

  49. X. Xu, S. Liu, T. D. Chuang, Y. W. Huang, S. M. Lei, K. Rapaka, C. Pang, V. Seregin, Y. K. Wang, and M. Karczewicz. Intra block copy in HEVC screen content coding extensions. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, PP(99):1–11, 2016.

    Google Scholar 

  50. H. Yu, Z. Lin, and F. Teo. An efficient coding scheme based on image alignment for H.264/AVC. In Circuits and Systems, 2009. ISCAS 2009. IEEE International Symposium on, pages 629–632, May 2009.

    Google Scholar 

  51. X. Zhang, P. A. Chou, M.-T. Sun, M. Tang, S. Wang, S. Ma, and W. Gao. Surface light field compression using a point cloud codec. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 9(1):163–176, 2018.

    Article  Google Scholar 

  52. X. Zhang, T. Huang, Y. Tian, and W. Gao. Background-modeling-based adaptive prediction for surveillance video coding. IEEE Transactions on Image Processing, 23(2):769–784, 2014.

    Article  MathSciNet  Google Scholar 

  53. Y. Zheng, G. Chen, Z. Wang, and E. Blasch. Image quality (iq) guided multispectral image compression. In Sensing and Analysis Technologies for Biomedical and Cognitive Applications 2016, volume 9871, page 98710C. International Society for Optics and Photonics, 2016.

    Google Scholar 

  54. R. Zhong, I. Schiopu, B. Cornelis, S.-P. Lu, J. Yuan, and A. Munteanu. Dictionary learning-based, directional, and optimized prediction for lenslet image coding. IEEE Transactions on Circuits and Systems for Video Technology, 29(4):1116–1129, 2018.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Missouri-Kansas City, Kansas City, MO, USA

    Li Li & Zhu Li

Authors
  1. Li Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Li Li or Zhu Li .

Editor information

Editors and Affiliations

  1. Air Force Office of Scientific Research, Arlington, VA, USA

    Erik P. Blasch

  2. InfoSymbiotics Systems Society, Boston, MA, USA

    Frederica Darema

  3. Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA

    Sai Ravela

  4. Air Force Research Lab, Rome, NY, USA

    Alex J. Aved

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Li, L., Li, Z. (2022). Light Field and Plenoptic Point Cloud Compression. In: Blasch, E.P., Darema, F., Ravela, S., Aved, A.J. (eds) Handbook of Dynamic Data Driven Applications Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-74568-4_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-74568-4_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-74567-7

  • Online ISBN: 978-3-030-74568-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation