Abstract
The communication of multi-view videos requires the transmission and storage of huge amounts of data. To reduce this storage and bandwidth requirements, a reduced set of videos together with the depth information can be used. The 3D geometry information in the depth maps is used in conjunction with the texture information to generate any intermediate view between two received video streams. Unlike texture information, depth data is characterized by large homogeneous areas and sharp edges, where the latter define object boundaries. These different features imply that encoding of depth maps with the standard texture encoder might not be optimal and thus different methods can be used to improve the coding efficiency of depth maps. This chapter presents coding algorithms used for the compression of depth maps in 3DTV applications.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
(2010) Report on experimental framework for 3D video coding. ISO/IEC JTC1/SC29/WG11, N11631
ITU-T and ISO/IEC JTC 1/SC 29 (MPEG) (2013) High efficiency video coding, Recommendation ITU-T H.265 and ISO/IEC 23008-2
Sullivan G, Ohm J, Han W, Wiegand T (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circ Syst Video Technol 22(12):1649–1668
ITU-T and ISO/IEC JTC1 (2010) Advanced video coding for generic audiovisual services, ITU-T Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4 AVC)
Müller K, Schwarz H, Marpe D, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Rhee F, Tech G, Winken M, Wiegand T (2013) 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans Image Process 22(9):3366–3378
Schwarz H, Bartnik C, Bosse S, Brust H, Hinz T, Lakshman H, Merkle P, Müller K, Rhee H, Tech G, Winken M, Marpe D, Wiegand T (2012) Extension of high efficiency video coding (HEVC) for multiview video and depth data. In: Proceedings of the 19th IEEE international conference on image processing. pp 205–208
Sullivan G, Boyce J, Chen Y, Ohm J, Segall C, Vetro A (2013) Standardized extensions of high efficiency video coding (HEVC). IEEE J Selected Topics Signal Process 7(6):1001–1016
Tanimoto M, Tehrani M, Fujii T, Yendo T (2011) Free-viewpoint TV—a review of the ultimate 3DTV and its related technologies. IEEE signal processing magazine. pp 67–76
Do L, Zinger S, Morvan Y, With P (2009) Quality improving techniques in DIBR for free-viewpoint video. In: Proceedings of 3DTV conference: the true vision—capture, transmission and display of 3D video
Müller K, Smolic A, Dix K, Kauff P, Wiegand T (2010) Reliability-based generation and view synthesis in layered depth video. In Proceedings of the IEEE 10th workshop on multimedia signal processing. pp 34–39
Li W, Zhou J, Li B, Sezan I (2009) Virtual view specification and synthesis for free viewpoint television. IEEE Trans Circ Syst Video Technol 19(4):533–546
Shoemake K (1985) Animation rotation with quaternion curves. In: Proceedings of 12th annual conference on computer graphics and interactive techniques, vol 19, no. 3, pp 245–254
Lee C, Ho Y (2009) View synthesis using depth map for 3D video. In: Proceedings of the Asia-Pacific Signal Information Processing Association annual summit and conference. pp 350–357
Oh K, Yea S, Vetro A, Ho Y (2010) Virtual view synthesis method and self-evaluation metrics for free viewpoint television and 3D video. Int J Imaging Syst Technol 20(4):378–390
Yang X, Lui J, Sun J, Li X, Liu W, Gao Y (2011) DIBR based view synthesis for free-viewpoint television. In: Proceedings of 3DTV conference: the true vision—capture, transmission and display of 3D video
Mori Y, Fukushima N, Fujii N, Tanimoto M (2008) View generation with 3D warping using depth information for FTV. In: Proceedings of 3DTV conference: the true vision—capture, transmission and display of 3D video
Zarb T, Debono, C (2014) Depth-based image processing for 3D video rendering applications. In: Proceedings of the 21st international conference on systems, signals and image processing. pp 215–218
Tran A, Harada, K (2013) View synthesis with depth information based on graph cuts for FTV. In: Proceedings of the 19th Korea-Japan joint workshop on frontiers of computer vision. pp 289–294
Xu J, Yan F, Cao X (2014) Stereoacuity-guided depth image based rendering. In: Proceedings of the IEEE international conference on multimedia and expo
Lei J, Zhang C, Fang Y, Gu Z, Ling N, Hou C (2015) Depth sensation enhancement for multiple virtual view rendering. IEEE Trans Multimedia 17(4):457–469
JCT3V-J1005 (2014) 3D-HEVC test model 10, joint collaborative team on 3D video coding extension development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11
JCT3V-G1100 (2014) Common test conditions of 3DV core experiments. Joint collaborative team on 3D video coding extension development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11
Jager F (2012) Simplified depth map intra coding with an optional depth lookup table. In: Proceedings of the international conference on 3D imaging. pp 1–4
Merkle P, Bartnik C, Müller K, Marpe D, Wiegand T (2012) 3D video: depth coding based on inter-component prediction of block partitions. In: Proceedings of the picture coding symposium. pp 149–152
Song Y, Ho Y-S (2013) Simplified inter-component depth modeling in 3D-HEVC. In: Proceedings of the IEEE 11th image, video and multidimensional signal processing workshop
Zhang M, Zhao C, Xu J, Bai H (2013) A Fast depth-map wedgelet partitioning scheme for intra prediction in 3D video coding. In: Proceedings of the IEEE international symposium on circuits and systems. pp 2852–2855
Merkle P, Muller K, Marpe D, Wiegand T (2015) Depth intra coding for 3D video based on geometric primitives. IEEE Trans Circ Syst Video Technol
Zamarin M, Salmistraro M, Forchhammer S, Ortega A (2013) Edge-preserving intra depth coding based on context-coding and H.264/AVC. Multimedia and expo (ICME), 2013 IEEE international conference on, pp 1–6
Oh B, Wey H, Park D (2012) Plane segmentation based intra prediction for depth map coding. In: Proceedings of the picture coding symposium. pp 41–44
Shen G, Kim W-S, Ortega A, Lee J, Wey H (2010) Edge-aware intra prediction for depth-map coding. In: Proceedings of the 2010 IEEE international conference on image processing. pp 3393–3396
Krishnamurthy R, Chai B, Tao H, Sethuraman S (2001) Compression and transmission of depth maps for image-based rendering. In: Proceedings of the 2001 IEEE international conference on image processing. pp 828–831
Merkle P, Morvan Y, Smolic A, Farin D, Müller K, With P, Wiegand T (2009) The effects of multiview depth video compression on multiview rendering. Image Commun 24(1–2):73–88
Zia W, Diepold K, Sarkis M (2010) Fast depth map compression and meshing with compressed tritree. In: Proceedings of the 9th Asian conference on computer vision—volume part II. pp 44–55
Shen G, Kim W-S, Narang S, Ortega A, Lee J, Wey H (2010) Edge-adaptive transforms for efficient depth map coding. In: Proceedings of the picture coding symposium
Hu W, Cheung G, Ortega A, Au O (2015) Multiresolution graph Fourier transform for compression of piecewise smooth images. IEEE Trans Image Process 24(1):419–433
Graziosi D, Rodrigues N, Pagliari C, Silva E, Faria S, Perez M, Carvalho M (2010) Multiscale recurrent pattern matching approach for depth map coding. In: Proceedings of the picture coding symposium. pp 294–297
Graziosi D, Rodrigues N, Pagliari C, Faria S, Silva E, Carvalho M (2010) Compressing depth maps using multiscale recurrent pattern image coding. Electron Lett 46(5):340–341
Francisco N, Rodrigues N, Silva E, Carvalho M, Faria S, Silva V, Reis M (2008) Multiscale recurrent pattern image coding with a flexible partition scheme. In: Proceedings of the 15th IEEE international conference on image processing
Lucas L, Rodrigues N, Pagliari C, Silva E, Faria S (2012) Efficient depth map coding using linear residue approximation and a flexible prediction framework. In: Proceedings of the 19th IEEE international conference on image processing
Lucas L, Rodrigues N, Pagliari C, Silva E, Faria S (2013) Predictive depth map coding for efficient virtual view synthesis. In: Proceedings of the 20th IEEE international conference on image processing
Lucas L, Wegner K, Rodrigues N, Pagliari C, Silva E, Faria S (2015) Intra predictive depth map coding using flexible block partitioning. IEEE Trans Image Process 24(11):4055–4068
Müller K, Merkle P, Tech G, Wiegand T (2012) 3D video coding with depth modeling modes and view synthesis optimization. In: Proceedings of the signal and information processing association annual summit and conference
Park C-S (2015) Edge-based intramode selection for depth-map coding in 3d-HEVC. IEEE Trans Image Process 24(1):155–162
Chen Y, Tech G, Wegner K, Yea S (2014) Test model 9 of 3D-HEVC and MV-HEVC, document JCT3V-I1003, ITU-T SG 16 WP3 and ISO/IEC JTC 1/Sc 29/WG 11
Gu Z, Zheng J, Ling N, Zhang P (2013) Fast depth modeling mode selection for 3D HEVC depth intra coding. In: Proceedings of the IEEE international conference on multimedia and expo workshops
Park C-S (2015) Efficient intra-mode decision algorithm skipping unnecessary depth-modelling modes in 3D-HEVC. Electronics Lett 51(10):756–758
Ricci K, Debono C (2015) A fast inter-component depth modeling technique for 3D high efficiency video coding. In: Proceedings of 3DTV-conference 2015—immersive and interactive 3D media experience over networks
Witten I, Neal R, Cleary J (1987) Arithmetic coding for data compression. Commun ACM 30(6):520–540
ISO/IEC JTC1/SC29/WG11 MPEG2013/M31520, Wegner K, Stankiewicz O, Tanimoto M, Domanski M (2013) Enhanced view synthesis reference software (VSRS) for free-viewpoint television
Bjøntegaard G (2001) Calculation of average PSNR differences between RD-curves. ITU-T SG 16 Q.6 VCEG, Doc. VCEG-M33
Wildeboer M, Yendo T, Tehrani M, Fujii T, Tanimoto M (2010) Color based depth up-sampling for depth compression. In: Proceedings of the picture coding symposium. pp 170–173
Yang Y, Zheng J (2013) Edge-guided depth map resampling for HEVC 3D video coding. In: Proceedings of the international conference on virtual reality and visualization. pp 132–137
Schwarz S, Olsson R, Sjöström M, Tourancheau S (2012) Adaptive depth filtering for HEVC 3D video coding. In: Proceedings of the picture coding symposium. pp 49–52
Gonzalez R, Woods R (2007) Digital image processing, 3rd edn. Prentice Hall
Zammit L (2015) Depth map processing for down-sampled MV-HEVC encoded depth video. M.Sc. Dissertation, University of Malta
Graziosi D, Dong T, Vetro A (2012) Depth map up-sampling based on edge layers. In: Proceedings of the Asia-Pacific signal information processing association annual summit and conference
Huiping D, Li Y, Jinbo Q, Juntao Z (2012) A joint texture/depth edge-directed up-sampling algorithm for depth map coding. In: Proceedings of the IEEE international conference on multimedia and expo. pp 646–650
Oh K-J, Yea S, Vetro A, Ho Y-S (2009) Depth reconstruction filter and down/up sampling for depth coding in 3-D video. IEEE Signal Process Lett 16:747–750
Lee S, Lee S, Wey H, Lee J (2012) 3D-AVC—CE6 related results on Samsung’s in-loop depth resampling. ISO/IEC JTC1/SC29/WG11 Document M23661, San Jose, USA
Graziosi D, Rodrigues N, Silva E, Carvalho M, Faria S, Tian D, Vetro A (2013) Analysis of depth map resampling filters for depth-based 3D video coding. Conference on telecommunications—ConfTele
Guarda A, Santos J, Graziosi D, Rodrigues N, Faria S (2014) A novel trilateral filter technique for depth map processing in 3D video coding. In: Proceedings of the 3DTV conference—3DTV-CON, Budapest, Hungary
Acknowledgements
The authors would like to thank the Interactive Visual Media group at Microsoft Research for providing the Ballet and Breakdancers test sequences, Nagoya University for providing the Balloons test sequence, and National Institute of Information and Communications Technology (NICT) for providing the Sharks test sequence for research purposes.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer Science+Business Media New York
About this chapter
Cite this chapter
Debono, C.J., Faria, S., Lucas, L., Rodrigues, N. (2017). Depth Map Coding for 3DTV Applications. In: Kondoz, A., Dagiuklas, T. (eds) Connected Media in the Future Internet Era. Springer, New York, NY. https://doi.org/10.1007/978-1-4939-4026-4_6
Download citation
DOI: https://doi.org/10.1007/978-1-4939-4026-4_6
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4939-4024-0
Online ISBN: 978-1-4939-4026-4
eBook Packages: EngineeringEngineering (R0)