Skip to main content
SpringerLink
Log in

Adaptive Bit Allocation for 3D Video Coding

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

In the multi-view video plus depth 3D video coding, texture image and depth map are coded jointly. The texture image is utilized for displaying and synthesizing the virtual view as reference image. The depth map provides the scene geometry information and is utilized to synthesize the virtual view at the terminal through Depth-Image Based Rendering technique. The distortion of the compressed texture image and depth map will be propagated to the synthesized virtual view. Besides the coding efficiency of texture image and depth map, bit allocation between texture image and depth map also has a great effect on the synthesized virtual view quality. Several methods are proposed for bit allocation between texture image and depth map, but most of them attempt to allocate a fixed target bitrate based on virtual view distortion model to achieve optimal synthesized virtual view quality, and the modeling process brings extra complexity. In practical application, the video sequence has different contents and fixed bit ratio cannot achieve optimal performance. In this paper, we propose an adaptive bit allocation algorithm for 3D video coding. First, we present a model to estimate the synthesized virtual view distortion, and then adjust the bit ratio between adjacent views and between texture image and depth map at Group of Picture level based on the virtual view quality fluctuation. We adjust the bit ratio to achieve the optimal virtual view quality for different video contents. Experimental results demonstrate that the proposed algorithm can optimally allocate bits to achieve optimal virtual view quality under different target bitrates and for different video contents, and the computational complexity of the proposed algorithm is extremely low.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. 3DV Sequences of Nagoya University. Nagoya University, Nagoya. http://www.tanimoto.nuee.nagoyau.ac.jp/mpeg/mpeg-ftv.html. Mar 2008

  2. 3DV Sequences of Poznan University. Poznan University, Poznan. ftp://multimedia.edu.pl/3DV/. Nov 2012

  3. 3DV Sequences of HHI. Fraunhofer Heinrich Hertz Institute, Berlin. ftp://ftp.hhi.de/HHIMPEG3DV. Sep 2013

  4. 3D-HTM reference software version 10.0rc1. https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-10.0rc1/

  5. P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey, V. Sainov, C. von Kopylow, A survey of 3DTV displays: techniques and technologies. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1647–1658 (2007)

    Article  Google Scholar 

  6. G. Bjontegaard, Calculation of average PSNR difference between RD-curves, in 13th VCEG-M33 Meeting 2001, Austin, TX, USA (2001)

  7. E. Bosc, V. Jantet, M. Pressigout, L. Morin, C. Guillemot, Bit-rate allocation for multi-view video plus depth, in 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON) (2011), pp. 1–4

  8. Y. Chen, M.M. Hannuksela, T. Suzuki, S. Hattori, Overview of the MVC+D 3D video coding standard. J. Vis. Commun. Image Represent. 25(4), 679–688 (2014)

    Article  Google Scholar 

  9. C. Fehn, Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV, in Proceedings of SPIE. International Society for Optics and Photonics Electronic Imaging (2004), pp. 93–104

  10. C. Fehn, R. de la Barré, S. Pastoor, Interactive 3-DTV-concepts and key technologies. Proc. IEEE 94(3), 524–538 (2006)

    Google Scholar 

  11. M. M. Hannuksela, Y. Chen, T. Suzuki, J.R. Ohm, G.J. Sullivan, 3D-AVC Draft Text 8, document JCT-3V JCT3V-F1002, Nov 2013

  12. P. Kauff, N. Atzpadin, C. Fehn, M. Müller, O. Schreer, A. Smolic, R. Tanger, Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Signal Process. Image Commun. 22(2), 217–234 (2007)

    Article  Google Scholar 

  13. W.S. Kim, A. Ortega, P. Lai, D. Tian, C. Gomila, Depth map coding with distortion estimation of rendered view, in IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics (2010), pp. 75430B–75430B-10

  14. J. Konrad, M. Halle, 3-D displays and signal processing. IEEE Signal Process. Mag. 24(6), 97–111 (2007)

    Article  Google Scholar 

  15. B. Li, H. Li, L. Li, J. Zhang, Rate control by R-lambda model for HEVC, in ITU-T SG16 Contribution, JCTVC-K0103, Shanghai (2012), pp. 1–5

  16. Y. Liu, Q. Huang, S. Ma, D. Zhao, W. Gao, Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Process. Image Commun. 24(8), 666–681 (2009)

    Article  Google Scholar 

  17. Y. Liu, Q. Huang, S. Ma, D. Zhao, W. Gao, S. Ci, H. Tang, A novel rate control technique for multiview video plus depth based 3D video coding. IEEE Trans. Broadcast. 57(2), 562–571 (2011)

    Article  Google Scholar 

  18. P. Merkle, A. Smolic, K. Muller, T. Wiegand, Multi-view video plus depth representation and coding, in Proceedings of IEEE ICIP (2007), pp. 201–204

  19. Y. Morvan, D. Farin, H.N. de With, Joint depth/texture bit-allocation for multi-view video compression, in Proceedings of Picture Coding Symposium (PCS 2007) (2007), pp. 265–268

  20. Y.K. Park, K. Jung, Y. Oh, S. Lee, J.K. Kim, G. Lee, H. Lee, K. Yun, N. Hur, J. Kim, Depth-image-based rendering for 3DTV service over T-DMB. Signal Process. Image Commun. 24(1), 122–136 (2009)

    Article  Google Scholar 

  21. D. Rusanovskyy, K. Mueller, A. Vetro, Common test conditions of 3DV core experiments, in Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V 2013) Document JCT3V-E1100, 5th Meeting, Vienna, Austria (2013)

  22. F. Shao, G. Jiang, W. Lin, M. Yu, Q. Dai, Joint bit allocation and rate control for coding multi-view video plus depth based 3D video. IEEE Trans. Multimed. 15(8), 1843–1854 (2013)

    Article  Google Scholar 

  23. L. Shen, Z. Zhang, Z. Liu, Effective CU size decision for HEVC intra coding. IEEE Trans. Image Process. 23(10), 4232–4241 (2014)

    Article  MathSciNet  Google Scholar 

  24. L. Shen, Z. Zhang, Z. Liu, Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Trans. Circuits Syst. Video Technol. 24(10), 1709–1722 (2014)

    Article  Google Scholar 

  25. G.J. Sullivan, J.R. Ohm, W.J. Han, T. Wiegand, Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1688 (2012)

    Article  Google Scholar 

  26. G. Tech, K. Wegner, Y. Chen, S. Yea, 3D-HEVC Test Model 3, MPEG number m28377, Geneve, Switzerland, Jan 2013

  27. Q. Wang, X. Ji, Q. Dai, N. Zhang, Free viewpoint video coding with rate-distortion analysis. IEEE Trans. Circuits Syst. Video Technol. 22(6), 875–889 (2012)

    Article  Google Scholar 

  28. J. Xiao, M.M. Hannuksela, T. Tillo, C. Zhu, Y. Zhao, Scalable bit allocation between texture and depth views for 3-D video streaming over heterogeneous networks. IEEE Trans. Circuits Syst. Video Technol. 25(1), 139–152 (2015)

    Article  Google Scholar 

  29. J. Xiao, T. Tillo, H. Yuan, Y. Zhao, Macroblock level bits allocation for depth maps in 3-D video coding. J. Signal Process. Syst. 74(1), 127–135 (2013)

    Article  Google Scholar 

  30. X. Xu, L.M. Po, K.W. Cheung, K.H. Ng, K.M. Wong, C.W. Ting, A foreground biased depth map refinement method for DIBR view synthesis, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2012), pp. 805–808

  31. C. Yang, P. An, D. Liu, L. Shen, Virtual view distortion estimation for depth map coding, in IEEE Visual Communications and Image Processing (VCIP), Singapore, Dec 2015, pp. 1–4

  32. H. Yuan, J. Liu, H. Xu, Z. Li, W. Liu, Coding distortion elimination of virtual view synthesis for 3D video system: theoretical analyses and implementation. IEEE Trans. Broadcast. 58(4), 558–568 (2012)

    Article  Google Scholar 

  33. H. Yuan, Y. Chang, J. Huo, F. Yang, Z. Lu, Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans. Circuits Syst. Video Technol. 21(4), 485–497 (2011)

    Article  Google Scholar 

  34. Y. Zhang, S. Kwong, L. Xu, S. Hu, C. Kuo, G. Jiang, Regional bit allocation and rate distortion optimization for multiview depth video coding with view synthesis distortion model. IEEE Trans. Image Process. 22(9), 3497–3512 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China, under Grants U1301257, 61172096, 61571285, 61422111 and 61301112.

Author information

Authors and Affiliations

  1. School of Communication and Information Engineering, Shanghai University, No. 149, Yanchang Road, Shanghai, 200072, China

    Chao Yang, Ping An, Liquan Shen & Deyang Liu

  2. Key Laboratory of Advanced Display and System Application, No. 149, Yanchang Road, Shanghai, 200072, China

    Ping An & Liquan Shen

Authors
  1. Chao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ping An
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liquan Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Deyang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ping An.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, C., An, P., Shen, L. et al. Adaptive Bit Allocation for 3D Video Coding. Circuits Syst Signal Process 36, 2102–2124 (2017). https://doi.org/10.1007/s00034-016-0402-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-016-0402-8

Keywords

Search

Navigation