Abstract
View synthesis optimization (VSO) is one of the core techniques for depth map coding in three dimensional high efficiency video coding (3D-HEVC). It improves the quality for synthesized views, while it also introduces heavy computational complexity caused by the calculation of synthesized view distortion change (SVDC) in practice. To reduce the complexity, this paper proposes a convolutional neural network-based VSO scheme in 3D-HEVC. First, the potential factors that may relate to the encoding complexity are explored. Then, based on this, a convolutional neural network (CNN) is embedded into the 3D-HEVC reference software HTM16.0 to predict the depth of coding units (CUs). The complexity of SVDC can be drastically reduced by avoiding the brute-force search for VSO in depth 0 and depth 1. Finally, for depth 2 and depth 3, the zero distortion area (ZDA) is determined based on texture smoothness and the SVDC calculation for that area is skipped. The experimental results show that the proposed scheme can reduce 76.7% encoding time without any significant loss for the 3D video quality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gerhard, T., Ying, C., Karsten, M., Jens-Rainer, O., Anthony, V., et al.: Overview of the multiview and 3D extensions of high efficiency video coding. IEEE Trans. Circ. Syst. Video Technol. 26(1), 35–49 (2016)
Li, Z., Meng, L., Xu, S., Li, Z., Shi, Y., Liang, Y.: A HEVC video steganalysis algorithm based on PU partition modes. Comput. Mater. Con. 59(2), 563–574 (2019)
Gerhard, T., Krzysztof, W., Ying, C., Sehoon, Y.: 3D-HEVC Draft Text 7. JCT-3V, doc. JCT3V-K1001, Geneva, CH (2015)
Sullivan, G.J., Boyce, J.M., Chen, Y., Ohm, J., Segall, C.A., et al.: Standardized extensions of high efficiency video coding (HEVC). IEEE J. Sel. Top. Sig. Process. 7(6), 1001–1016 (2013)
Zhang, Q., Wu, Q., Wang, X.: Early SKIP mode decision for three-dimensional high efficiency video coding using spatial and interview correlations. J. Electron. Imaging 23, 053017 (2014)
Zheng, Z., Huo, J., Li, B., Yuan, H.: Fine virtual view distortion estimation method for depth map coding. IEEE Sig. Process. Lett. 25(3), 417–421 (2018)
Guo, L., Zhou, L., Tian, X., Chen, Y.: Hole-filling map-based coding unit size decision for dependent views in three-dimensional high-efficiency video coding. J. Electron. Imaging 25(3), 033020 (2016)
Zhang, Y., Kwong, S., Hu, S., Kuo, C.-C.J.: Efficient multiview depth coding optimization based on allowable depth distortion in view synthesis. IEEE Trans. Image Process. 23(11), 4879–4892 (2014)
Ma, S., Wang, S., Gao, W.: Low complexity adaptive view synthesis optimization in HEVC based 3D video coding. IEEE Trans. Multimedia 16(1), 266–271 (2014)
Yang, M., Zheng, N., Zhu, C., Wang, F.: A novel method of minimizing view synthesis distortion based on its non-monotonicity in 3D video. IEEE Trans. Image Process. 26(11), 5122–5137 (2017)
Gerhard, T., Heiko, S., Karsten, M., Thomas, W.: 3D video coding using the synthesized view distortion change. In: 2012 Picture Coding Symposium, Krakow, Poland, May 2012
Liao, Y.-W., Chen, M.-J., Yeh, C.-H., Lin, J.-R., Chen, C.-W.: Efficient inter-prediction depth coding algorithm based on depth map segmentation for 3D-HEVC. Multimed. Tools Appl. 78(8), 10181–10205 (2018). https://doi.org/10.1007/s11042-018-6547-7
Yang, M., Zhu, C., Lan, X., Zheng, N.: Efficient estimation of view synthesis distortion for depth coding optimization. IEEE Trans. Multimedia 21(4), 863–874 (2019)
Jing, R., Zhang, Q., Wang, B., Cui, P., Yan, T., Huang, J.: CART-based fast CU size decision and mode decision algorithm for 3D-HEVC. SIViP 13(2), 209–216 (2018). https://doi.org/10.1007/s11760-018-1347-0
Mario, S., Gustavo, S., Cesar, M., Luciano, A.: Fast 3D-HEVC depth maps intra-frame prediction using data mining. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, April 2018
Cui, Q., McIntosh, S., Sun, H.: Identifying materials of photographic images and photorealistic computer generated graphics based on deep CNNs. Comput. Mater. Con. 55(2), 229–241 (2018)
Fang, W., Zhang, F., Sheng, V.S., Ding, Y.: A method for improving CNN-based image recognition using DCGAN. Comput. Mater. Con. 57(1), 167–178 (2018)
Xu, M., Li, T., Wang, Z., Deng, X., Yang, R., et al.: Reducing complexity of HEVC: a deep learning approach. IEEE Trans. Image Process. 27(10), 5044–5059 (2018)
Dou, H., Chan, Y.-L., Jia, K.-B., Siu, W.-C.: Segment–based view synthesis optimization scheme in 3D-HEVC. J. Vis. Commun. Image Represent. 42, 104–111 (2016)
Acknowledgement
This work was supported in part by the Project for the National Natural Science Foundation of China under Grants No. 61672064, Beijing Laboratory of Advanced Information Networks under Grants No. PXM2019_014204_500029, and the Beijing Natural Science Foundation under Grant No. 4172001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, C., Jia, Kb., Liu, Py. (2020). A Convolutional Neural Network-Based Complexity Reduction Scheme in 3D-HEVC. In: Sun, X., Wang, J., Bertino, E. (eds) Artificial Intelligence and Security. ICAIS 2020. Lecture Notes in Computer Science(), vol 12239. Springer, Cham. https://doi.org/10.1007/978-3-030-57884-8_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-57884-8_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-57883-1
Online ISBN: 978-3-030-57884-8
eBook Packages: Computer ScienceComputer Science (R0)