Abstract
Because of the immersive experience provided, 360-degree video is developing rapidly. However, high data volume is needed to create the panoramic video for the end-user, which conflicts with the limited transmission resources. In this paper, we present a novel transmission approach based on video content to guarantee an acceptable objective video quality for end-users by reusing the transmitted video data, in which transmission resources are saved by transmitting part of video data. The approach is designed to address challenges, such as the increment of video data volume from the tile-based encoding and the inconsistency between transmitted tiles and the viewed region using a single mode of tile partition. We also employ public network traces to simulate the network condition in our work. Simulated results show that our proposed approach can save bandwidth of \(5\% \sim 12\%\) only at the price of slightly eroding the perceived quality measured by SSIM value than transmitting all video data in the viewport. Moreover, the bandwidth saving varies with different video content.
Similar content being viewed by others
References
4G/LTE bandwidth logs, Ghent University, Belgium. https://users.ugent.be/jvdrhoof/dataset-4g/. Accessed 6 Jan 2021
Anwar MS, Wang J, Ullah A, Khan W, Li Z, Ahmad S (2018) User profile analysis for enhancing QoE of 360 panoramic video in virtual reality environment. In: ICVRV, pp 106–111
Azevedo R, Birkbeck N, Janatra I, Adsumilli B, Frossard P (2020) A viewport-driven multi-metric fusion approach for 360-degree video quality assessment. In: ICME, pp 1–1
Bentaleb A, Taani B, Begen A, Timmerer C, Zimmermann R (2019) A survey on bitrate adaptation schemes for streaming media over HTTP. IEEE Commun Surv Tutor 21(1):562–585
Brunnström K, Beker SA, De Moor K, Dooms A, Egger S, Garcia MN, Hossfeld T, Jumisko-Pyykkö S, Keimel C, Larabi MC, Lawlor B et al (2013) Qualinet white paper on definitions of quality of experience
Concolato C, Feuvre J, Denoual F, Mazé F, Ouedraogo N, Taquet J (2018) Adaptive streaming of HEVC tiled videos using MPEG-DASH. IEEE Trans Circuits Syst Video Technol 28(7):1981–1992
Corbillon X, Simone F, Simon G (2017) 360-degree video head movement dataset. In: Proc. of ACM on Multimedia Syst, pp 99–204
Covaci A, Trestian R, Saleme EB, Comsa IS, Assres G, Santos CA, Ghinea G (2019) 360∘ mulsemedia: a way to improve subjective QoE in 360∘ videos. In: Proc. MM, pp 2378–2386
Dasari M, Bhattacharya A, Vargas S, Sahu P, Balasubramanian A, Das R (2020) Streaming 360∘ videos using super-resolution. In: INFOCOM, pp 1–1
Dobrian F, Sekar V, Stoica I, Zhang H (2011) Understanding the impact of video quality on user engagement. In: Proc. of SIGCOMM workshops, pp 362–373
Draft VP9 bitstream and decoding process specification. https://www.webmproject.org/vp9/. Accessed 6 June 2021
Duanmu Z, Rehman A, Wang Z (2018) A quality-of-experience database for adaptive video streaming. IEEE Trans Broadcast 64(2):474–487
End-to-end optimizationsfor dynamic streaming. https://engineering.fb.com/2017/02/22/virtual-reality/end-to-end-optimizations-for-dynamic-streaming/. Accessed 3 March 2021
Ghaznavi-Youvalari R, Zare A, Aminlou A, Hannuksela M, Gabbouj M (2018) Shared coded picture technique for tile-based viewport-adaptive streaming of omnidirectional video. IEEE Trans Circuits Syst Video Technol 29 (10):3106–3120
Guan Y, Zheng C, Zhang X, Guo Z, Jiang J (2019) Pano: optimizing 360∘ video streaming with a better understanding of quality perception. In: Proc. of ACM SIGCOMM, pp 394–407
H.264 Specification. https://www.itu.int/rec/T-REC-H.264. Accessed 6 June 2021
H.265 Specification. https://www.itu.int/rec/T-REC-H.265. Accessed 6 June 2021
He J, Qureshi M, Qiu L, Li J, Li F, Han L (2018) Rubiks: practical 360-degree streaming for smartphones. In: Proceedings of the 16th ACM Annual International Conference Mobile System Application and Service, pp 482–494
How to watch Netflix in VR. https://arvrtips.com/netflix-in-virtual-reality/. Accessed 3 March 2021
Hu Q, Zhou J, Zhang X, Shi Z, Gao Z (2021) Viewport-adaptive 360-degree video coding. Multimed Tools Appl 79(17-18):12205–12226
iQiyi VR Channel. https://www.iqiyi.com. Accessed 16 March 2021
Lungaro P, Sjöberg R, Valero A, Mittal A, Tollmar K (2018) Gaze-aware streaming solutions for the next generation of mobile VR experiences. IEEE Trans Vis Comput Graph 24(4):1535–1544
Nguyen D, Tran H, Pham A, Thang T (2019) An optimal tile-based approach for viewport-adaptive 360-degree video streaming. IEEE J Emerg Sel Topics Circuits Syst 9(1):29–42
Nguyen A, Yan Z, Nahrstedt K (2018) Your attention is unique: detecting 360-degree video saliency in head-mounted display for head movement prediction. In: Proc. of MM, pp.1190–1198
Nguyen A, Yan Z, Nahrstedt K (2018) Your attention is unique: detecting 360-degree video saliency in head-mounted display for head movement prediction. In: Proc. of MM, pp 1190–1198
Ozcinar C, De Abreu A, Smolic A (2017) Viewport-aware adaptive 360∘ video streaming using tiles for virtual reality. In: Proceedings of the IEEE Conference ICIP, pp 2174–2178
Per-TitleEncode Optimization. https://netflixtechblog.com/per-title-encode-optimization-7e99442b62a2. Accessed 6 Dec 2020
Qian F, Han B, Xiao Q, Gopalakrishnan V (2018) Flare: practical viewport-adaptive 360-degree video streaming for mobile devices. In: Proc. of ACM MobiCom, pp 99–114
Qian F, Ji L, Han B, Gopalakrishnan V (2016) AllThings cellular optimizing 360 video delivery over cellular networks. In: Proc. of AllThingsCellular, pp. 1–1
Sodagar I (2011) The MPEG-DASH standard for multimedia streaming over the internet. IEEE Multimedia 18:62–67
Song J, Yang F, Zhang W, Zou W, Fan Y, Di P (2020) A fast FoV-switching DASH system based on tiling mechanism for practical omnidirectional video services. IEEE Trans Multimed 22(9):2366–2381
Sun Y, Lu A, Yu L (2017) Weighted-to-spherically-uniform quality evaluation for omnidirectional video. IEEE Signal Proc Let 24(9):1408–1412
The future of Hulu may be in VR. https://www.dailydot.com/upstream/hulu-vr/. Accessed 16 March 2021
Virtual reality (VR) headset unit sales worldwide from 2019 to 2024. https://www.statista.com/statistics/677096/vr-headsets-worldwide/. Accessed 16 Oct 2021
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Proc 13(4):600–612
Wang Z, Li F (2021) Convolutional neural network based low complexity HEVC intra encoder. Multimed Tools Appl 80:2441–2460
Xiao M, Zhou C, Liu Y, Chen S (2017) OpTile: toward optimal tiling in 360-degree video streaming. In: Proc. of ACM MM, pp 708–716
Xie L, Xu Z, Ban Y, Zhang X, Guo Z (2017) 360ProbDASH: improving QoE of 360 video streaming using tile-based HTTP adaptive streaming. In: Proc. of MM, pp 315–323
Xie S, Xu Y, Shen Q, Ma Z, Zhang W (2020) Modeling the perceptual quality of viewport adaptive omnidirectional video streaming. IEEE Trans Circuits Syst Video Technol 30(9):3029–3042
Yan Z, Liu Q, Zhang T, Chen C (2018) CrowdDBS: a crowdsourced brightness scaling optimization for display energy reduction in mobile video. IEEE Trans Mobile Comput 17(11):2536–2549
Yaqoob A, Bi T, Muntean GM (2020) A survey on adaptive 360∘ video streaming: solutions, challenges and opportunities. IEEE Commun Surv Tutor 22(4):2801–2838
Yeo H, Jung Y, Kim J, Shin J, Han D (2018) Neural adaptive content-aware internet video delivery. In: USENIX OSDI, pp 645–661
Yin X, Jindal A, Sekar V, Sinopoli B (2015) A control-theoretic approach for dynamic adaptive video streaming over http. In: ACM SigComm, pp 325–338
YouTube live in 360 degrees encoder settings. https://support.google.com/youtube/answer/6396222. Accessed 3 March 2021
Yu M, Lakshman H, Girod B (2015) A framework to evaluate omnidirectional video coding schemes. In: Proc. IEEE Int. Symp. Mixed Augmented Real., pp 31–36
Zhang J, Gao Q, Zhang G (2020) Edge cache replacement strategy for SVC-encoding tile-based 360-degree panoramic streaming. In: Conf HotICN, pp. 1–1
Zhang X, Hu X, Zhong L, Shirmohammadi S, Zhang L (2018) Cooperative tile-based 360-degree panoramic streaming in heterogeneous networks using scalable video coding. IEEE Trans Circuits Syst Video Technol 30(1):217–231
Zhou C, Li Z, Osgood J, Liu Y (2018) On the effectiveness of offset projections for 360-degree video streaming. ACM Trans Multimedia Comput Commun Appl 14(3s):1–24
Zou W, Zhang W, Song J, Yang F, Callet P (2019) A novel method for assessing the spatial presence of omnidirectional videos: comparing virtual and real world. IEEE Access 7:183405–183413
Zou W, Zhang W, Yang F (2021) Modeling the perceptual quality for viewport-adaptive omnidirectional video streaming considering dynamic quality boundary artifact. IEEE Trans Circuits Syst Video Technol:1–1
Acknowledgements
This research work was supported in part by the National Science Foundation of China (61671365, U1903213), and in part by the Key Research and Development Program of Shaanxi Province (2020KW-009).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflicts of interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, H., Li, F. & Yan, Z. A novel transmission approach based on video content for 360-degree streaming. Multimed Tools Appl 81, 34067–34085 (2022). https://doi.org/10.1007/s11042-022-11938-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-11938-9