Skip to main content
SpringerLink
Log in

Subjective QoE Models

  • Chapter
  • First Online:
Computational Inference and Control of Quality in Multimedia Services

Part of the book series: Springer Theses ((Springer Theses))

  • 313 Accesses

Abstract

The subjective QoE methods are concerned with quantifying the experienced quality of the users. Because these methods measure the subjective quality in an unmediated manner, their measurements are commonly used as ‘ground truth’ for evaluation of other methods [1]. In an end-to-end approach for QoE management, subjective estimation and subjective models are of key importance. It allows for evaluating the performance of the system and enables the loop back signal that closes the control loop. In this chapter these subjective QoE methods are discussed. QoE estimation methods based on rating, Just Noticable Differences and difference scalling are presented as well as their computational counterparts that enable building of the QoE models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. S. Winkler, P. Mohandas, The evolution of video quality measurement: from PSNR to hybrid metrics. IEEE Trans. Broadcast. 54(3), 660668 (2008). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4550731

  2. G. Exarchakos, V. Menkovski, L. Druda, A. Liotta, Network analysis on Skype end-to-end video quality. Int. J. Pervasive Comput. Commun. 11(1) (2015). http://www.emeraldinsight.com/doi/abs/10.1108/IJPCC-08-2014-0044

  3. G. Exarchakos, L. Druda, V. Menkovski, P. Bellavista, A. Liotta, Skype resilience to high motion videos. Int. J. Wavelets Multiresolut. Inf. Process. 11(3) (2013). http://dx.doi.org/10.1142/S021969131350029X

  4. F. Agboma, A. Liotta, Quality of experience management in mobile content delivery systems. J. Telecommun. Syst. special issue on the Quality of Experience issues in Multimedia Provision, 49(1), 85-98 (2012). doi:10.1007/s11235-010-9355-6

  5. V. Menkovski, G. Exarchakos, A. Liotta, A. Cuadra Sánchez, Quality of experience models for multimedia streaming. Int. J. Mobile Comput. Multimedia Commun. 2(4), 1–20 (2010). http://www.igi-global.com/ijmcmc/. doi:10.4018/jmcmc.2010100101. ISSN: 1937-9412

  6. M. Alhaisoni, A. Liotta, Characterization of signalling and traffic in Joost. J. P2P Networking Appl. special issue on Modelling and Applications of Computational P2P, 2, 75–83 (2009). doi:10.1007/s12083-008-0015-5, ISSN: 1936-6450

  7. M. Torres Vega, E. Giordano, D.C. Mocanu, D. Tjondronegoro, A. Liotta, Cognitive no-reference video quality assessment for mobile streaming services, in Proceedings of the 7th International Workshop on Quality of Multimedia Experience, Messinia, Greece, 26–29 May 2015 (IEEE). http://www.qomex.org

  8. A. Liotta, L. Druda, G. Exarchakos, V. Menkovski, Quality of experience management for video streams: the case of Skype, in Proceedings of the 10th International Conference on Advances in Mobile Computing and Multimedia, Bali, Indonesia, 3–5 Dec 2012 (ACM). http://dx.doi.org/10.1145/2428955.2428977

  9. V. Menkovski, A. Liotta, Intelligent control for adaptive video streaming, in Proceedings of the International Conference on Consumer Electronics, Las Vegas, US, 11–14 Jan 2013 (IEEE). http://dx.doi.org/10.1109/ICCE.2013.6486825

  10. A. Liotta, D. Constantin Mocanu, V. Menkovski, L. Cagnetta, G. Exarchakos, Instantaneous video quality assessment for lightweight devices, in Proceedings of the 11th International Conference on Advances in Mobile Computing and Multimedia, Vienna, Austria, 2–4 Dec 2013 (ACM). http://dx.doi.org/10.1145/2536853.2536903

  11. D.C. Mocanu, A. Liotta, A. Ricci, M. Torres Vega, G. Exarchakos, When does lower bitrate give higher quality in modern video services?, in Proceedings of the 2nd IEEE/IFIP International Workshop on Quality of Experience Centric Management, Krakow, Poland, 9 May 2014 (IEEE). http://dx.doi.org/10.1109/NOMS.2014.6838400

  12. D.C. Mocanu, G. Exarchakos, A. Liotta, Deep learning for objective quality assessment of 3D images, in Proceedings of IEEE International Conference on Image Processing, Paris, France, 27–30 Oct 2014 (IEEE)

    Google Scholar 

  13. M. Torres Vega, S. Zou, D. Constantin Mocanu, E. Tangdiongga, A.M.J. Koonen, A. Liotta, End-to-end performance evaluation in high-speed wireless networks, in Proceedings of the 10th International Conference on Network and Service Management, Rio de Janeiro, Brazil, 17–21 Nov 2014 (IEEE)

    Google Scholar 

  14. D. Constantin Mocanu, G. Santandrea, W. Cerroni, F. Callegati, A. Liotta, Network performance assessment with quality of experience benchmarks, in Proceedings of the 10th International Conference on Network and Service Management, Rio de Janeiro, Brazil, 17–21 Nov 2014 (IEEE)

    Google Scholar 

  15. D. Constantin Mocanu, G. Exarchakos, H.B. Ammar, A. Liotta, Reduced reference image quality assessment via Boltzmann machines, in Proceedings of the 3rd IEEE/IFIP IM 2015 International Workshop on Quality of Experience Centric Management, Ottawa, Canada, 11–15 May 2015 (IEEE)

    Google Scholar 

  16. M. Torres Vega, D. Constantin Mocanu, R. Barresi, G. Fortino, A. Liotta, Cognitive streaming on android devices, in Proceedings of the 1st IEEE/IFIP IM 2015 International Workshop on Cognitive Network and Service Management, Ottawa, Canada, 11–15 May 2015 (IEEE). http://www.cogman.org

  17. J. Okyere-Benya, M. Aldiabat, V. Menkovski, G. Exarchakos, A. Liotta, Video quality degradation on IPTV networks, in Proceedings of International Conference on Computing, Networking and Communications, Maui, Hawaii, USA, 30 Jan– 2 Feb 2012 (IEEE)

    Google Scholar 

  18. G. Exarchakos, V. Menkovski, A. Liotta, Can Skype be used beyond video calling?, in Proceedings of the 9th International Conference on Advances in Mobile Computing and Multimedia, Ho Chi Minh City, Vietnam, 5–7 Dec 2011 (ACM)

    Google Scholar 

  19. V. Menkovski, G. Exarchakos, A. Cuadra-Sanchez, A. Liotta, Estimations and remedies for quality of experience in multimedia streaming, in Proceedings of the 3rd International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services, Nice, France, 22–27 Aug 2010 (IEEE)

    Google Scholar 

  20. V. Menkovski, G. Exarchakos, A. Liotta, Machine learning approach for quality of experience aware networks, in Proceedings of Computational Intelligence in Networks and Systems, Thessaloniki, Greece, 24–26 Nov 2010 (IEEE)

    Google Scholar 

  21. F. Agboma, A. Liotta, User-centric assessment of mobile content delivery, in Proceedings of the 4th International Conference on Advances in Mobile Computing and Multimedia, Yogyakarta, Indonesia, 4–6 Dec 2006

    Google Scholar 

  22. F. Agboma, A. Liotta, Managing the user’s quality of experience, in Proceedings of the second IEEE/IFIP International Workshop on Business-driven IT Management (BDIM 2007), Munich, Germany, 21 May 2007 (IEEE)

    Google Scholar 

  23. F. Agboma, M. Smy, A. Liotta, QoE analysis of a peer-to-peer television system, in Proceedings of the International Conference on Telecommunications, Networks and Systems, Amsterdam, Netherlands, 22–24 July 2008

    Google Scholar 

  24. F. Agboma, A. Liotta, QoE-aware QoS management, in Proceedings of the 6th International Conference on Advances in Mobile Computing and Multimedia, Linz, Austria, 24–26 Nov 2008

    Google Scholar 

  25. V. Menkovski, A. Oredope, A. Liotta, A. Cuadra-Sanchez, Optimized online learning for QoE prediction, in Proceedings of the 21st Benelux Conference on Artificial Intelligence, Eindhoven, The Netherlands, 29–30 Oct 2009, pp. 169–176. http://wwwis.win.tue.nl/bnaic2009/proc.html. ISSN: 1568-7805

  26. V. Menkovski, A. Oredope, A. Liotta, A. Cuadra-Sanchez, Predicting quality of experience in multimedia streaming, in Proceedings of the 7th International Conference on Advances in Mobile Computing and Multimedia, Dec 2009 (ACM). http://dl.acm.org/citation.cfm?id=1821766. ISBN: 978-1-60558-659-5

  27. V. Menkovski, G. Exarchakos, A. Cuadra-Sanchez, A. Liotta, Measuring quality of experience on a commercial mobile TV platform, in Proceedings of the 2nd International Conference on Advances in Multimedia, Athens, Greece, 13–19 June 2010 (IEEE)

    Google Scholar 

  28. V. Menkovski, G. Exarchakos, A. Liotta, Online QoE prediction, in Proceedings of the 2nd IEEE International Workshop on Quality of Multimedia Experience, Trondheim, Norway, 21–23 June 2010 (IEEE)

    Google Scholar 

  29. V. Menkovski, G. Exarchakos, A. Liotta, Online learning for quality of experience management, in Proceedings of the Annual Machine Learning Conference of Belgium and The Netherlands, Leuven, Belgium, 27–28 May 2010. http://dtai.cs.kuleuven.be/events/Benelearn2010/submissions/benelearn2010_submission_20.pdf

  30. V. Menkovski, G. Exarchakos, A. Liotta, The value of relative quality in video delivery. J. Mobile Multimedia 7(3), 151162 (2011). http://dl.acm.org/citation.cfm?id=2230537

  31. F. Agboma, A. Liotta, Addressing user expectations in mobile content delivery. Mob. Inf. Syst. 3(3, 4), 153164 (2007). http://dl.acm.org/citation.cfm?id=1376820.1376823

  32. W.H. Ehrenstein, A. Ehrenstein, Psychophysical methods, in Modern Techniques in Neuroscience Research, 1999, pp. 1211–1241. http://uni-leipzig.de/~isp/isp/history/texts/PSYPHY-M.PDF

  33. V. Menkovski, G. Exarchakos, A. Liotta, Tackling the sheer scale of subjective QoE, in Proceedings of 7th International ICST Mobile Multimedia Communications Conference, Cagliari, Italy, 5–7 Sept 2011, vol. 29, pp. 1–15 (Springer, Lecture Notes of ICST, 2012). http://www.springerlink.com/content/p1443m265r25756x/. doi:10.1007/978-3-642-30419-4_1

  34. ITU, 500-11, Methodology for the subjective assessment of the quality of television pictures, recommendation ITU-R BT. 500-11, ITU Telecom. Standardization Sector of ITU (2002)

    Google Scholar 

  35. F. De Simone, M. Naccari, M. Tagliasacchi, F. Dufaux, S. Tubaro, T. Ebrahimi, Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel, in International Workshop on Quality of Multimedia Experience, QoMEx 2009. IEEE 2009, pp. 204–209 (2009)

    Google Scholar 

  36. Q. Huynh-Thu, M. Ghanbari, Modelling of spatio-temporal interaction for video quality assessment. Signal Process.: Image Commun. 25(7), 535–546 (2010)

    Google Scholar 

  37. K. Seshadrinathan, R. Soundararajan, A. Bovik, L. Cormack, Study of subjective and objective quality assessment of video. IEEE Trans. Image Process. 19(6), 1427–1441 (2010)

    Article  MathSciNet  Google Scholar 

  38. I. Recommendation, 910, Subjective video quality assessment methods for multimedia applications, recommendation ITU-T P. 910, ITU Telecom. Standardization Sector of ITU (1999)

    Google Scholar 

  39. M. Barkowsky, M. Pinson, R. Pépion, P. Le Callet, Analysis of freely available subjective dataset for HDTV including coding and transmission distortions, in Fifth International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM-10) (2010)

    Google Scholar 

  40. S. Stevens, On the psychophysical law. Psychol. Rev. 64(3), 153 (1957)

    Article  Google Scholar 

  41. D. Krantz, A theory of context effects based on cross-context matching. J. Math. Psychol. 5(1), 1–48 (1968)

    Article  MathSciNet  MATH  Google Scholar 

  42. D. Krantz, A theory of magnitude estimation and cross-modality matching. J. Math. Psychol. 9(2), 168–199 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  43. D. Krantz, R. Luce, P. Suppes, A. Tversky, Foundations of Measurement Volume I: Additive and Polynomial Representations, vol. 1 (Dover Publications, New York, 2006)

    Google Scholar 

  44. R. Shepard, On the status of ‘direct’ psychophysical measurement. Minn. Stud. Philos. Sci. 9, 441–490 (1978)

    Google Scholar 

  45. R. Shepard, Psychological relations and psychophysical scales: on the status of direct psychophysical measurement. J. Math. Psychol. 24(1), 21–57 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  46. A. Watson, Proposal: measurement of a JND scale for video quality, IEEE G-2.1. 6 Subcommittee on Video Compression Measurements (2000)

    Google Scholar 

  47. S. Winkler, On the properties of subjective ratings in video quality experiments, in International Workshop on Quality of Multimedia Experience, QoMEx 2009. IEEE 2009, pp. 139–144 (2009)

    Google Scholar 

  48. V. Menkovski, A. Liotta, Adaptive psychometric scaling for video quality assessment. Signal Process.: Image Commun. 27(8), 788–799 (2012)

    Google Scholar 

  49. V.Q.E. Group et al., Report on the validation of video quality models for high definition video content, Technical report (2010)

    Google Scholar 

  50. P. Corriveau, C. Gojmerac, B. Hughes, L. Stelmach, All subjective scales are not created equal: the effects of context on different scales. Signal Process. 77(1), 1–9 (1999)

    Article  MATH  Google Scholar 

  51. P. Brooks, B. Hestnes, User measures of quality of experience: why being objective and quantitative is important. IEEE Network 24(2), 8–13 (2010)

    Article  Google Scholar 

  52. G.T. Fechner, Elements of Psychophysics (Holt, Rinehart and Winston, New York, 1966)

    Google Scholar 

  53. J.D. McCarthy, M.A. Sasse, D. Miras, Sharp or smooth?: comparing the effects of quantization vs. frame rate for streamed video, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ser. CHI’04 (ACM, New York, NY, USA, 2004), p. 535542. http://doi.acm.org/10.1145/985692.985760

  54. E.H. Weber, E.H. Weber On the Tactile Senses (Psychology Press, Hove, 1996)

    Google Scholar 

  55. A. Watson, L. Kreslake, Measurement of visual impairment scales for digital video, in Human Vision, Visual Processing, and Digital Display. Proceedings. SPIE 4299 (2001)

    Google Scholar 

  56. L. L. Thurstone, The Measurement of Values, vol. vii (University of Chicago Press, Oxford, England, 1959)

    Google Scholar 

  57. L. Maloney, J. Yang, Maximum likelihood difference scaling. J. Vision 3(8) (2003)

    Google Scholar 

  58. C. Charrier, L. Maloney, H. Cherifi, K. Knoblauch, Maximum likelihood difference scaling of image quality in compression-degraded images. JOSA A 24(11), 3418–3426 (2007)

    Article  Google Scholar 

  59. K. Knoblauch, L. Maloney et al., MLDS: Maximum likelihood difference scaling in R. J. Stat. Softw. 25(2), 1–26 (2008)

    Article  Google Scholar 

  60. D. Green, J. Swets et al., Signal Detection Theory and Psychophysics, vol. 1974 (Wiley, New York, 1966)

    Google Scholar 

  61. H. Bergsten, JavaServer Pages, 3rd edn. (O’Reilly Media Inc., Sebastopol, 2003)

    Google Scholar 

  62. JW player: Overview | LongTail video | home of the JW player. http://www.longtailvideo.com/players

  63. L. Aimar, L. Merritt, E. Petit, M. Chen, J. Clay, M. Rullgrd, C. Heine, A. Izvorski, VideoLAN—x264, the best H.264/AVC encoder. http://www.videolan.org/developers/x264.html

  64. R. Team et al., R: A language and environment for statistical computing (R Foundation Statistical Computing, Vienna, 2008)

    Google Scholar 

  65. K. Knoblauch, psyphy: Functions for analyzing psychophysical data in R, R package version 0.0-5. http://CRAN.R-project.org/package=psyphy (2007)

  66. B. Efron, R. Tibshirani, An Introduction to the Bootstrap (Chapman & Hall/CRC Monographs on Statistics and Applied Probability) (Chapman and Hall/CRC, New York, 1994)

    Google Scholar 

  67. F. Wichmann, N. Hill, The psychometric function: I. Fitting, sampling, and goodness of fit. Attention Percept. Psychophys. 63(8), 1293–1313 (2001)

    Article  Google Scholar 

  68. V. Menkovski, G. Exarchakos, A. Liotta, Adaptive testing for video quality assessment, in Proceedings of Quality of Experience for Multimedia Content Sharing, Lisbon, Portugal, 29 June 2011 (ACM)

    Google Scholar 

  69. S. Kullback, R.A. Leibler, On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951). http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aoms/1177729694

  70. K. Eckschlager, K. Danzer, Information Theory in Analytical Chemistry (Wiley, New York, 1994)

    Google Scholar 

  71. M. Alhaisoni, A. Liotta, M. Ghanbari, Scalable P2P video streaming. Int. J. Bus. Data Commun. Networking 6(3), 49–65 (2010). doi:10.4018/jbdcn.2010070103. ISSN: 1548-0631

  72. M. Alhaisoni, M. Ghanbari, A. Liotta, Localized multistreams for P2P streaming. Int. J. Dig. Multimedia Broadcast. Article ID 843574, 12 pp. Hindawi 2010 (2010). doi:10.1155/2010/843574

  73. M. Alhaisoni, A. Liotta, M. Ghanbari, Resource-awareness and trade-off optimization in P2P video streaming. Int. J. Adv. Media Commun. special issue on High-Quality Multimedia Streaming in P2P Environments 4(1), 59–77 (2010). doi:10.1504/IJAMC.2010.030005. ISSN: 1741-8003

  74. A. Liotta, Farewell to deterministic networks, in Proceedings of the 19th IEEE Symposium on Communications and Vehicular Technology in the Benelux, Eindhoven, The Netherlands, 16 Nov 2012 (IEEE). http://dx.doi.org/10.1109/SCVT.2012.6399413

Download references

Author information

Authors and Affiliations

  1. Philips Research, Eindhoven, Noord-Brabant, The Netherlands

    Vlado Menkovski

Authors
  1. Vlado Menkovski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vlado Menkovski .

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Menkovski, V. (2015). Subjective QoE Models. In: Computational Inference and Control of Quality in Multimedia Services. Springer Theses. Springer, Cham. https://doi.org/10.1007/978-3-319-24792-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24792-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24790-8

  • Online ISBN: 978-3-319-24792-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation