Abstract
Application response time is a critical performance metric to assess the quality of software products. It is also an objective metric for user experience evaluation. In this paper, we present a novel method named CVART (Computer Vision–based Application Response Time measurement) for measuring the response time (latency) of an application. In our solution, we use image recognition and deep learning techniques to capture visible changes in the display of the device running the application to compute the application response time of an operation that triggers these visual changes. Appling CVART can bring multiple benefits compared to traditional methods. First, it allows measuring the response time that reflects a real user experience. Second, the solution enables the measurement of operations that are extremely hard or impossible to measure when using traditional methods. Third, it does not require application instrumentation, which is infeasible in many use cases. Finally, the method does not depend on any specific application or software platform, which allows building performance measurement and application monitoring tools that work on multiple platforms and on multiple devices. For demonstration, we present one use case of applying CVART to measure the application response time of virtual desktops hosted in the cloud or datacenter, and we evaluate its efficiency on measurement at large scale.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., Bottou, L., Bengio, Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of Computer Vision and Pattern Recognition Conference, pp. 2818–2826 (2016)
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 (2017)
Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent Neural Network Regularization. arXiv:1409.2329 (2014)
Szeliski, R.: Computer Vision: Algorithms and Applications. Springer, New York (2010). https://doi.org/10.1007/978-1-84882-935-0
Carroll, J.: Computer vision and deep learning technology at the heart of Amazon Go. www.vision-systems.com
Zeldovich, N., Chandra, R.: Interactive performance measurement with VNCplay. In: Proceedings of the FREENIX Track: 2005 USENIX Annual Technical Conference, Anaheim, CA, USA, 10–15 April 2005 (2005)
Friston, S., Steed, A.: Measuring latency in virtual environments. IEEE Trans. Vis. Comput. Graphics 20(4), 616–625 (2014)
Norton, T.R.: End-to-end response time: where to measure? In: Proceedings of the 1999 Computer Measurement Group Conference (1999)
Tang, Y.: Deep learning using linear support vector machines. In: Proceedings of International Conference on Machine Learning, Challenges in Representation Learning Workshop, Atlanta, Georgia, USA (2013)
Berryman, A., Calyam, P., Honigford, M., Lai, A.: VDBench: a benchmarking toolkit for thin-client based virtual desktop environments. In: Proceedings of 2nd IEEE International Conference on Cloud Computing Technology and Science, pp. 480–481 (2010)
Agrawal, B., et al.: VMware view planner: measuring true virtual desktop experience at scale. VMware Tech. J. (2009)
Wang, S.Y., Wang, W.J.: Benchmarking the performance of XenDesktop Virtual DeskTop Infrastructure (VDI) platform. In: Proceedings of the 14th International Conference on Networks, pp. 37–43 (2015)
Casas, P., Seufert, M., Egger, S., Schatz, R.: Quality of experience in remote virtual desktop services. In: Proceedings of 2013 IFIP/IEEE International Symposium on Integrated Network Management, pp. 1352–1357 (2013)
Top 5 Use Cases for VDI. http://www.gomindsight.com/blog/top-5-use-cases-vdi
VMware Always On Point of Care Solution Reference Implementation Case Study for European Healthcare Provider, Technical White Paper (2012)
Young, R., Laing, D.: IDC MarketScape: Worldwide Virtual Client Computing Software 2015 Vendor Assessment. IDC MarketScape (2015)
Young, R., Laing, D.: IDC MarketScape: Worldwide Virtual Client Computing Software 2016 Vendor Assessment. IDC MarketScape (2016)
Arakelian, C., Halstead, C.: Blast extreme display protocol in Horizon 7. In: VMware Technical White Paper (2016)
Pandey, A., Vu, L., Puthiyaveettil, V., Sivaraman, H., Kurkure, U., Bappanadu, A.: An automation framework for benchmarking and optimizing performance of remote desktops in the cloud. In: The Proceedings of the 2017 International Conference for High Performance Computing & Simulation (2017)
Horizon 7. https://www.vmware.com/products/horizon.html
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Data Mining, Inference, and Prediction, 2nd edn. Springer, Berlin (2009). https://doi.org/10.1007/978-0-387-84858-7
TensoreFlow. https://www.tensorflow.org
Kurkure, U., Sivaraman, H., Vu, L.: Machine learning using virtualized GPU in cloud environment. In: Proceedings of the 12th Workshop on Virtualization in High-Performance Cloud Computing (2017)
Acknowledgements
The authors would like to thank Juan Garcia-Rovetta and Bruce Herndon, Julie Brodeur for their support of this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Vu, L., Kurkure, U., Sivaraman, H., Bappanadu, A. (2018). Large Scale Application Response Time Measurement Using Image Recognition and Deep Learning. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2018. Lecture Notes in Computer Science(), vol 11241. Springer, Cham. https://doi.org/10.1007/978-3-030-03801-4_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-03801-4_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03800-7
Online ISBN: 978-3-030-03801-4
eBook Packages: Computer ScienceComputer Science (R0)