Abstract
Performance engineering of software could benefit from a closer integration of the use of performance models, and the use of measured data. Models can contribute to early warning of problems, exploration of solutions, and scalability evaluation, and when they are fitted to data they can summarize the data as a special powerful form of fitted function. Present industrial practice virtually ignores models, because of the effort to create them, and concern about how well they fit the system when it is implemented. The first concern is being met by automated generation from software specifications. The second concern can be met by fitting the models to data as it becomes available. This will adapt the model to the new situation and validate it, in a single step. The present paper summarizes the fitting process, using standard tools of nonlinear regression analysis, and shows it in action on examples of queueing and extended queueing models. The examples are a background for a discussion about the relationship between the models, and measurement data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Balsamo, S., DiMarco, A., Inverardi, P., Simeoni, M.: Model-based Performance Prediction in Software Development. IEEE Trans. on Software Eng. 30(5), 295–310 (2004)
Barber, S.: Beyond performance testing, parts 1-14, IBM DeveloperWorks, Rational Technical Library (2004), www-128.ibm.com/developerworks/rational/library/4169.html
Bogardi-Meszoly, A., Levendovszky, T., Charaf, H., Hashimoto, T.: Improved Evaluation Algorithm for Performance Prediction with Error Analysis. In: Proc. 11th Int. Conf. on Intelligent Engineering Systems, pp. 301–306 (2007)
IBM, IBM Rational PurifyPlus, Purify, PureCoverage, and Quantify: Getting Started, G126-5339-00 (May 2002)
Franks, G., Majumdar, S., Neilson, J., Petriu, D., Rolia, J., Woodside, M.: Performance Analysis of Distributed Server Systems. In: Proc. Sixth International Conference on Software Quality (6ICSQ), Ottawa, pp. 15–26 (1996)
Franks, G., Petriu, D., Woodside, M., Xu, J., Tregunno, P.: Layered bottlenecks and their mitigation. In: Proc of 3rd Int. Conference on Quantitative Evaluation of Systems QEST 2006, Riverside, CA, September 2006, pp. 103–114 (2006)
Jain, R.: The Art of Computer Systems Performance Analysis. John Wiley & Sons Inc., Chichester (1991)
Kutner, M.H., Nachtsheim, C.J., Neter, J., Li, W.: Applied Linear Statistical Models, 5th edn. McGraw-Hill, New York (2005)
Litoiu, M., Zheng, T., Woodside, M.: Service System Resource Management Based on a Tracked Layered Performance Model. In: Proc. IEEE Int. Conf. on Autonomic Computing, Dublin (June 2006)
Liu, Y., Fekete, A., Gorton, I.: Design-Level Performance Prediction of Component-Based Applications. IEEE Trans. on Software Engineering 31(11), 928–941 (2005)
Miller, B.P., Callaghan, M.D., Cargille, J.M., Hollingsworth, J.K., Irvin, R.B., Karavanic, K.L., Kunchithapadam, K., Newhall, T.: The Paradyn Parallel Performance Measurement Tool. IEEE Computer 28(11), 37–46 (1995)
Rolia, J.A., Sevcik, K.C.: The Method of Layers. IEEE Trans. on Software Engineering 21(8), 689–700 (1995)
Roth, P.C., Miller, B.P.: On-line Automated Performance diagnosis on Thousands of Processes. In: ACM SigPLAN Symp. on Principles and Practices of Parallel Programming (PPOPP 2006), New York (March 2006)
Smith, C.U., Williams, L.G.: Performance Solutions. Addison-Wesley, Reading (2002)
Storm, A.J., Garcia-Arellano, C., Lightstone, S.S., Diao, Y., Surendra, M.: Adaptive self-tuning memory in DB2. In: Proc. 32nd Int. Conf. on Very large databases, Seoul, pp. 1081–1092 (2006)
Tantawi, A.N.: Method and system for dynamic performance modeling of computer application services. USA, Patent Application 20070299638 (2007)
Vugrin, K.W., Swiler, L.P., Roberts, R.M., Stucky-Mack, N.J., Sullivan, S.P.: Confidence Region Estimation: Techniques for Nonlinear Regression: Three Case Studies. Sandia Laboratories Report SAND2005-6893 (October 2005)
Woodside, M., Petriu, D.C., Petriu, D.B., Shen, H., Israr, T., Merseguer, J.: Performance by Unified Model Analysis (PUMA). In: Proc. WOSP 2005, Mallorca, pp. 1–12 (2005)
Woodside, C.M., Zheng, T., Litoiu, M.: The Use of Optimal Filters to Track Parameters of Performance Models. In: Proc. 2nd Int. Conf. on Quantitative Evaluation of Systems, Torino, Italy, pp. 74–84 (2005)
Woodside, M., Franks, G., Petriu, D.C.: The Future of Software Performance Engineering. In: Proc Future of Software Engineering 2007, at ICSE 2007, May 2007, pp. 171–187, Order Number P2829. IEEE Computer Society, Los Alamitos (2007)
WOSP, The Proceedings of the ACM International Workshop on Software and Performance. ACM Press (1998-2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Woodside, M. (2008). The Relationship of Performance Models to Data. In: Kounev, S., Gorton, I., Sachs, K. (eds) Performance Evaluation: Metrics, Models and Benchmarks. SIPEW 2008. Lecture Notes in Computer Science, vol 5119. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69814-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-69814-2_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69813-5
Online ISBN: 978-3-540-69814-2
eBook Packages: Computer ScienceComputer Science (R0)