Abstract
In this paper I address the divide that has emerged between the field of performance evaluation and the field of computer and communication system design. After looking back briefly on the history of the field, I subsequently reflect on the reasons why the field of performance evaluation has become so isolated. I then continue with a set of eight recommendations, based on my experience in performing projects with industry, that will help in reconnecting, and that will result in a better uptake of the newest techniques and tools in the field of design of computer and communication systems. Following these recommendations will probably push scientists a little out of their comfort zone, however, I feel the potential extra reward of seeing our work truly applied is more than worth it.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
ACM Special Interest Group for Computer Performance Evaluation. https://www.sigmetrics.org/, https://dblp.org/db/conf/sigmetrics/index.html. Accessed 5 July 2021
van den Berg, F.G.B.: Automated performance evaluation of service-oriented systems. Ph.D. thesis, University of Twente. https://research.utwente.nl/en/publications/automated-performance-evaluation-of-service-oriented-systems (2017). Accessed 5 July 2021
Bertsekas, D.P., Gallager, R.G.: Data Networks, 2nd edn. Prentice-Hall, Upper Saddle River (1992)
Brinch Hansen, P.: Operating System Principles. Prentice-Hall, Englewood Cliffs (1973)
Bux, W.: Modeling token ring networks-a survey. In: Herzog, U., Paterok, M. (eds.) Messung, Modellierung und Bewertung von Rechensystemen. Informatik-Fachberichte, vol. 154, pp. 192-221. Springer, Heidelberg (1987). https://doi.org/10.1007/978-3-642-73016-0_13
Coffmann, E.G., Denning, P.J.: Operating Systems Theory. Prentice-Hall, Englewood Cliffs (1973)
Digital Twin Research. https://www.digital-twin-research.nl/. Accessed 5 July 2021
Embedded Systems Institute: I served as scientific director during 2009–2013. https://esi.nl/. Accessed 5 July 2021
Frenkel, A.K.: Allan L. Scherr, big blue’s time-sharing pioneer. Commun. ACM 30(10), 824–828 (1987)
Ferrari, D.: Computer Systems Performance Evaluation. Prentice Hall, Upper Saddle River (1978)
Ferrari, D.: Considerations on the insularity of performance evaluation. IEEE Transactions on Software Engineering 12(2), 21–32 (1986)
Haverkort, B.R.: Performance Evaluation of Computer Communication Systems: A Model-Based Approach. Wiley, Hoboken (1998)
Hennessy, J.L., Patterson, D.A.: Computer Architecture: A Quantitative Approach, 3rd edn. Morgan Kaufman, Burlington (2003)
Journal of Applied Probability. Cambridge University Press. https://www.cambridge.org/core/journals/journal-of-applied-probability. Accessed 5 July 2021
Kleinrock, L.: Queueing Systems, Volume II: Computer Applications. Prentice-Hall, Upper Saddle River (1987)
Kurose, J.F., Ross, K.: Computer Networking: A Top-Down Approach, 7th edn. Pearson, London (2017)
IEEE Modeling and Simulation of Computer and Telecommunication Systems. https://dblp.org/db/conf/mascots/index.html. Accessed 5 July 2021
Onvural, R.O.: Asynchronous Transfer Mode Networks. Artech House, Norwood (1994)
Performance Evaluation. Elsevier. https://www.journals.elsevier.com/performance-evaluation. Accessed 5 July 2021
Quantitative Evaluation of Systems. https://www.qest.org/, https://dblp.org/db/conf/qest/index.html. Accessed 5 July 2021
Queueing Systems: Theory and Applications. Springer. https://www.springer.com/journal/11134. Accessed 5 July 2021
Reiser, M., Lavenberg, S.S.: Mean-value analysis of closed multi-chain queuing networks. Journal of the ACM 27(2), 313–322 (1980)
Silberschatz, A., Galvin, P.B.: Operating System Concepts, 4th edn. Addison-Wesley, Readin (1994)
Tanenbaum, A.S.: Computer Networks, 3rd edn. Prentice-Hall, Hoboken (1996)
Tanenbaum, A.S., Bos, H.: Modern Operating Systems. Pearson, London (2014)
Tichy, W.T.: Should computer scientists experiment more? IEEE Computer 31(5), 32–40 (1998)
Walrand, J.: Communication Networks: A First Course. Aksen Associates Incorporated Publishers, Stamford (1991)
Acknowledgements
First of all, I would like to thank all the lecturers and fellow scientists that I have been privileged to work with and to learn from over the last 35 years. It is this cooperation and interaction that makes science move forward! Secondly, I would like to thank the program committee chairs for QEST’2021, Alessandro Abate and Andrea Marin, for inviting me as keynote speaker, as well as for their constructive criticism on an earlier version of this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Haverkort, B.R. (2021). Performance Evaluation: Model-Driven or Problem-Driven?. In: Abate, A., Marin, A. (eds) Quantitative Evaluation of Systems. QEST 2021. Lecture Notes in Computer Science(), vol 12846. Springer, Cham. https://doi.org/10.1007/978-3-030-85172-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-85172-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85171-2
Online ISBN: 978-3-030-85172-9
eBook Packages: Computer ScienceComputer Science (R0)