Skip to main content
SpringerLink
Log in

Challenges and Issues in Benchmarking MPI

  • Conference paper
Recent Advances in Parallel Virtual Machine and Message Passing Interface (EuroPVM/MPI 2006)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4192))

Abstract

Benchmarking MPI is a contentious subject at best. Micro- benchmarks are used because they are easy to port and, hypothetically, measure an important system characteristic in isolation. The unfortunate reality is that it is remarkably difficult to create a benchmark that is a fair measurement in the context of modern system. Software optimizations and modern processor architecture perform extremely efficiently on benchmarks, where it would not in an application context. This paper explores the challenges faced when benchmarking the network in a modern microprocessor climate and the remarkable impacts on the results that are obtained.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Netperf, http://www.netperf.org

  2. Pallas MPI Benchmarks, http://www.pallas.com/e/products/pmb/index.htm

  3. Alexandrov, A., Ionescu, M.F., Schauser, K.E., Sheiman, C.: LogGP: Incorporating long messages into the LogP model. Journal of Parallel and Distributed Computing 44(1), 71–79 (1997)

    Article  Google Scholar 

  4. Bell, C., Bonachea, D., Cote, Y., Duell, J., Hargrove, P., Husbands, P., Iancu, C., Welcome, M., Yelick, K.: An evaluation of current high-performance networks. In: 17th International Parallel and Distributed Processing Symposium (IPDPS 2003) (April 2003)

    Google Scholar 

  5. Culler, D.E., Karp, R.M., Patterson, D.A., Sahay, A., Schauser, K.E., Santos, E., Subramonian, R., von Eicken, T.: LogP: Towards a realistic model of parallel computation. In: Proceedings 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 1–12 (1993)

    Google Scholar 

  6. Lawry, W., Wilson, C., Maccabe, A.B., Brightwell, R.: COMB: A portable benchmark suite for assessing MPI overlap. In: IEEE International Conference on Cluster Computing, Poster paper (September 2002)

    Google Scholar 

  7. Liu, J., Chandrasekaran, B., Wu, J., Jiang, W., Kini, S., Yu, W., Buntinas, D., Wyckoff, P., Panda, D.K.: Performance comparison of MPI implementations over InfiniBand, Myrinet and Quadrics. In: The International Conference for High Performance Computing and Communications (SC 2003) (November 2003)

    Google Scholar 

  8. Martin, R.P., Vahdat, A.M., Culler, D.E., Anderson, T.E.: Effects of communication latency, overhead, and bandwidth in a cluster architecture. In: Proceedings of the 24th Annual International Symposium on Computer Architecture (June 1997)

    Google Scholar 

  9. Petrini, F., Kerbyson, D.J., Pakin, S.: The case of the missing supercomputer performance: Identifying and eliminating the performance variability on the ASCI Q machine. In: Proceedings of the 2003 Conference on High Performance Networking and Computing (November 2003)

    Google Scholar 

  10. Riesen, R., Brightwell, R., Maccabe, A.B.: Measuring MPI latency variance. In: Dongarra, J., Laforenza, D., Orlando, S. (eds.) EuroPVM/MPI 2003. LNCS, vol. 2840, pp. 112–116. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  11. Snell, Q.O., Mikler, A., Gustafson, J.L.: NetPIPE: A network protocol independent performance evaluator. In: Proceedings of the IASTED International Conference on Intelligent Information Management and Systems (June 1996)

    Google Scholar 

  12. Underwood, K.D.: The impacts of message rate on applications programming (submitted)

    Google Scholar 

  13. Underwood, K.D., Brightwell, R.: The impact of MPI queue usage on message latency. In: Proceedings of the International Conference on Parallel Processing (ICPP), Montreal, Canada (August 2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sandia National Laboratries, P.O. Box 5800, MS-1110, Albuquerque, NM, 87185-1110

    Keith D. Underwood

Authors
  1. Keith D. Underwood
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Forschungszentrum JĂĽlich, ZAM, 52425, JĂĽlich, Germany

    Bernd Mohr

  2. NEC Europe Ltd., NEC Laboratories Europe, Rathausallee 10, D-53757, Sankt Augustin, Germany

    Jesper Larsson Träff

  3. Dolphin Interconnect Solutions ASA R&D Germany, Siebengebirgsblick 26, 53343, Wachtberg, Germany

    Joachim Worringen

  4. Computer Science Department, University of Tennessee, 37996-3450, Knoxville, TN, USA

    Jack Dongarra

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Underwood, K.D. (2006). Challenges and Issues in Benchmarking MPI. In: Mohr, B., Träff, J.L., Worringen, J., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2006. Lecture Notes in Computer Science, vol 4192. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11846802_47

Download citation

  • DOI: https://doi.org/10.1007/11846802_47

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-39110-4

  • Online ISBN: 978-3-540-39112-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation