Skip to main content
SpringerLink
Log in

Parallel Programming Models Applicable to Cluster Computing and Beyond

  • Conference paper
Numerical Solution of Partial Differential Equations on Parallel Computers

Part of the book series: Lecture Notes in Computational Science and Engineering ((LNCSE,volume 51))

Summary

This chapter centers mainly on successful programming models that map algorithms and simulations to computational resources used in high-performance computing. These resources range from group-based or departmental clusters to high-end resources available at the handful of supercomputer centers around the world. Also covered are newer programming models that may change the way we program high-performance parallel computers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alphaserver SC user guide, 2000. Bristol, Quadrics Supercomputer World Ltd.

    Google Scholar 

  2. R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. R. Kohn, L. McInnes, S. R. Parker, and B. A. Smolinski. Toward a common component architecture for high-performance scientific computing. In Proceedings of the 8th High Performance Distributed Computing (HPDC’99), 1999. URL: http://www.cca-forum.org.

    Google Scholar 

  3. S. Balay, W. D. Gropp, L. C. McInnes, and B. F. Smith. PETSc users manual. Technical Report ANL-95/11-Revision 2.1.0, Argonne National Laboratory, 2001.

    Google Scholar 

  4. R. Bariuso and A. Knies. SHMEM’s User’s Guide. SN-2515 Rev. 2.2, Cray Research, Inc., Eagan, MN, USA, 1994.

    Google Scholar 

  5. M. Bull. OpenMP 2.5 and 3.0. In Proceedings of the Workshop on OpenMP Applications and Tools, WOMPAT 2004, Houston, TX, May 17–18 2004. (Invited talk).

    Google Scholar 

  6. P. M. Burton, B. Carruthers, G. S. Fischer, B. H. Johnson, and R.W. Numrich. Converting the halo-update subroutine in the MET Office unified model to Co-Array Fortran. In W. Zwieflhofer and N. Kreitz, editors, Developments in Teracomputing: Proceedings of the Ninth ECMWF Workshop on the Use of High Performance Computing in Meteorology, pp. 177–188. World Scientific Publishing, 2001.

    Google Scholar 

  7. J. B. Carter, J. K. Bennett, and W. Zwaenepoel. Implementation and performance of Munin. In Proceedings of the 13th ACM Symp. on Operating Systems Principles (SOSP-13), pages 152–164, 1991.

    Google Scholar 

  8. Center for Programming Models for Scalable Parallel Computing. URL: http://www.pmodels.org.

    Google Scholar 

  9. R. Chandra, L. Dagum, D. Kohr, D. Maydan, J. McDonald, and R. Menon. Parallel Programming in OpenMP. Morgan Kaufmann Publishers, San Francisco, CA, 2001.

    Google Scholar 

  10. D. Chen, S. Dwarkadas, S. Parthasarathy, E. Pinheiro, and M. L. Scott. Interweave: A middleware system for distributed shared state. In Languages, Compilers, and Run-Time Systems for Scalable Computers, pages 207–220, 2000.

    Google Scholar 

  11. E. Chow, A. Cleary, and R. Falgout. HYPRE User’s manual, version 1.6.0. Technical Report UCRL-MA-137155, Lawrence Livermore National Laboratory, Livermore, CA, 1998.

    Google Scholar 

  12. D. Clark. OpenMP: A parallel standard for the masses. IEEE Concurrency, 6(1):10–12, January–March 1998.

    Article  Google Scholar 

  13. C. Coarfa, Y. Dotsenko, J. L. Eckhardt, and J. Mellor-Crummey. Co-array Fortran performance and potential: An NPB experimental study. In The 16th International Workshop on Languages and Compilers for Parallel Computing (LCPC 2003), College Station, Texas, October 2003.

    Google Scholar 

  14. Cray Research. Application Programmer’s Library Reference Manual, 2nd edition, Nov. 1995. Publication SR-2165.

    Google Scholar 

  15. L. Dagum and R. Menon. OpenMP: An industry standard API for shared-memory programming. IEEE Computational Science & Engineering, 5(1):46–55, January–March 1998.

    Article  Google Scholar 

  16. S. Dong and G. E. Karniadakis. Dual-level parallelism for deterministic and stochastic CFD problems. In Proceedings of Supercomputing, SC02, Baltimore, MD, 2002.

    Google Scholar 

  17. J. Dongarra, I. Foster, G. Fox, W. D. Gropp, K. Kennedy, L. Torczon, and A. White, editors. Sourcebook of Parallel Computing. Morgan Kaufmann, 2003.

    Google Scholar 

  18. P. F. Dubois. Ten Good Practices In Scientific Programming. Computing in Science & Engineering, 1(1), January–February 1999.

    Google Scholar 

  19. S. Dwarkadas, N. Hardavellas, L. Kontothanassis, R. Nikhil, and R. Stets. Cashmere-VLM: Remote memory paging for software distributed shared memory. In Proceedings of the 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing, pages 153–159. IEEE Computer Society, Apr. 1999.

    Google Scholar 

  20. Earth Simulator home page, http://www.es.jamstec.go.jp.

    Google Scholar 

  21. T. A. El-Ghazawi, W. W. Carlson, and J. M. Draper. UPC Language Specifications Version 1.1.1, October 2003. URL: http://www.gwu.edu/~upc/docs/upc_spec_1.1.1.pdf.

    Google Scholar 

  22. T. Elrad, R. E. Filman, and A. Bader. Aspect-Oriented Programming. Communications of the ACM, 44(10):29–32, October 2001.

    Article  Google Scholar 

  23. R. D. Falgout, J. E. Jones, and U. M. Yang. The design and implementation of hypre, a library of parallel high performance preconditioners. In A. M. Bruaset and A. Tveito, editors, Numerical Solution of Partial Differential Equations on Parallel Computers, volume 51 of Lecture Notes in Computational Science and Engineering, pages 267–294. Springer-Verlag, 2005.

    Google Scholar 

  24. M. Folk, A. Cheng, and K. Yates. HDF5: A file format and I/O library for high performance computing applications. In Proceedings of Supercomputing’99 (CD-ROM). ACM SIGARCH and IEEE, Nov. 1999.

    Google Scholar 

  25. FORTRAN 77 Binding of X3H5 Model for Parallel Programming Constructs. Draft Version, ANSI X3H5, 1992.

    Google Scholar 

  26. P. C. Forum. PCF Parallel FORTRAN Extensions. FORTRAN Forum, 10(3), September 1991. (Special issue).

    Google Scholar 

  27. Global Array Project. URL: http://www.emsl.pnl.gov/docs/global.

    Google Scholar 

  28. W. D. Gropp. Learning from the success of MPI. In B. Monien, V. K. Prasanna, and S. Vajapeyam, editors, High Performance Computing — HiPC 2001, number 2228 in Lecture Notes in Computer Science, pages 81–92. Springer, Dec. 2001.

    Google Scholar 

  29. W. D. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir. MPI—The Complete Reference: Volume 2, The MPI-2 Extensions. MIT Press, Cambridge, MA, 1998.

    Google Scholar 

  30. W. D. Gropp, E. Lusk, and A. Skjellum. Using MPI: Portable Parallel Programming with the Message Passing Interface, 2nd edition. MIT Press, Cambridge, MA, 1999.

    Google Scholar 

  31. W. D. Gropp, E. Lusk, and T. Sterling, editors. Beowulf Cluster Computing with Linux. MIT Press, 2nd edition, 2003.

    Google Scholar 

  32. W. D. Gropp, E. Lusk, and R. Thakur. Using MPI-2: Advanced Features of the Message-Passing Interface. MIT Press, Cambridge, MA, 1999.

    Google Scholar 

  33. R. Hempel and D. W. Walker. The emergence of the MPI message passing standard for parallel computing. Computer Standards and Interfaces, 21(1):51–62, 1999.

    Article  Google Scholar 

  34. High Performance Fortran Forum. High Performance Fortran language specification. Scientific Programming, 2(1–2):1–170, 1993.

    Google Scholar 

  35. J. M. D. Hill, B. McColl, D. C. Stefanescu, M.W. Goudreau, K. Lang, S. B. Rao, T. Suel, T. Tsantilas, and R. H. Bisseling. BSPlib: The BSP programming library. Parallel Computing, 24(14):1947–1980, Dec. 1998.

    Article  Google Scholar 

  36. C. A. R. Hoare. Communicating sequential processes. Communications of the ACM, 21(8):666–677, Aug. 1978.

    Article  MATH  MathSciNet  Google Scholar 

  37. J. Hoeflinger. Towards industry adoption of OpenMP. In Proceedings of the Workshop on OpenMP Applications and Tools, WOMPAT 2004, Houston, TX, May 17–18 2004. Invited Talk.

    Google Scholar 

  38. F. Hoffman. Writing hybrid MPI/OpenMP code. Linux Magazine, 6(4):44–48, April 2004. URL: http://www.linux-mag.com/2004-04/extreme_01.html.

    Google Scholar 

  39. Y. Hu, H. Lu, A. L. Cox, and W. Zwaenepoel. OpenMP for networks of SMPs. In Proceedings of the 13th International Parallel Processing Symposium, April 1999.

    Google Scholar 

  40. P. Hyde. Java Thread Programming. SAMS, 1999.

    Google Scholar 

  41. IEEE Standard for Information Technology-Portable Operating System Interface (POSIX). IEEE Standard No.: 1003.1, 2004.

    Google Scholar 

  42. W. Jiang, J. Liu, H.-W. Jin, D. K. Panda, W. D. Gropp, and R. Thakur. High performance MPI-2 one-sided communication over InfiniBand. Technical Report ANL/MCS-P1119-0104, Mathematics and Computer Science Division, Argonne National Laboratory, 2004.

    Google Scholar 

  43. G. Jost, J. Labarta, and J. Gimenez. What multilevel parallel programs do when you are not watching: A performance analysis case study comparing MPI/OpenMP, MLP, and nested OpenMP. In Proceedings of the Workshop on OpenMP Applications and Tools, WOMPAT 2004, pages 29–40, Houston, TX, May 17–18 2004. (Invited talk).

    Google Scholar 

  44. P. Keleher, A. L. Cox, S. Dwarkadas, and W. Zwaenepoel. TreadMarks: Distributed shared memory on standard workstations and operating systems. In Proceedings of the Winter 94 Usenix Conference, pages 115–131, January 1994.

    Google Scholar 

  45. R. A. Kendall, E. Aprà, D. E. Bernholdt, E. J. Bylaska, M. Dupuis, G. I. Fann, R. J. Harrison, J. Ju, J. A. Nichols, J. Nieplocha, T. P. Straatsma, T. L. Windus, and A. T. Wong. High performance computational chemistry; an overview of NWChem a distributed parallel application. Computer Physics Communications, 128:260–283, 2002.

    Article  Google Scholar 

  46. M. G. Knepley, R. F. Katz, and B. Smith. Developing a geodynamics simulator with petsc. In A. M. Bruaset and A. Tveito, editors, Numerical Solution of Partial Differential Equations on Parallel Computers, volume 51 of Lecture Notes in Computational Science and Engineering, pages 413–438. Springer-Verlag, 2005.

    Google Scholar 

  47. C. Koelbel, D. B. Loveman, R. S. Schreiber, G. L. Steele, and M. E. Zosel. The High Performance Fortran Handbook. MIT Press, 1994.

    Google Scholar 

  48. B. Leasure, editor. PCF Fortran: Language Definitons, Version 3.1. The Parallel Computing Forum, Champaign, IL, 1990.

    Google Scholar 

  49. J. Li, W. Liao, A. Choudhary, R. Ross, R. Thakur, W. D. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale. Parallel netCDF: A high-performance scientific I/O interface. In Proceedings of SC2003, Nov. 2003.

    Google Scholar 

  50. Z. Li, Y. Saad, and M. Sosonkina. pARMS: A parallel version of the algebraic recursive multilevel solver. Numerical Linear Algebra with Applications, 10:485–509, 2003.

    Article  MathSciNet  MATH  Google Scholar 

  51. R. K. Lie Huang, Barbara Chapman. OpenMP on distributed memory via global arrays. In Proceedings of Parallel Computing 2003 (ParCo2003), Dresden, Germany, September 2–5 2003.

    Google Scholar 

  52. Message Passing Interface Forum. MPI: A Message-Passing Interface standard. International Journal of Supercomputer Applications, 8(3/4):165–414, 1994.

    Google Scholar 

  53. Message Passing Interface Forum. MPI2: A Message Passing Interface standard. International Journal of High Performance Computing Applications, 12(1–2):1–299, 1998.

    Google Scholar 

  54. Message Passing Toolkit: MPI programmer’s manual, document number: 007-3687-010, 2003. Mountain View, CA, Silicon Graphics Inc.

    Google Scholar 

  55. Mpi papers. URL: http://www.mcs.anl.gov/mpi/papers.

    Google Scholar 

  56. K. Nakajima and H. Okuda. Parallel Iterative Solvers for Unstructured Grids Using and OpenMP/MPI Hybrid Programming Model for GeoFEM Platfrom on SMP Cluster Architectures. Lecture Notes in Computer Science, 2327:437–448, 2002.

    Article  Google Scholar 

  57. B. Nichols, D. Buttlar, and J. P. Farrel. Pthreads Programming. O’Reilly & Associates, Inc, 1996.

    Google Scholar 

  58. J. Nieplocha, R. Harrison, M. Krishnan, B. Palmer, and V. Tipparaju. Combining shared and distributed memory models: Evolution and recent advancements of the Global Array Toolkit. In Proceedings of POOHL’2002 workshop of ICS-2002, New York, NY, 2002.

    Google Scholar 

  59. J. Nieplocha, R. J. Harrison, and R. J. Littlefield. Global Arrays: A portable “shared memory” programming model for distributed memory computers. In Proceedings of Supercomputing 1994, SC94, pages 340–349, 1994.

    Google Scholar 

  60. J. Nieplocha, R. J. Harrison, and R. J. Littlefield. Global Arrays: A nonuniform memory access programming model for high-performance computers. The Journal of Supercomputing, 10:197–220, 1996.

    Article  Google Scholar 

  61. R. W. Numrich, J. Reid, and K. Kim. Writing a multigrid solver using Co-Array Fortran. In B. Kågströom, J. Dongarra, E. Elmroth, and J. Waśsniewski, editors, Applied Parallel Computing: Large Scale Scientific and Industrial Problems, volume 1541 of Lecture Notes in Computer Science, pages 390–399. Springer, 1998.

    Google Scholar 

  62. R. W. Numrich and J. K. Reid. Co-Array Fortran for parallel programming. ACM Fortran Forum, 17(2):1–31, 1998.

    Google Scholar 

  63. OpenMP Architecture Review Board. OpenMP Fortran Application Program Interface, Version 2.0. November 2000. URL: http://www.openmp.org/drupal/mp-documents/fspec20.pdf.

    Google Scholar 

  64. OpenMP Architecture Review Board. OpenMP C and C++ Application Program Interface, Version 2.0. March 2002. URL: http://www.openmp.org/drupal/mp-documents/cspec20.pdf.

    Google Scholar 

  65. OpenMP Architecture Review Board home page, http://www.openmp.org.

    Google Scholar 

  66. K. Parzyszek and R. A. Kendall. GPSHMEM: Application to kernel benchmarks. In Proceedings of the Fourteenth IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 2002), pages 404–409. ACTA Press, Anaheim, CA, 2002.

    Google Scholar 

  67. K. Parzyszek, J. Nieplocha, and R. A. Kendall. A generalized portable SHMEM library for high performance computing. In M. Guizani and X. Shen, editors, Proceedings of the IASTED Parallel and Distributed Computing and Systems 2000, pages 401–406. IASTED, Calgary, 2000.

    Google Scholar 

  68. Y. Saad. SPARSKIT: A basic tool kit for sparse matrix computations. Technical Report 90-20, NASA Ames Research Center, Moffett Field, CA, 1990.

    Google Scholar 

  69. H. Sakagami, H. Murai, Y. Seo, and M. Yokokawa. 14.9 TFLOPS three-dimensional fluid simulation for fusion science with HPF on the Earth Simulator. In Proceedings of Supercomputing, 2002.

    Google Scholar 

  70. Scali Library User’s Guide, 2002. Published by Scali, Oslo, Norway.

    Google Scholar 

  71. C. L. Seitz. The cosmic cube. Communications of the ACM, 28(1):22–33, Jan. 1985.

    Article  MathSciNet  Google Scholar 

  72. B. Smith, P. Bjørstad, and W. D. Gropp. Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge University Press, New York, 1996.

    MATH  Google Scholar 

  73. M. Snir, S. W. Otto, S. Huss-Lederman, D. W. Walker, and J. Dongarra. MPI: The Complete Reference. MIT Press, Cambridge, MA, 1995.

    Google Scholar 

  74. T. Straatsma, E. Aprà, T. Windus, W. E. d. J. E. J. Bylaska, S. Hirata, M. Valiev, M. T. Hackler, L. L. Pollack, R. J. Harrison, M. Dupuis, D. Smith, J. Nieplocha, V. Tipparaju, M. Krishnan, A. A. Auer, E. Brown, G. Cisneros, G. I. Fann, H. Fruchtl, J. Garza, K. Hirao, R. A. Kendall, J. Nichols, K. Tsemekhman, K. Wolinski, J. Anchell, D. Bernholdt, P. Borowski, T. Clark, D. Clerc, H. Dachsel, M. Deegan, K. K. Dyall, D. Elwood, E. Glendening, M. Gutowski, A. Hess, J. Jaffe, B. Johnson, J. Ju, R. Kobayashi, R. Kutteh, Z. Lin, R. Littlefield, X. Long, B. Meng, T. Nakajima, S. Niu, M. Rosing, G. Sandrone, M. Stave, H. Taylor, G. Thomas, J. van Lenthe, A. Wong, and Z. Zhang. NWChem, A computational chemistry package for parallel computers, Version 4.6, 2004. Pacific Northwest National Laboratory, Richland, WA.

    Google Scholar 

  75. R. Thakur, W. D. Gropp, and B. Toonen. Minimizing synchronization overhead in the implementation of MPI one-sided communication. In D. Kranzlmüller, P. Kacsuk, and J. Dongarra, editors, Recent Advances in Parallel Virtual Machine and Message Passing Interface, Lecture Notes in Computer Science, pages 57–67. Springer Verlag, 2004. 11th European PVM/MPI User’s Group Meeting, Budapest, Hungary.

    Google Scholar 

  76. The Cluster Enabled Omni OpenMP Compiler. URL: http://phase.hpcc.jp/Omni/Omni-doc/omni-scash.html.

    Google Scholar 

  77. The Open Group. System Interfaces and Headers, Issue 4, Version 2. 1992. URL: http://www.opengroup.org/public/pubs/catalog/c435.htm.

    Google Scholar 

  78. K. Yelick, L. Semenzato, G. Pike, C. Miyamoto, B. Liblit, A. Krishnamurthy, P. Hilfinger, S. Graham, D. Gay, P. Colella, and A. Aiken. Titanium: A high-performance Java dialect. Concurrency: Practice And Experience, 10(11–13):825–836, 1998.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Scalable Computing Laboratory, Ames Laboratory, USDOE, Ames, IA, 50011, USA

    Ricky A. Kendall & Masha Sosonkina

  2. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, 60439, USA

    William D. Gropp

  3. Supercomputing Institute, University of Minnesota, Minneapolis, MN, 55455, USA

    Robert W. Numrich

  4. California Institute of Technology, Pasadena, CA, 91125, USA

    Thomas Sterling

Authors
  1. Ricky A. Kendall
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Masha Sosonkina
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. William D. Gropp
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert W. Numrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thomas Sterling
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Simula Research Laboratory, P.O. Box 134, 1325, Lysaker, Fornebu, Norway

    Are Magnus Bruaset  & Aslak Tveito  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kendall, R.A., Sosonkina, M., Gropp, W.D., Numrich, R.W., Sterling, T. (2006). Parallel Programming Models Applicable to Cluster Computing and Beyond. In: Bruaset, A.M., Tveito, A. (eds) Numerical Solution of Partial Differential Equations on Parallel Computers. Lecture Notes in Computational Science and Engineering, vol 51. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-31619-1_1

Download citation

Publish with us

Policies and ethics

Search

Navigation