Skip to main content
SpringerLink
Log in

Parallelization of 3D MPDATA Algorithm Using Many Graphics Processors

  • Conference paper
  • First Online:
Parallel Computing Technologies (PaCT 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9251))

Included in the following conference series:

Abstract

EULAG (Eulerian/semi-Lagrangian fluid solver) is an established numerical model for simulating thermo-fluid flows across a wide range of scales and physical scenarios. The multidimensional positive definite advection transport algorithm (MPDATA) is among the most time-consuming components of EULAG. In this study, we focus on adapting the 3D MPDATA computations to clusters with graphics processors. Our approach is based on a hierarchical decomposition including the level of cluster, as well as an optimized distribution of computations between GPU resources within each node. To implement the resulting computing scheme, the MPI standard is used across nodes, while CUDA is applied inside nodes. We present performance results for the 3D MPDATA code running on the NVIDIA GeForce GTX TITAN graphics card, as well as on the Piz Daint cluster equipped with NVIDIA Tesla K20x GPUs. In particular, the sustained performance of 138 Gflop/s is achieved for a single GPU, which scales up to more than 11 Tflop/s for 256 GPUs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ciznicki, M., Kopta, P., Kulczewski, M., Kurowski, K., Gepner, P.: Elliptic solver performance evaluation on modern hardware architectures. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 155–165. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  2. Datta, K., Kamil, S., Williams, S., Oliker, L., Shalf, J., Yelick, K.: Optimization and performance modeling of stencil computations on modern microprocessors. SIAM Rev. 51(1), 129–159 (2009)

    Article  MATH  Google Scholar 

  3. Hager, G., Wellein, G.: Introduction to High Performance Computing for Science and Engineers. CRC Press, Boca Raton (2011)

    Google Scholar 

  4. Khajeh-Saeed, A., et al.: Computational fluid dynamics simulations using many graphics processors. Comput. Sci. Eng. 14(3), 10–19 (2012)

    Article  Google Scholar 

  5. Krotkiewicz, M., Dabrowski, M.: Efficient 3D stencil computations using CUDA. Parallel Comput. 39, 533–548 (2013)

    Article  MathSciNet  Google Scholar 

  6. Kurowski, K., Kulczewski, M., Dobski, M.: Parallel and GPU based strategies for selected CFD and climate modeling models. Environ. Sci. Eng. 3, 735–747 (2011)

    Article  Google Scholar 

  7. Nguyen, A., Satish, N., Chhugani, J., Changkyu, K., Dubey, P.: 3.5-D blocking optimization for stencil computations on modern CPUs and GPUs. In: Proceedings of 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–13 (2010)

    Google Scholar 

  8. Piotrowski, Z., Wyszogrodzki, A., Smolarkiewicz, P.: Towards petascale simulation of atmospheric circulations with soundproof equations. Acta Geophys. 59, 1294–1311 (2011)

    Article  Google Scholar 

  9. Prusa, J., Smolarkiewicz, P., Wyszogrodzki, A.: EULAG, a computational model for multiscale flows. Comput. Fluids 37, 1193–1207 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  10. Rivera, G., Tseng, Ch.-W.: Tiling optimizations for 3D scientific computations. In: SC 2000 Proceedings of ACM/IEEE Conference on Supercomputing (2000)

    Google Scholar 

  11. Rojek, K., Szustak, L.: Parallelization of EULAG model on multicore architectures with GPU accelerators. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2011, Part II. LNCS, vol. 7204, pp. 391–400. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  12. Rojek, K., Szustak, L., Wyrzykowski, R.: Performance analysis for stencil-based 3D MPDATA algorithm on GPU architecture. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 145–154. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  13. Rojek, K., Ciznicki, M., Rosa, B., Kopta, P., Kulczewski, M., Kurowski, K., Piotrowski, Z., Szustak, L., Wojcik, D., Wyrzykowski, R.: Adaptation of fluid model EULAG to graphics processing unit architecture. Concurrency Comput. Pract. Experience 27(4), 937–957 (2015)

    Article  Google Scholar 

  14. Smolarkiewicz, P.: Multidimensional positive definite advection transport algorithm: an overview. Int. J. Numer. Meth. Fluids 50, 1123–1144 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  15. Szustak, L., Rojek, K., Gepner, P.: Using intel xeon phi coprocessor to accelerate computations in MPDATA algorithm. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2013, Part I. LNCS, vol. 8384, pp. 582–592. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  16. Wójcik, D.K., Kurowski, M.J., Rosa, B., Ziemiański, M.Z.: A study on parallel performance of the EULAG F90/95 code. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2011, Part II. LNCS, vol. 7204, pp. 419–428. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  17. Wyrzykowski, R., Szustak, L., Rojek, K.: Parallelization of 2D MPDATA EULAG algorithm on hybrid architectures with GPU accelerators. Parallel Comput. 40(8), 425–447 (2014)

    Article  MathSciNet  Google Scholar 

  18. Wyrzykowski, R., Szustak, L., Rojek, K., Tomas, A.: Towards efficient decomposition and parallelization of MPDATA on hybrid CPU-GPU cluster. In: Lirkov, I., Margenov, S., Waśniewski, J. (eds.) LSSC 2013. LNCS, vol. 8353, pp. 457–464. Springer, Heidelberg (2014)

    Google Scholar 

  19. Wyszogrodzki, A.A., Piotrowski, Z.P., Grabowski, W.W.: Parallel implementation and scalability of cloud resolving EULAG model. In: Wyrzykowski, R., Dongarra, J., Karczewski, K., Waśniewski, J. (eds.) PPAM 2011, Part II. LNCS, vol. 7204, pp. 252–261. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  20. Xue, W., Yang, C., Fu, H., Xu, Y., Liao, J., Gan, L., Lu, Y., Ranjan, R., Wang, L.: Ultra-scalable CPU-MIC acceleration of mesoscale atmospheric modeling on Tianhe-2. IEEE Trans. Comput. (2014). doi:10.1109/TC.2014.2366754 (to appear)

  21. GPUDirect RDMA. http://docs.nvidia.com/cuda/gpudirect-rdma/index.html

  22. NVIDIA GeForce GTX TITAN Specification. http://www.geforce.com/hardware/ desktop-gpus/geforce-gtx-titan/specifications

  23. PizDaint & PizDora. http://www.cscs.ch/computers/piz_daint/index.html

  24. Top 500 Supercomputing Sites. http://www.top500.org

Download references

Acknowledgments

This work was supported by the Polish National Science Centre under grant no. UMO-2011/03/B/ST6/03500, and National Centre for Research and Development under grant no. POIG.02.03.00-24-093/13-00, as well as by the grant from the Swiss National Supercomputing Centre (CSCS) under project ID d25.

Author information

Authors and Affiliations

  1. Czestochowa University of Technology, Dabrowskiego 69, 42-201, Czestochowa, Poland

    Krzysztof Rojek & Roman Wyrzykowski

Authors
  1. Krzysztof Rojek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Roman Wyrzykowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roman Wyrzykowski .

Editor information

Editors and Affiliations

  1. Russian Academy of Sciences, Novosibirsk, Russia

    Victor Malyshkin

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Rojek, K., Wyrzykowski, R. (2015). Parallelization of 3D MPDATA Algorithm Using Many Graphics Processors. In: Malyshkin, V. (eds) Parallel Computing Technologies. PaCT 2015. Lecture Notes in Computer Science(), vol 9251. Springer, Cham. https://doi.org/10.1007/978-3-319-21909-7_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-21909-7_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-21908-0

  • Online ISBN: 978-3-319-21909-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation