Abstract
Large-scale parallel data analysis, where global information from a variety of problem domains is resolved in a distributed memory space, relies on communication. Three communication algorithms motivated by data analysis workloads—merge based reduction, swap based reduction, and neighborhood exchange—are presented, and their performance is benchmarked. These algorithms communicate custom data types among blocks assigned to processes in flexible ways, and their performance is optimized by tunable parameters. Performance is compared with an MPI implementation and with previous communication algorithms on an IBM Blue Gene/P supercomputer at a variety of message sizes and process counts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kumar, S., Dozsa, G., Berg, J., Cernohous, B., Miller, D., Ratterman, J., Smith, B., Heidelberger, P.: Architecture of the Component Collective Messaging Interface. In: Lastovetsky, A., Kechadi, T., Dongarra, J. (eds.) EuroPVM/MPI 2008. LNCS, vol. 5205, pp. 23–32. Springer, Heidelberg (2008)
Sack, P., Gropp, W.: Faster Topology-Aware Collective Algorithms through Non-Minimal Communication. In: Proceedings of the 17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2012, pp. 45–54. ACM, New York (2012)
Ma, K.-L., Painter, J.S., Hansen, C.D., Krogh, M.F.: Parallel Volume Rendering Using Binary-Swap Compositing. IEEE Computer Graphics and Applications 14(4), 59–68 (1994)
Peterka, T., Ross, R., Kendall, W., Gyulassy, A., Pascucci, V., Shen, H.-W., Lee, T.-Y., Chaudhuri, A.: Scalable Parallel Building Blocks for Custom Data Analysis. In: Proceedings of the 2011 IEEE Large Data Analysis and Visualization Symposium, LDAV 2011, Providence, RI (2011)
Peterka, T., Ross, R., Nouanesengsy, B., Lee, T.-Y., Shen, H.-W., Kendall, W., Huang, J.: A Study of Parallel Particle Tracing for Steady-State and Time-Varying Flow Fields. In: Proceedings of IPDPS 2011, Anchorage AK (2011)
Xu, L., Lee, T.Y., Shen, H.W.: An Information-Theoretic Framework for Flow Visualization. IEEE Transactions on Visualization and Computer Graphics 16, 1216–1224 (2010)
Gyulassy, A., Peterka, T., Pascucci, V., Ross, R.: Characterizing the Parallel Computation of Morse-Smale Complexes. In: Proceedings of IPDPS 2012, Shanghai, China (2012)
Schaap, W.E.: DTFE: The Delaunay Tesselation Field Estimator, University of Groningen, The Netherlands, Ph.D. Dissertation (2007)
Chen, J., Silver, D., Jiang, L.: The Feature Tree: Visualizing Feature Tracking in Distributed AMR Datasets. In: Proceedings of the 2003 IEEE Symposium on Parallel and Large-Data Visualization and Graphics, PVG 2003. IEEE Computer Society, Washington, DC (2003)
Peterka, T., Goodell, D., Ross, R., Shen, H.W., Thakur, R.: A Configurable Algorithm for Parallel Image-Compositing Applications. In: Proceedings of SC 2009, Portland OR (2009)
Porter, T., Duff, T.: Compositing Digital Images. In: Proceedings of 11th Annual Conference on Computer Graphics and Interactive Techniques, pp. 253–259 (1984)
Moreland, K., Kendall, W., Peterka, T., Huang, J.: An Image Compositing Solution at Scale. In: Proceedings of SC 2011, Seattle, WA (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Peterka, T., Ross, R. (2012). Versatile Communication Algorithms for Data Analysis. In: Träff, J.L., Benkner, S., Dongarra, J.J. (eds) Recent Advances in the Message Passing Interface. EuroMPI 2012. Lecture Notes in Computer Science, vol 7490. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33518-1_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-33518-1_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33517-4
Online ISBN: 978-3-642-33518-1
eBook Packages: Computer ScienceComputer Science (R0)