Abstract
The MPI-2 standard defines a class of file access routines providing a shared file pointer. All processes using those routines update the same file pointer when accessing the file. Coordination between ranks happens implicitly in the MPI library, relieving the application developer of this responsibility. The shared file pointer routines, however, have found little interest from developers because of several issues ranging from routine usability and portability to performance. We consider the use of these routines in the HDF5 library, a high-level I/O library built on top of MPI, and in Vampir, a performance analysis toolkit. We highlight some of the reasons preventing their adoption and discuss how these routines could be modified to increase their usability. We also propose a novel implementation using the new MPI one-sided routines provided by the upcoming MPI-3.0 standard.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chaarawi, M., Gabriel, E., Keller, R., Graham, R.L., Bosilca, G., Dongarra, J.J.: OMPIO: A Modular Software Architecture for MPI I/O. In: Cotronis, Y., Danalis, A., Nikolopoulos, D.S., Dongarra, J. (eds.) EuroMPI 2011. LNCS, vol. 6960, pp. 81–89. Springer, Heidelberg (2011)
Cope, J., Iskra, K., Kimpe, D., Ross, R.: Portable and Scalable MPI Shared File Pointers. In: Cotronis, Y., Danalis, A., Nikolopoulos, D.S., Dongarra, J. (eds.) EuroMPI 2011. LNCS, vol. 6960, pp. 312–314. Springer, Heidelberg (2011)
Gropp, W., Lusk, E., Doss, N., Skjellum, A.: A high-performance, portable implementation of the MPI message passing interface standard. Parallel Computing 22(6), 789–828 (1996)
Hierarchical Data Format Group: HDF5 Reference Manual, release 1.6.3, National Center for Supercomputing Application (NCSA), University of Illinois at Urbana-Champaign (September 2004)
Ilsche, T., Schuchart, J., Cope, J., Kimpe, D., Jones, T., Knüpfer, A., Iskra, K., Ross, R., Nagel, E.W., Poole, S.: Enabling event tracing at leadership-class scale through I/O forwarding middleware. In: Proceedings of the 21st International ACM Symposium on High Performance Parallel and Distributed Computing (HPDC 2012) (to appear, 2012)
Knüpfer, A., Brunst, H., Doleschal, J., Jurenz, M., Lieber, M., Mickler, H., Müller, M.S., Nagel, W.E.: The Vampir performance analysis toolset. In: Resch, M., Keller, R., Himmler, V., Krammer, B., Schulz, A. (eds.) Tools for High Performance Computing, pp. 139–155. Springer, Heidelberg (2008)
Kulkarni, K., Gabriel, E.: Evaluating Algorithms for Shared File Pointer Operations in MPI I/O. In: Allen, G., Nabrzyski, J., Seidel, E., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2009, Part I. LNCS, vol. 5544, pp. 280–289. Springer, Heidelberg (2009)
Lang, S., Latham, R., Ross, R., Kimpe, D.: Interfaces for coordinated access in the file system. In: IEEE International Conference on Cluster Computing and Workshops, CLUSTER 2009, pp. 1–9 (2009)
Latham, R., Ross, R., Thakur, R.: Implementing MPI-IO atomic mode and shared file pointers using MPI one-sided communication. Int. J. High Perform. Comput. Appl. 21(2), 132–143 (2007)
Li, J., Liao, W.K., Choudhary, A., Ross, R., Thakur, R., Gropp, W., Latham, R., Siegel, A., Gallagher, B., Zingale, M.: Parallel netCDF: A high-performance scientific I/O interface. In: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, SC 2003, p. 39. ACM, New York (2003), http://doi.acm.org/10.1145/1048935.1050189
Message Passing Interface Forum: MPI-2: Extensions to the Message Passing Interface (July 1997), http://www.mpi-forum.org/
Ross, R., Latham, R., Gropp, W., Thakur, R., Toonen, B.: Implementing MPI-IO atomic mode without file system support. In: IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2005, vol. 2, pp. 1135–1142 (May 2005)
Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, pp. 23–32 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chaarawi, M., Dinan, J., Kimpe, D. (2012). On the Usability of the MPI Shared File Pointer Routines. In: Träff, J.L., Benkner, S., Dongarra, J.J. (eds) Recent Advances in the Message Passing Interface. EuroMPI 2012. Lecture Notes in Computer Science, vol 7490. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33518-1_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-33518-1_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33517-4
Online ISBN: 978-3-642-33518-1
eBook Packages: Computer ScienceComputer Science (R0)