Abstract
Communications overhead is one of the most important factors affecting performance in message-passing multicomputers. We present evidence that there exists communications locality, and that this locality is “structured”. We propose a number of heuristics that can be used to “predict” the target of subsequent communication requests. Communication latency is hidden through reconfiguring the network concurrently to the computation. Quantitative results obtained from standard parallel benchmarks run on IBM SP systems are also presented.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Afsahi, A. and Dimopoulos, N.J. (1997). Collective Communications on a Reconfigurable Optical Interconnect. Proc. of the Int’l Conf. on Principles of Distributed Systems, pp. 167–181.
Afsahi, A. and Dimopoulos, N.J. (1999a). Hiding Communication Latency in Reconfigurable Message-Passing Environments. Proc. of the IPPS/SPDP 1999, 13th Int’l Parallel Processing Symp. and 10th Symp. on Parallel and Distributed Processing, pp. 55–60.
Afsahi, A. and Dimopoulos, N.J. (1999b). Hiding Communication Latency in Reconfigurable Message-Passing Environments. Technical report ECE-99-3, Dept. of Electrical and Computer Engineering, Univ. of Victoria.
Bailey, D.H. Barszcz, E., Dagum, L. and Simon, H.D. (1994). NAS Parallel Benchmark Result 3-94. Proc. of the Scalable High-Performance Comp. Conf., pp. 111–120.
Bourdin H., Ferriera, A., and Marcus, K. (1995). A Comparative Study of One-to-Many WDM Lightwave Interconnection Networks for Multiprocessors. Proc. of the 2nd Int’l Conf. on Massively Parallel Processing using Optical Interconnections, pp. 257–263.
Dao, B.V., Yalamanchili, S., and Duato, J. (1997). Architectural Support for Reducing Communication Overhead in Multiprocessor Interconnection Networks. Proc. 3rd Int’l Symp. on High Perf. Comp. Architecture, pp. 343–352.
Dongarra, J.J. and Dunigan, T. (1997). Message-Passing Performance of Various Computers. Concurrency, Vol. 9, No. 10, pp. 915–926.
Eicken, T.V., Culler, D.E., Goldstein, S.C., and Schauser, K.E. (1992). Active Messages: A Mechanism for Integrated Communication and Computation. Proc. of the 19th Ann. Int’l Symp. on Computer Architecture, pp. 256–265.
Hioki, S. (1996). Construction of Staples in Lattice Gauge Theory on a Parallel Computer. Parallel Computing, 22–10, pp. 1335–1344.
Kim, J. and Lilja, D.J. (1998). Characterization of Communication Patterns in Message-Passing Parallel Scientific Application Programs. Proc. of the Workshop on Communication, Architecture, and Applications for Network-based Parallel Computing. Int’ Symp. on High Performance Computer Architecture, pp. 202–216.
de Lahaut, D.G. and Germain C. (1994). Static Communications in Parallel Scientific Programs. Proc. of PARLE’94, Parallel Architecture and Languages.
Louri, A. and Sung, H.K. (1994). An Optical Multi-Mesh Hypercube: A Scalable Optical Interconnection Network for Massively Parallel Computing. J. of Lightwave Technology, Vol. 12, No. 4, pp. 704–716.
Marchand, P.J., Krishnamoorthy, A.V., Yayla, G.I., Esener, S.C., and Efron, U. (1997). Optically Augmented 3-D Computer: System Technology and Architecture. J. of Parallel and Distributed Computing, Special Issue on Optical Interconnects, Feb. 25, pp. 20–35.
Message Passing Interface Forum: MPI: A Message-Passing Interface Standard. (1995). Version 1.1.
Mowry, T. and Gupta, A. (1991). Tolerating Latency Through Software-Controlled Prefetching in Shared-Memory Multiprocessors. J. of Parallel and Distributed Computing, 12(2), pp. 87–106.
Nordin, R.A., Levi, A.F., Nottenburg, R.N., O’Gorman, J., Tanbun-Ek, T., and Logan, R.A. (1992). A System Perspective on Digital Interconnection Technology. IEEE J. of Lightwave Technology, Vol. 10, pp. 801–827.
Pakin, S., Lauria, M., and Chien, A. (1995). High Perf. Messaging on Workstation: Illinois Fast Messages (FM) for Myrinet. Proc. of Supercomputing’95.
Panajotov, K., Nieuborg, N., Goulet, A., Veretennicoff, I., and Thienpont, H. (1998). A Free-Space Reconfigurable Optical Interconnection based on Polarization-Switching VCSEL’s and Polarization-Selective Diffractive Optical Channels. Proc. of Optics in Computing, pp. 151–154.
Sakr, M.F., Levitan, S.P., Chiarulli, D.M., Horne, B.G., and Giles, C.L. (1997). Predicting Multiprocessor Memory Access Patterns with Learning Models. Proc. of the 14th Int’ Conf. on Machine Learning. pp. 305–312.
Worley, P.H. and Foster, I.T. (1994). Parallel Spectral Transform Shallow Water Model: A Runtime-tunable parallel benchmark code. Proc. of the Scalable High Performance Computing Conference. pp. 207–214.
Yayla, G.I., Marchand, P.J., and Esener, S.C. (1998). Speed and Energy Analysis of Digital Interconnections: Comparison of On-chip, Off-chip and Free-Space Technologies. App. Optics, Vol. 37, No. 2, pp. 205–227.
Yuan X., Melhem, R., and Gupta R. (1996). Compiled Communication for All-Optical TDM Networks. Proc. of Supercomputing’96.
Zhang. Z. and Torrellas, J. (1995). Speeding Up Irregular Applications in Shared-Memory Multiprocessors: Memory Binding and Group Prefetching. Proc. of the 22nd Ann. Symp. on Computer Architecture, pp. 188–199.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Kluwer Academic Publishers
About this chapter
Cite this chapter
Afsahi, A., Dimopoulos, N.J. (2002). Communication Latency Hiding in Reconfigurable Message-Passing Environments: Quantitative Studies. In: Pollard, A., Mewhort, D.J.K., Weaver, D.F. (eds) High Performance Computing Systems and Applications. The International Series in Engineering and Computer Science, vol 541. Springer, Boston, MA. https://doi.org/10.1007/0-306-47015-2_19
Download citation
DOI: https://doi.org/10.1007/0-306-47015-2_19
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-7923-7774-0
Online ISBN: 978-0-306-47015-8
eBook Packages: Springer Book Archive