Abstract
In a parallel computing scenario, a complex task is typically split among many computing nodes, which are engaged to perform portions of the task in a parallel fashion. Except for a very limited class of application, computing nodes need to coordinate with each other in order to carry out the parallel execution in a consistent way. As a consequence, a synchronization overhead arises, which can significantly impair the overall execution performance. Typically, synchronization is achieved by adopting a centralized synchronization barrier involving all the computing nodes. In many application domains, though, such kind of global synchronization can be relaxed and a lean synchronization schema, namely local synchronization, can be exploited. By using local synchronization, each computing node needs to synchronize only with a subset of the other computing nodes. In this work, we evaluate the performance of the local synchronization mechanism when compared to the global synchronization approach. As a key performance indicator, the efficiency index is considered, which is defined as the ratio between useful computation time and total computation time, including the synchronization overhead. The efficiency trend is evaluated both analytically and through numerical simulation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In the case that \(l_i\) are independent (not necessarily identically distributed) random variables with a continuous distribution having support on a bounded interval, the mean computation time \(\lim _{k \rightarrow \infty } \mathbf {E}(T_i(k))/k\) is always a constant, irrespective of the number of nodes N.
- 2.
It is well-known that the right part is the approximation of the expected maximum of N i.i.d. random variables with exponential distribution equal to \(\mu \sum _{i=1}^N i^{-1}\), with \(\sum _{i=1}^N i^{-1}\) being the \(N^{th}\) harmonic number.
References
Cicirelli, F., Forestiero, A., Giordano, A., Mastroianni, C.: Transparent and efficient parallelization of swarm algorithms. ACM Trans. Auton. Adapt. Syst. 11(2), 14 (2016)
Cicirelli, F., Forestiero, A., Giordano, A., Mastroianni, C.: Parallelization of space-aware applications: modeling and performance analysis. J. Netw. Comput. Appl. 122, 115–127 (2018)
Gong, Z., Tang, W., Bennett, D.A., Thill, J.C.: Parallel agent-based simulation of individual-level spatial interactions within a multicore computing environment. Int. J. Geogr. Inf. Sci. 27(6), 1152–1170 (2013)
Tang, W., Bennett, D.A., Wang, S.: A parallel agent-based model of land use opinions. J. Land Use Sci. 6(2–3), 121–135 (2011)
Mastroianni, C., Cesario, E., Giordano, A.: Efficient and scalable execution of smart city parallel applications. Concurrency Comput. Pract. Experience 30(20), e4258 (2018)
Zheng, Y., Capra, L., Wolfson, O., Yang, H.: Urban computing: concepts, methodologies, and applications. ACM Trans. Intell. Syst. Technol. 5(3), 1–55 (2014)
Blecic, I., Cecchini, A., Trunfio, G.A., Verigos, E.: Urban cellular automata with irregular space of proximities. J. Cell. Automata 9(2–3), 241–256 (2014)
Atzori, L., Iera, A., Morabito, G.: The internet of things: a survey. Comput. Netw. 54(15), 2787–2805 (2010)
Lee, I., Lee, K.: The Internet of Things (IoT): applications, investments, and challenges for enterprises. Bus. Horiz. 58(4), 431–440 (2015)
Krishnan, Y.N., Bhagwat, C.N., Utpat, A.P.: Fog computing–network based cloud computing. In: 2nd IEEE International Conference on Electronics and Communication Systems (ICECS), pp. 250–251 (2015)
Hu, P., Dhelim, S., Ning, H., Qiu, T.: Survey on fog computing: architecture, key technologies, applications and open issues. J. Netw. Comput. Appl. 98, 27–42 (2017)
Ekanayake, J., Fox, G.: High performance parallel computing with clouds and cloud technologies. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) CloudComp 2009. LNICSSTE, vol. 34, pp. 20–38. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-12636-9_2
Fujimoto, R.: Parallel and Distributed Simulation Systems. Wiley, New York (2000)
Cicirelli, F., Forestiero, A., Giordano, A., Mastroianni, C., Razumchik, R.: Global and local synchronization in parallel space-aware applications. In: 32nd European Conference on Modelling and Simulation, ECMS 2018, Wilhelmshaven, Germany, May 2018
Borovkov, A.A.: Ergodicity and stability theorems for a class of stochastic equations and their applications. Theory Probab. Appl. 23(2), 227–247 (1979)
David, H.A., Nagaraja, H.N.: Order Statistics, 3rd edn. Wiley, New York (2003)
Ang, A.S., Tang, W.: Probability Concepts in Engineering Planning and Design, vol. II. Rainbow Bridge (1984)
Madala, S., Sinclair, J.B.: Performance of synchronous parallel algorithms with regular structures. IEEE Trans. Parallel Distrib. Syst. 2(1), 105–116 (1991)
Whitt, W., Crow, C., Goldberg, D.: Two-moment approximations for maxima. Oper. Res. 55(3), 532–548 (2007)
Baccelli, F., Konstantopoulos, P.: Estimates of cycle times in stochastic petri nets. In: Karatzas, I., Ocone, D. (eds.) Applied Stochastic Analysis, pp. 1–20. Springer, Heidelberg (1992). https://doi.org/10.1007/BFb0007044
Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: modeling the characteristics of rigid jobs. J. Parallel Distrib. Comput. 63(11), 1105–1122 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Cicirelli, F., Giordano, A., Mastroianni, C. (2020). Improving Efficiency in Parallel Computing Leveraging Local Synchronization. In: Sergeyev, Y., Kvasov, D. (eds) Numerical Computations: Theory and Algorithms. NUMTA 2019. Lecture Notes in Computer Science(), vol 11973. Springer, Cham. https://doi.org/10.1007/978-3-030-39081-5_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-39081-5_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39080-8
Online ISBN: 978-3-030-39081-5
eBook Packages: Computer ScienceComputer Science (R0)