Abstract
After MapReduce Parallel refactoring, there occurred an enormous difference between the latest version of MapReduce parallel and the old version on framework. At first, this paper introduces the framework of the old version of MapReduce parallel, the procedure of implementing tasks and how to schedule tasks and resources allocations, etc. Also, it points out the limitation of the old parallel. Then, it explained a framework of the new version of MapReduce parallel, task scheduling and resource allocation and so on. From the perspective of the framework, task scheduling, and resource allocation, it compared these two different versions, putting forward the advantages of the renewed framework. New-generation of MapReduce has a framework YARN, which is sharing model, in other words, it’s an application program that enables various compiling of computing structures to run on the same cluster. Meanwhile, it makes operation and maintenance more smooth and makes full use of cluster resource.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Pace, M.F.: BSP vs MapReduce. Procedia Comput. Sci. 9, 246–255 (2012)
Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: OSDI 2004 Proceedings of the 6th Conference on Symposium on Operating Systems Design & Implementation, vol. 6, p. 10 (2004)
Ghemawat, S., Gobioff, H., Leung, S.T.: The Google file system. ACM SIGOPS Oper. Syst. Rev. 37(5), 29–43 (2003)
Li, J.J., Cui, J., Wang, D., et al.: Survey of MapReduce parallel programming model. Acta Electronica Sinica 39(11), 2635–2642 (2011)
Jiang, D., Zheng, Z., Jiexin, Z., et al.: Survey of MapReduce parallel programming model. Comput. Sci. 42(S1), 537–541, 564 (2015)
Liyun, K., Xiaoyue, W., RuJiang, B.: Analysis of MapReduce principle and its main implementation platforms. New Technol. Libr. Inf. Serv. 2, 60–67 (2012)
Han, H.: Researches on optimization of resource allocation for mapreduce scheduling. South China University of Technology, Guangzhou (2013)
Shuqi, X.: The research on high performance task scheduling technology based on mapreduce in cloud computing. Beijing University of Technology, Beijing (2013)
Wang, B., Chen, L.: Analysis and application of mechanism of Hadoop RPC communication. J. Xi’an Univ. Posts Telecommun. 6, 74–77 (2012)
Xiang, R., Lingjuan, L.: A method for frequent set mining based on MapReduce. J. Xi’an Univ. Posts and Telecommun. 4, 37–39, 43 (2011)
Wentao, Z., Zhongliang, A., Zhonglin, L., et al.: A kind of high-priority job scheduler implementation based on YARN. Comput. Eng. Softw. 3, 84–88 (2016)
Chuntao, D., Wenting, L., Qingxia, C., et al.: Research on the framework and resource scheduling mechanisms of Hadoop YARN. Inf. Commun. Technol. 1, 77–84 (2015)
http://hadoop.apache.org/. [EB/OL] (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, X., Liu, Q. (2018). A Comparative Summary of the Latest Version of MapReduce Parallel and Old Version from the Perspective of Framework. In: Li, K., Li, W., Chen, Z., Liu, Y. (eds) Computational Intelligence and Intelligent Systems. ISICA 2017. Communications in Computer and Information Science, vol 874. Springer, Singapore. https://doi.org/10.1007/978-981-13-1651-7_9
Download citation
DOI: https://doi.org/10.1007/978-981-13-1651-7_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1650-0
Online ISBN: 978-981-13-1651-7
eBook Packages: Computer ScienceComputer Science (R0)