Abstract
Today, data is experiencing a real explosion in terms of quantity and nature. They also play a very important role in reading the past, managing the present, and planning for the future. To this end, organizations consider them a treasure and are always looking for a veritable way to manage and exploit them. And since the old data management system has some weaknesses compared to Bigdata, and designers have designed systems called NoSQL to overcome these weaknesses, there is a critical need for migrating data to the new NoSQL system, to keep their old data and take advantage of the power of NoSQL systems. To meet this need, several approaches have been developed by researchers to ensure this migration. The problem is that these approaches transform data, structures, or both. These approaches at best mimic a relational database with its limitations and constraints in another NoSQL environment, which loses much of its efficiency when applying relational processing in the migration result database. In this article, we will develop a smart approach that aims to migrate the three essential parts of relational databases: the first is data, the second is structure, and the third is the semantic component, using an ETL to be defined. The solution respects the analytical part of the relational systems that we have developed and the transformations of the structure and semantics of the relational system towards the NoSQL system, to develop our model of our ETL, which consists of three phases: data extraction, transforming this data, and uploading it to the destination system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
The digital universe of opportunities. https://www.emc.com/leadership/digitaluniverse/2014iview/executive-summary.htm
Data is eating the world. https://whatsthebigdata.com/2017/04/18/idc-163-trillion-gigabytesof-data-will-be-created-in-2025/
Sokolova, M.V., Gómez, F.J., Borisoglebskaya, L.N.: Migration from an SQL to a hybrid SQL/NoSQL data model. J. Manag. Anal. 7(1), 1–11 (2020)
de Oliveira, V.F., de Oliveira Pessoa, M.A., Junqueira, F., Miyagi, P.E.: SQL and NoSQL Databases in the Context of Industry 4.0 (2021)
Gamero, D., Dugenske, A., Kurfess, T., Saldana, C., Fu, K.: SQL and NoSQL databases for cyber physical production systems in internet of things for manufacturing (IoTfM). In: International Manufacturing Science and Engineering Conference, vol. 85079, p. V002T07A014. American Society of Mechanical Engineers (2021
Dai, J.: SQL to NoSQL: what to do and how. In: IOP Conference Series: Earth and Environmental Science, vol. 234, no. 1, p. 012080. IOP Publishing (2019
Chang, M.-L.E., Chua, H.: SQL and NoSQL database comparison: from performance perspective in supporting semi-structured data. In: Arai, K., Kapoor, S., Bhatia, R. (eds.) Advances in Information and Communication Networks: Proceedings of the 2018 Future of Information and Communication Conference (FICC), Vol. 1, pp. 294–310. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-03402-3_20
Flores, A., Ramírez, S., Toasa, R., Vargas, J., Urvina-Barrionuevo, R., Lavin, J.M.: Performance evaluation of NoSQL and SQL queries in response time for the E-government. In: 2018 International Conference on eDemocracy & eGovernment (ICEDEG), pp. 257–262. IEEE (2018
Reetishwaree, S., Hurbungs, V.: Evaluating the performance of SQL and NoSQL databases in an IoT environment. In: 2020 3rd International Conference on Emerging Trends in Electrical, Electronic and Communications Engineering (ELECOM), pp. 229–234. IEEE (2020)
Sahatqija, K., Ajdari, J., Zenuni, X., Raufi, B., Ismaili, F.: Comparison between relational and NOSQL databases. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 0216–0221. IEEE (2018
Mukherjee, S.: The battle between NoSQL Databases and RDBMS. Available at SSRN 3393986 (2019)
Sokolova, M.V., Gómez, F.J., Borisoglebskaya, L.N.: Migration from an SQL to a hybrid SQL/NoSQL data model. J. Manag. Anal. 7(1), 1–11 (2020)
Ramzan, S., Bajwa, I.S., Ramzan, B., Anwar, W.: Intelligent data engineering for migration to NoSQL based secure environments. IEEE Access 7, 69042–69057 (2019)
Hanine, M., Bendarag, A., Boutkhoum, O.: Data migration methodology from relational to NoSQL databases. World Acad. Sci. Eng. Technol. Int. J. Comput. Electr. Autom. Control Inf. Eng. 9(12), 2369–2373 (2016)
Stanescu, L., Brezovan, M., Burdescu, D.D.: An algorithm for mapping the relational databases to mongodb-a case study. Int. J. Comput. Sci. Appl. 14(1), 1–16 (2017)
Sharma, K., Attar, V.: Generalized big data test framework for ETL migration. In: 2016 International Conference on Computing, Analytics and Security Trends (CAST), pp. 528–532. IEEE (2016)
Hu, H., Wen, Y., Chua, T.S., Li, X.: Toward scalable systems for big data analytics: a technology tutorial. IEEE Access 2, 652–687 (2014)
DB-Engines - Knowledge Base of Relational and NoSQL Database Management Systems (2015). https://db-engines.com/en/
Rocha, L., Vale, F., Cirilo, E., Barbosa, D., Mourão, F.: A framework for migrating relational datasets to NoSQL. Procedia Comput. Sci. 51, 2593–2602 (2015)
Yangui, R., Nabli, A., Gargouri, F.: Automatic transformation of data warehouse schema to NoSQL data base: comparative study. Procedia Comput. Sci. 96, 255–264 (2016)
Liao, Y.T., et al.: Data adapter for querying and transformation between SQL and NoSQL database. Futur. Gener. Comput. Syst. 65, 111–121 (2016)
Bansel, A., González-Vélez, H., Chis, A.E.: Cloud-based NoSQL data migration. In: 2016 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP), pp. 224–231. IEEE (2016)
Atzeni, P., Bugiotti, F., Cabibbo, L., Torlone, R.: Data modeling in the NoSQL world. Comput. Stand. Interf. 67, 103149 (2020)
Vathy-Fogarassy, Á., Hugyák, T.: Uniform data access platform for SQL and NoSQL database systems. Inf. Syst. 69, 93–105 (2017)
Hamouda, S., Zainol, Z.: Document-oriented data schema for relational database migration to NoSQL. In: 2017 International Conference on Big Data Innovations and Applications (innovate-data), pp. 43–50. IEEE (2017)
Bante, P.M., Rajeswari, K.: Big data analytics using hadoop map reduce framework and data migration process. In: 2017 International Conference on Computing, Communication, Control and Automation (ICCUBEA), pp. 1–5. IEEE (2017)
https://www.dummies.com/programming/big-data/hadoop/acid-versus-base-data-stores/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Erraji, A., Maizate, A., Ouzzif, M. (2022). New ETL Process for a Smart Approach of Data Migration from Relational System to MongoDB System. In: Motahhir, S., Bossoufi, B. (eds) Digital Technologies and Applications. ICDTA 2022. Lecture Notes in Networks and Systems, vol 454. Springer, Cham. https://doi.org/10.1007/978-3-031-01942-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-01942-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-01941-8
Online ISBN: 978-3-031-01942-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)