Abstract
The task of entity resolution is to find records that describe the same entity in the real world, so as to solve the problem of data duplication. This paper proposes an unsupervised entity resolution method based on machine learning. This method first uses LSTM to convert records into vectors with semantic information. Next, we use the improved random forest method to map the records into the n-dimensional space to realize the partition operation of the records, and consider that the records in the same partition point to the same entity. Finally, we use an improved Affinity Propagation Clustering (AP) to cluster the partitions to determine whether the records in different partitions point to the same entity. Through experiments on real data sets, the effectiveness of the algorithm for solving entity resolution tasks is proved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yu, L., Nie, T., Shen, D., Kou, Y.: An approach for progressive set similarity join with GPU accelerating. In: Wang, G., Lin, X., Hendler, J., Song, W., Xu, Z., Liu, G. (eds.) WISA 2020. LNCS, vol. 12432, pp. 155–167. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60029-7_14
Naumann, F., Herschel, M.: An introduction to duplicate detection. Synth. Lect. Data Manage. 2(1), 1–87 (2010)
Papadakis, G., Ioannou, E., Palpanas, T.: Entity resolution: past, present and yet-to-come. In: 23th International Conference on Extending Database Technology, pp. 647–650. Copenhagen, Denmark (2020)
Getoor, L., Machanavajjhala, A.: Entity resolution: theory, practice & open challenges. Proc. VLDB Endowm. 5(12), 2018–2019 (2012)
Elfeky, M., Verykios V., Elmagarmid, A.: Tailor: a record linkage toolbox. In: 18th International Conference on Data Engineering, pp. 17–28 (2002)
Tejada, S., Knoblock, C., Minton, S.: Learning object identification rules for information integration. Inf. Syst. 26(8), 607–633 (2001)
Bilenko, M., Mooney, R.: Adaptive duplicate detection using learnable string similarity measures. In: 9th International Conference on Knowledge Discovery and Data Mining, pp. 39–48 (2003)
Christen, P.: Automatic record linkage using seeded nearest neighbour and support vector machine classification. In: 14th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 151–159 (2008)
Newcombe, H.B., Kennedy, J.M., Axford, S.J., James, A.P.: Automatic linkage of vital records. Science 3381(130), 954–959 (1959)
Lu, C., Huang, G.Y.: Entity resolution in sparse encounter network using Markov logic network. IEEE Access 9, 83055–83066 (2021)
Fisher, J., Christen, P., Wang, Q.: Active learning based entity resolution using markov logic. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J.Z., Wang, R. (eds.) PAKDD 2016. LNCS (LNAI), vol. 9652, pp. 338–349. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31750-2_27
Culotta, A., McCallum, A.: Joint deduplication of multiple record types in relational data. In: 14th International Conference on Information and Knowledge Management, pp. 257–258 (2005)
Verykios, V., Elmagarmid, A., Houstis, E.: Automating the approximate record-matching process. Inf. Sci. 126(1–4), 83–98 (2000)
Arasu, A., Götz, M., Kaushik, R.: On active learning of record matching packages. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 783–794 (2010)
Fu, Y.F., Zhu, X.Q., Li, B.: A survey on instance selection for active learning. Knowl. Inf. Syst. 35(2), 249–283 (2013)
de Carvalho, M.G., Laender, A.H.F., Gonçalves, M.A., da Silva, A.S.: A genetic programming approach to record deduplication. IEEE Trans. Knowl. Data Eng. 24(3), 399–412 (2012)
Isele, R., Bizer, C.: Active learning of expressive linkage rules using genetic programming. J. Web Semant. 23, 2–15 (2013)
Sharma, A.K., Chaurasia, S., Srivastava, D.K.: Sentimental short sentences classification by using CNN deep learning model with fine tuned Word2Vec. Procedia Comput. Sci. 167, 1139–1147 (2020)
Liu, G., Guo, J.B.: Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337, 325–338 (2019)
Flake, G.W., Tarjan, R.E., Tsioutsiouliklis, K.: Graph clustering and minimum cut trees. Int. Math. 1(4), 385–408 (2003)
Perbet, F., Stenger, B., Maki, A.: Random forest clustering and application to video segmentation. In: Proceedings of the 2009 British Machine Vision Conference, pp. 1–10 (2009)
Bodenhofer, U., Kothmeier, A., Hochreiter, S.: APCluster: an R package for affinity propagation clustering. Bioinformatics 27(17), 2463–2464 (2011)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Grant Nos. 62002262, 71804123), and the TJUT College Students’ Innovative Entrepreneurial Training Plan Program (Grant No. 202010060040).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, W., Sun, C., Xu, L., Chen, W., Hou, Z. (2021). Unsupervised Entity Resolution Method Based on Random Forest. In: Xing, C., Fu, X., Zhang, Y., Zhang, G., Borjigin, C. (eds) Web Information Systems and Applications. WISA 2021. Lecture Notes in Computer Science(), vol 12999. Springer, Cham. https://doi.org/10.1007/978-3-030-87571-8_32
Download citation
DOI: https://doi.org/10.1007/978-3-030-87571-8_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87570-1
Online ISBN: 978-3-030-87571-8
eBook Packages: Computer ScienceComputer Science (R0)
