Abstract
Clustering of Arabic documents is considered as a vital aspect of obtaining optimal results from unsupervised learning. Its aim is to automatically group similar documents into a single cluster using different similarities or distance measures. However, diverse similarities and distance measures are available and their effectiveness in document clustering with a syntactic structure of the stemming is still not obvious. Therefore, this study aims to evaluate the impact of five similarity/distance measures (i.e., cosine similarity, the Jaccard coefficient, Pearson’s correlation coefficient, Euclidean distance, and averaged Kullback-Leibler divergence) with two stemming algorithms (i.e., morphology- and syntax-based lemmatization; and morphology-based Information Science Research Institute (ISRI) stemming on clustering Arabic text dataset. We aim to identify the best performing similarity and distance measures and determine which measure is most suitable for Arabic document clustering. Our experimental method, which is based on syntactic structure and morphology, outperformed other stemming methods that use any of the five similarity/distance measures for Arabic document clustering. The best performing similarity/distance measures are cosine similarity and Euclidean distance, respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Khoja, S.: APT: Arabic Part-Of-Speech Tagger. In: Proceedings of the Student Workshop at NAACL, pp. 20–25 (2001)
Hani, A., Mahmoud, A., Martha, W.: Stemming Methodologies Over Individual Query Words for an Arabic Information Retrieval System. J. Am. Soc. Inf. Sci. 50(6), 524–529 (1999)
Eiman A.: Patent Application Publication of Lemmatizing, Stemming, and Query Expansion Method and System, Pub.No.:US 2010/0082333 A1, Pub.Date: Apr.1 (2010), http://Www.Google.Com/Patents/US20100082333?Printsec=Abstract&Source=Gbs_Overview_R&Cad=0#V=Onepage&Q&F=False
Ozgür, A.: A Supervised and Unsupervised Machine Learning Techniques for Text Document Categorization, Master. Thesis. Bogazici University, Turkey (2004)
Khoja, S., Garside, R.: Stemming Arabic Text. In: Computing Department. Lancaster University, UK (1999), http://Www.Comp.Lancs.Ac.Uk/Computing/Users/Khoja/Stemmer.Ps
Larkey, S., Ballesteros, L., Connell, E.: Improving Stemming for Arabic Information Retrieval: Light Stemming and Co-Occurrence Analysis. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, Finland (2002)
Hull, D.: Stemming Algorithms: A Case Study for Detailed Evaluation. Journal of the American Society for Information Science 47(1), 70–84 (1996)
Korenius, T., Laurikkala, J., Järvelin, K., Juhola, M.: Stemming and Lemmatization in the Clustering of Finnish Text Documents. In: Proceedings of the Thirteenth ACM International Conference on Information and Knowledge Management CIKM, pp. 625–633 (2004)
Vinsensius, V., Stéphane, B.: Indexing The Indonesian Web: Language Identification and Miscellaneous Issues. In: Presented at Tenth International World Wide Web Conference, Hong Kong (2001)
Tai, Y., Ong, S., Abullah, A.: On Designing an Automated Malaysian Stemmer for the Malay Language. In: Proceedings of the Fifth International Workshop on Information Retrieval With Asian Languages, Hong Kong, pp. 207–208 (2000)
Popovi, M., Willett, P.: The Effectiveness of Stemming for Natural Language Access to Slovene Textual Data. Journal of the American Society for Information Science 43(5), 384–390 (1992)
Moulinier, I., Mcculloh, J., Lund, E.: West Group at CLEF: Non-English Monolingual Retrieval, Cross-Language Information Retrieval and Evaluation. In: Peters, C. (ed.) Proceedings of the CLEF 2000 Workshop, pp. 176–187. Springer (2001)
Kraaij, W., Pohlmann, R.: Viewing Stemming as Recall Enhancement. In: Proceedings of SIGIR, pp. 40–48. ACM, USA (1996)
Greengrass, M., Robertson, M., Schinke, R., Willett, P.: Processing Morphological Variants in Searches of Latin Text. Information Research News 6(4), 2–5 (1996)
Ekmekcioglu, C., Lynch, F., Willett, P.: Stemming and N-Gram Matching for Term Conflation in Turkish Texts. Information Research 2(2) (1996)
Carlberger, J., Dalianis, H., Hassel, M., Knutsson, O.: Improving Precision in Information Retrieval for Swedish Using Stemming. In: Proceedings of NODALIDA, pp. 21–22 (2001)
Monz, C., de Rijke, M.: Shallow Morphological Analysis in Monolingual Information Retrieval for Dutch, German, and Italian. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.) CLEF 2001. LNCS, vol. 2406, pp. 262–277. Springer, Heidelberg (2002)
Anil, P., Jitendra, A., Nishchol, M.: Analysis of Different Similarity Measure Functions and their Impacts on Shared Nearest Neighbor Clustering Approach. International Journal of Computer Applications 40(16), 1–5 (2012)
Qusay, B., Masnizah, M.: Effect of IRSI Stemming on Similarity Measure for Arabic Document Clustering. In: Proceedings of the Seventh Asia Information Retrieval Societies Conference, pp. 584–593. Springer, Dubai (2011)
Baeza-Yates, R., And Ribeiro-Neto, B.: Modern Information Retrieval, vol. 463. ACM, New York (1999)
Larsen, B., Aone, C.: Fast and Effective Text Mining Using Linear-Time Document Clustering. In: Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD, pp. 16–22 (1999)
Froud, H., Benslimane, R., Lachkar, A., Ouatik, S.A.: Stemming And Similarity Measures For Arabic Documents Clustering. In: 5th International Symposium on I/V Communications and Mobile Network (ISVC), pp. 1–4 (2010)
Khreisat, L.: Arabic Text Classification Using N-Gram Frequency Statistics A Comparative Study. In: Proceedings of Data Mining DMIN, pp. 78–82 (2006)
Taghva, K., Elkhoury, R., Coombs, J.: Arabic Stemming Without A Root Dictionary. In: ITCC International Conference on Information Technology: Coding and Computing, vol. 1, pp. 152–157 (2005)
Jain, K., Dubes, C.: Algorithms for Clustering Data. Michigan State University (1988)
Steinbach, M., Karypis, G., Kumar, V.: A Comparison of Document Clustering Techniques. In: KDD Workshop on Text Mining, pp. 525–526 (2000)
Al-Salemi, B., Mohammad, A.: Statistical Bayesian Learning for Automatic Arabic Text Categorization. J. Comput. Sci. 7(1), 39–45 (2011)
Berry, W., Browne, M.: Understanding Search Engines: Mathematical Modeling and Text Retrieva. SIAM (1999) ISBN: 0898714370
Manning, D., Raghavan, P., Schütze, H.: Corporation.:Introduction to Information Retrieval, 1st edn. Cambridge University Press, Cambridge (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Bsoul, Q., Al-Shamari, E., Mohd, M., Atwan, J. (2014). Distance Measures and Stemming Impact on Arabic Document Clustering. In: Jaafar, A., et al. Information Retrieval Technology. AIRS 2014. Lecture Notes in Computer Science, vol 8870. Springer, Cham. https://doi.org/10.1007/978-3-319-12844-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-12844-3_28
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12843-6
Online ISBN: 978-3-319-12844-3
eBook Packages: Computer ScienceComputer Science (R0)