Abstract
This paper describes a stemming technique that depends principally on a target language’s lexicon, organised as an automaton of word strings. The clear distinction between the lexicon and the procedure itself allows the stemmer to be customised for any language with little or even no changes to the program’s source code. An implementation of the stemmer, with a medium sized Portuguese lexicon is evaluated using Paice’s [16] evaluation method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adamson, G., Boreham, J.: The Use of an Association Measure Based on Character Structure to Identify Semantically Related Pairs of Words and Document Titles. Information Storage and Retrieval 10 (1974)
Aronoff, M., Anshen, F.: Morphology and the Lexicon: Lexicalization and Productivity. In: Spencer, A., Zwicky, A. (eds.) The Handbook of Morphology, Blackwell Publishers, Malden (1998)
Couto, M.: Representação de Léxicos através de Autômatos Finitos. MsC Dissertation. ICMC Universidade de São Paulo, São Carlos (1999)
Crystal, D.: An Encyclopedic Dictionary of Language and Languages. Penguin, London (1992)
Dawson, J.: Suffix Removal and Word Conflation. ALLC Bulletin, Michelmas (1974)
Frakes, W.B.: Stemming Algorithms. In: Frakes, W.B., Baeza-Yates, R. (eds.) Information Retrieval: Data Structures & Algorithms, Prentice-Hall, Englewood Cliffs (1992)
Fuller, M., Zobel, J.: Conflation-based Comparison of Stemming Algorithms. In: Proc. of the Third Australian Document Computing Symposium, Sydney, Australia (1998)
Hafer, M., Weiss, S.: Word Segmentation by Letter Succession Varieties. Information Storage and Retrieval 10 (1974)
Harman, D.: How Effective is Suffixing? Journal of the American Society for Information Science 42(1) (1991)
Hull, D.A.: Stemming Algorithms: A Case Study for Detailed Evaluation. Journal of the American Society for Information Science 47(1), 70–84 (1996)
Kowaltolwski, T., Lucchesi, C., Stolfi, J.: Finite Automata and Efficient Lexicon Implementation. Technical Report, Instituto de Computação, Universidade de Campinas (1998)
Krovetz, R.: Viewing morphology as an inference process. In: ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 191-202 (1993)
Lovins, J. B.: Development of a Stemming Algorithm. Mechanical Translation and Computational Linguistics 11(1&2) (1968)
Pacheco, H.: Uma Ferramenta de Auxílio à Redação. MsC Dissertation. Universidade Federal de Minas Gerais (1996)
Paice, C.: Another Stemmer. ACM Sigir Forum 24(3) (1990)
Paice, C.: An Evaluation Method for Stemming Algorithms. In: ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 42–50 (1994)
Porter, M.: An Algorithm for Suffix Stripping. Program 14(3) (1980)
Pequeno Vocabulário Ortográfico da Língua Portuguesa. Academia Brasileira de Letras, Rio de Janeiro (1999)
Revuz, D.: Minimisation of acyclic deterministic automata in linear time. Theoretical Computer Science 92 (1992)
Salton, G.: Automatic Information Organization and Retrieval. McGraw Hill, New York (1968)
Snowball, http://snowball.tartarus.org
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Silva, G., Oliveira, C. (2003). The Implementation and Evaluation of a Lexicon-Based Stemmer. In: Nascimento, M.A., de Moura, E.S., Oliveira, A.L. (eds) String Processing and Information Retrieval. SPIRE 2003. Lecture Notes in Computer Science, vol 2857. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39984-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-39984-1_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20177-9
Online ISBN: 978-3-540-39984-1
eBook Packages: Springer Book Archive