Abstract
Language learning has thus far not been a hot application for machine-learning (ML) research. This limited attention for work on empirical learning of language knowledge and behaviour from text and speech data seems unjustified. After all, it is becoming apparent that empirical learning of Natural Language Processing (NLP) can alleviate NLP's all-time main problem, viz. the knowledge acquisition bottleneck: empirical ML methods such as rule induction, top down induction of decision trees, lazy learning, inductive logic programming, and some types of neural network learning, seem to be excellently suited to automatically induce exactly that knowledge that is hard to gather by hand. In this paper we address the question why NLP is an interesting application for empirical ML, and provide a brief overview of current work in this area.
Chapter PDF
Similar content being viewed by others
Keywords
- Natural Language Processing
- Machine Translation
- Inductive Logic Programming
- Rule Induction
- Lazy Learning
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Aha, D., Kibler, D., and Albert, M. (1991). Instance-based learning algorithms. Machine Learning, 7:37–66.
Andernach, T. (1996) A machine learning approach to the classification of dialogue utterances. In K. Oflazer and H. Somers (Eds.), Proceedings of the Second International Conference on New Methods in Language Processing, NeMLaP, pp. 98–109.
Black, E., Jelinek, F., Lafferty, J, Mercer, R., and Roukos, S. (1992). Decision tree models applied to the labeling of text with parts-of-speech Darpa Workshop on Speech and Natural Language.
Cardie, C. (1994). Domain-Specific Knowledge Acquisition for Conceptual Sentence Analysis. Ph.D. Thesis, University of Massachusetts, Amherst, MA.
Cardie, C. (1996). Embedded Machine Learning Systems for Natural Language Processing: A General Framework. In S. Wermter, E. Riloff, and G. Scheler, (Eds.), Connectionist, Statistical and Symbolic Approaches to Learning for Natural Language Processing, pp. 315–328. Berlin: Springer-Verlag.
Clark, P., and Boswell, R. (1991). Rule induction with cN2: Some recent improvements. In Machine Learning: Proceedings of the Fifth European Conference, pp. 151–163.
Cussens, J. (1996). Part-of-speech disambiguation using ILP. Technical report PRG-TR-25-96, Oxford University Computing Laboratory.
Daelemans, W. (1995). Memory-based lexical acquisition and processing. In P. Steffens (Ed.), Machine Translation and the Lexicon, pp. 85–98. Berlin: Springer-Verlag.
Daelemans, W. (1996). Abstraction considered harmful: Lazy learning of language processing. In H. J. van den Herik and A. Weijters (Eds.), Proceedings of the 6th Belgian-Dutch Conference on Machine Learning, Maastricht, The Netherlands, pp. 3–12.
Daelemans, W., Berck, P, and Gillis, S. (1996). Unsupervised discovery of phonological categories through supervised learning of morphological rules. In Proceedings of the 16th International Conference on Computational Linguistics, Copenhagen, Denmark, pp. 95–100.
Daelemans, W., Van den Bosch, A., and Weijters, A. (to appear). igtree: using trees for classification in lazy learning algorithms. Artificial Intelligence Review, special issue on Lazy Learning. To appear.
Dehaspe, L., Blockeel, H., and De Raedt, L. (1996). Induction, logic and natural language processing. In Proceedings of the Joint elsnet/compulog-net/eagles Workshop on Computational Logic for Natural Language Processing, South Queensferry, Scotland.
Dietterich, T. G., Hild, H., and Bakiri, G. (1990). A comparison of id3 and Backpropagation for English text-to-speech mapping. Technical Report 90-20-4, Oregon State University.
Jones, D. Analogical Natural Language Processing. London: UCL Press, 1996.
Kolodner, J. (1992). Case-Based Reasoning. San Mateo, CA: Morgan Kaufmann.
Lavrac, N. and Dzeroski, S. (1994). Inductive Logic Programming. Chichester, UK: Ellis Horwood.
Lawrence, S., Fong, S., and Giles, C. Lee (1991) Natural language grammatical inference: A comparison of recurrent neural networks and machine learning methods. In S. Wermter, E. Riloff, and G. Scheler (Eds.), Connectionist, Statistical, and Symbolic Approaches to Learning for Natural Language Processing, pp. 33–47. Berlin: Springer-Verlag.
Litman, D. J. (1996). Cue phrase classification using machine learning. Journal of Artificial Intelligence Research, 5:53–94, 1996.
Magerman, D. (1995). Statistical decision tree models for parsing. In Proceedings of the Association for Computational Linguistics., 1995.
Muggleton, S., and De Raedt, L. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19, 20:629–679.
Muggleton, S., Page, D., and Srinivasan, A. (1996). Learning to read by theory revision. Technical Report PRG-TR-26-96, Oxford University Computing Laboratory.
Ng, H. T. and H. B. Lee (1996). Integrating multiple knowledge sources to disambiguate word sense: an exemplar-based approach. In Proceedings of the annual meeting of the ACL, ACL-96.
Quinlan, J. R. (1993). c4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann.
Reilley, R. G. and Sharkey, N. E., Eds. (1992). Connectionist Approaches to Natural Language Processing. Hillsdale, NJ: Lawrence Erlbaum Associates.
Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1, pp. 318–362. Cambridge, MA: The MIT Press.
Schmid, H. (1994). Probabilistic Part-of-Speech Tagging Using Decision Trees. Proceedings of the International Conference on New Methods in Language Processing, NeMLaP, Manchester, 44–49.
Van den Bosch, A., and Daelemans, W. (1993). Data-oriented methods for graphemeto-phoneme conversion. In Proceedings of the 6th Conference of the EACL, pp. 45–53.
Van den Bosch, A., Daelemans, W., and Weijters, A. (1996). Morphological analysis as classification: an inductive-learning approach In K. Oflazer and H. Somers (Eds.), Proceedings of the Second International Conference on New Methods in Language Processing, NeMLaP, Ankara, Turkey, pp. 79–89.
Van den Bosch, A. (forthcoming). Machines Learning to Pronounce Words: Empirical Learning and Modularisation of Morpho-Phonology. PhD-Thesis, Universiteit Maastricht.
Weijters, A. (1991). A simple look-up procedure superior to NETtalk? In Proceedings of the International Conference on Artificial Neural Networks, Espoo, Finland.
Wermter, S., Riloff, E., and Scheler, G. (1996). Connectionist, Statistical, and Symbolic Approaches to Learning for Natural Language Processing. Berlin: Springer-Verlag.
Wettschereck, D., Aha, D. W. & Mohri, T. (1996). A review and comparative valuation of feature weighting methods for lazy learning algorithms. Technical Report AIC-95-012. Washington, DC: NRL Navy Center for Applied Research in AI.
Wolters, M. (1995). A dual-route approach to grapheme-to-phoneme conversion. In Proceedings of the International Conference on Artificial Neural Networks.
Zelle, J. M., and Mooney, R. J. (1994). Inducing deterministic Prolog parsers from treebanks: A machine learning approach. In Proceedings of the Twelfth National Conference on Artificial Intelligence, Seattle, WA, pp. 748–753.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Daelemans, W., van den Bosch, A., Weijters, T. (1997). Empirical learning of Natural Language Processing tasks. In: van Someren, M., Widmer, G. (eds) Machine Learning: ECML-97. ECML 1997. Lecture Notes in Computer Science, vol 1224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-62858-4_97
Download citation
DOI: https://doi.org/10.1007/3-540-62858-4_97
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-62858-3
Online ISBN: 978-3-540-68708-5
eBook Packages: Springer Book Archive