Abstract
Phrase-based models and class-based models are both variants of classical n-gram models. In this paper, we propose an approach by merging phrase-based models and class-based models together. In the phrase-based part, we use bilingual parallel corpus to extract phrases with a method deriving from phrase-based translation models. Then we partition these phrases into phrase classes by minimizing the loss of the average mutual information with the aid of a count matrix. Our experimental results suggest that phrase-based models can capture more key information than word-based models and class-based models can capture the relationship among similar words or phrases and thus solve the problem of data sparseness in some sense.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jurafsky, D., Martin, J.H.: Speech and Language Processing: An introduction to speech recognition, computational linguistics and natural language processing. Prentice-Hall, Englewood Cliffs (2006)
Brown, P.F., DellaPietra, V.J., deSouza, P.V., Lai, J.C., Mercer, R.L.: Class-based n-gram models of natural language. Computational Linguistics 18(4), 467–479 (1992)
Gao, J., Goodman, J.T., Miao, J.: The Use of Clustering Techniques for Language Modeling - Application to Asian Language. Computational Linguistics and Chinese Language Processing 6(1) (2001)
Zitouni, I., Smaili, K., Haton, J.-P.: Statistical Language Modeling Based on Variable-Length Sequences. Computer Speech and Language 17, 27–41 (2003)
Koehn, P., Och, F., Marcu, D.: Statistical Phrase-Based Translation. In: Proceedings of. the Conference on Human Language Technology, pp. 127–133 (2003)
Och, F.J., Ney, H.: A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics 29(1), 19–51 (2003)
Katz, S.M.: Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer. IEEE Transactions on Acoustics, Speech and Signal Processing ASSP-35, 400–401 (1987)
Clakson,P., Rosenfeld, R.: Statistical Language Modeling using the CMU-Cambridge Toolkit. In: Proceedings of Euro-speech, vol 5 (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mao, J., Cheng, G., He, Y. (2007). Phrase-Based Statistical Language Modeling from Bilingual Parallel Corpus. In: Chen, B., Paterson, M., Zhang, G. (eds) Combinatorics, Algorithms, Probabilistic and Experimental Methodologies. ESCAPE 2007. Lecture Notes in Computer Science, vol 4614. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74450-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-74450-4_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74449-8
Online ISBN: 978-3-540-74450-4
eBook Packages: Computer ScienceComputer Science (R0)