Abstract
We present a method to automatically detect sentenceboundaries(SBs) in Japanese speech transcriptions. Our method uses a Japanese morphological analyzer that is based on a cost calculation and selects as the best result the one with the minimum cost. The idea behind using a morphological analyzer to identify candidates for SBs is that the analyzer outputs lower costs for better sequences of morphemes. After the candidate SBs have been identified, the unsuitable candidates are deleted by using lexical information acquired from the training corpus. Our method had a 77.24% precision, 88.00% recall, and 0.8277 F-Measure, for a corpus consisting of lecture speech transcriptions in which the SBs are not given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Matsumoto, Y., Kitauchi, A., Tatsuo Yamashita, Y.H., Matsuda, H., Takaoka, K., Asahara, M.: Morphological Analysis System ChaSen version 2.2.9 Manual (2002)
The National Institute for Japanese Language: The Corpus of Spontaneous Japanese (The monitor version 2001) Guidance of monitor public presentation (2001), http://www.kokken.go.jp/public/monitor_kokai001.html
The National Institute for Japanese Language: The Corpus of Spontaneous Japanese (The monitor version 2002) Guidance of monitor public presentation (2002), http://www.kokken.go.jp/public/monitor_kokai002.html
Stevenson, M., Gaizauskas, R.: Experiments on Sentence Boundary Detection. In: Proc. of ANLP-NAACL 2000, pp. 84–89 (2000)
Shitaoka, K., Kawahara, T., Okuno, H.G.: Automatic Transformation of Lecture Transcription into Document Style using Statistical Framework. In: IPSJ SIG Notes on Spoken Language Processing, vol. 41-3 (2002) (in Japanese)
Nobata, C., Sekine, S., Uchimoto, K., Isahara, H.: Sentence Segmentation and Sentence Extraction. In: Proc. of the Second Spontaneous Speech Science and Technology Workshop, pp. 527–534 (2002) (in Japanese)
Seligman, M., Hosaka, J., Singer, H.: “Pause Units” and Analysis of Spontaneous Japanese Dialogues: Preliminary Studies. In: ECAI 1996 workshop on “Dialogue Processing in Spoken Language Systems”, pp. 100–112 (1996)
Palmer, D.D., Hearst, M.A.: Adaptive sentence boundary disambiguation. In: Proc. of the fourth Conference on Applied Natural Language Processing, pp. 78–83 (1994)
Reynar, J.C., Ratnaparkhi, A.: A maximum entropy approach to identifying sentence boundaries. In: Proc. of the fifth Conference on Applied Natural Language Processing, pp. 16–19 (1997)
Beeferman, D., Berger, A., Lafferty, J.: CYBERPUNC: A lightweight punctu IEEE International Conference on Acoustics, Speech and Signaation annotation system for speech. In: Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 689–692 (1998)
Shieber, S.M., Tao, X.: Comma restoration using constituency information. In: Proc. of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp. 221–227 (2003)
Tooyama, Y., Nagata, M.: Insertion methods of punctuation marks for speech recognition systems. In Technical Report of IEICE, NLC2000-5 (2000) (in Japanese)
Asahara, M., Matsumoto, Y.: Extended Hidden Markov Model for Japanese Morphological Analyzer. In: IPSJ SIG Notes on Spoken Language Processing, vol. 031 (2000) (in Japanese)
Asahara, M., Matsumoto, Y.: IPADIC user’s manual version 2.5 (2002)
Liu, D., Zong, C.: Utterance Segmentation Using Combined Approach Based on Bi-directional N-gram and Maximum Entropy. In: Proc. of ACL 2003 Workshop: The Second SIGHAN Workshop on ChineeseLanguage Processing, pp. 16–23 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tajima, S., Nanba, H., Okumura, M. (2005). Detecting Sentence Boundaries in Japanese Speech Transcriptions Using a Morphological Analyzer. In: Su, KY., Tsujii, J., Lee, JH., Kwong, O.Y. (eds) Natural Language Processing – IJCNLP 2004. IJCNLP 2004. Lecture Notes in Computer Science(), vol 3248. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30211-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-540-30211-7_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24475-2
Online ISBN: 978-3-540-30211-7
eBook Packages: Computer ScienceComputer Science (R0)