Abstract
Hearing-impaired students often need complementary technologies to assist them in understanding college lectures. Several universities already use PC captioning. Captionists sometime input unfamiliar technical terms and proper nouns in a lecture inaccurately. We call these words “input-difficult words (IDWs).” In this research, we evaluate performance-detecting IDWs by using real lectures from our university. We propose a method to automatically extract IDWs from lecture materials. We conducted an experiment to measure performance-detecting IDWs from lectures by changing the interpolation weight of the language model. In this experiment, we used four real lectures. A high F-measure of 0.889 was achieved.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kato, N., Kawano, S., Miyoshi, S., Nishioka, T., Murakami, H., Minagawa, H., Wakatsuki, D., Shirasawa, M., Ishihara, Y., Naito, I.: Subjective evaluation of displaying keywords for speech to text service operators. Trans. Hum. Interface Soci. 9(2), 195–203 (2007). (in Japanese)
Akita, Y., Kuwahara, N., Kawahara, T.: Automatic classification of usability of ASR result for real-time captioning of lectures. In: 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), pp. 19–22 (2015)
Gaur, Y., Metze, F., Miao, Y., Bigham, J. P.: Using keyword spotting to help humans correct captioning faster. In: 16th Annual Conference of the International Speech Communication Association, pp. 2829–2833 (2015)
Ikeda, N., Takeuchi, Y., Matsumoto, T., Kudo, H., Ohnishi, N.: Support system for lecture captioning using keyword detection by automatic speech recognition. In: Miesenberger, K., Bühler, C., Penaz, P. (eds.) ICCHP 2016. LNCS, vol. 9759, pp. 377–383. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41267-2_53
Qin, L.: Learning out-of-vocabulary words in automatic speech recognition. Ph. D. thesis, CMU University (2013)
Illina, I., Fohr, D.: Out-of-vocabulary word probability estimation using RNN language model. In: Proceedings of the 8th Language & Technology Conference (2017)
Mirzaei, M.S., Meshgi, K., Kawahara, T.: Listening difficulty detection to foster second language listening with a partial and synchronized caption system. In: Borthwick, K., Bradley, L., Thouësny, S. (eds) CALL in a Climate of Change: Adapting to Turbulent Global Conditions Short Papers From EUROCALL 2017, pp. 211–216 (2017)
Munteanu, C., Penn, G., Beacker, R.: Web-based language modelling for automatic lecture transcription. In: Proceedings of the 8th Annual Conference of the International Speech Communication Association, No.ThD.P3a-2, pp. 2353–2356 (2007)
Kawahara, T., Nemoto, Y., Akita, Y.: Automatic lecture transcription by exploiting presentation slide information for language model adaptation, In: Proceedings of the ICASSP, pp. 4929–4932 (2008). (in Japanese)
Ito, A.: Palmkit (2009). http://palmkit.sourceforge.net/
Stolcke, A.: SRILM–an extensible language modeling toolkit. In: Proceedings of the ICSLP (2002)
Furui, S.: Recent advances in spontaneous speech recognition and understanding. In: Proceedings of the ISCA & IEEE Workshop on Spontaneous Speech Processing and Recognition, pp. 1–6 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Takeuchi, Y., Kojima, D., Sano, S., Kanamori, S. (2018). Detection of Input-Difficult Words by Automatic Speech Recognition for PC Captioning. In: Miesenberger, K., Kouroupetroglou, G. (eds) Computers Helping People with Special Needs. ICCHP 2018. Lecture Notes in Computer Science(), vol 10896. Springer, Cham. https://doi.org/10.1007/978-3-319-94277-3_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-94277-3_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94276-6
Online ISBN: 978-3-319-94277-3
eBook Packages: Computer ScienceComputer Science (R0)