Abstract
In this paper, we suggest an improved weight decision rule depending on SNNR (Signal Plus Noise to Noise Ratio) and fuzzy value for simultaneous multi-modality including a synchronization between audio-gesture modalities. In order to insure the validity of the suggested weight decision rule, we implement a wireless PDA-based Multi-Modal Fusion Architecture (hereinafter, MMFA) by coupling embedded speech and KSSL recognizer, which fuses and recognizes 130 word-based instruction models that are represented by speech and KSSL (Korean Standard Sign Language), and then translates recognition result into synthetic speech (TTS) and visual illustration in real-time. In the experimental results, the average recognition rate of the MMFA fusing 2 sensory channels based on wireless PDA was 96.54% in clean environments (e.g. office space), and 93.21% in noisy environments, with the 130 word-based instruction models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jacko, J.A., et al.: Handbook for Human Computer Interaction. Lawrence Erlbaum & Associates, Mahwah (2003)
Wooldridge, M.J., Jennings, N.R.: Intelligent agents: Theory and practice. Know. Eng. Review 10(2), 115–152 (1995)
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. In: Proceedings of the IEEE, February 1989, vol. 77(2), pp. 257–286 (1989)
Kim, S.-G.: Korean Standard Sign Language Tutor, 1st edn. Osung Publishing Company, Seoul (2000)
Kim, J.-H., et al.: An Implementation of KSSL Recognizer for HCI Based on Post Wearable PC and Wireless Networks KES 2006. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006. LNCS (LNAI), vol. 4251, pp. 788–797. Springer, Heidelberg (2006)
Kim, J.-H., et al.: Hand Gesture Recognition System using Fuzzy Algorithm and RDBMS for Post PC. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, pp. 170–175. Springer, Heidelberg (2005)
Vasantha kandasamy, W.B.: Smaranda Fuzzy Algebra. American Research Press, Seattle (2003)
Chen, C.H.: Fuzzy Logic and Neural Network Handbook, 1st edn. McGraw-Hill, New York (1992)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, JH., Hong, KS. (2007). An Improved Weight Decision Rule Using SNNR and Fuzzy Value for Multi-modal HCI. In: Masulli, F., Mitra, S., Pasi, G. (eds) Applications of Fuzzy Sets Theory. WILF 2007. Lecture Notes in Computer Science(), vol 4578. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73400-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-73400-0_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73399-7
Online ISBN: 978-3-540-73400-0
eBook Packages: Computer ScienceComputer Science (R0)