Abstract
Sign language is the mode of communication used by hearing and speech impaired people to communicate with each other. The majority of the normal population does not have an understanding of sign language. To link this gap, the need for an automated sign language translator. Much research has been done for Indian Sign Language Recognition (SLR). Real-time sign language recognition is the need of an hour. We need to develop the SLR, which works efficiently in a real-time situation. Our study covers the development of the real-time Indian Sign Language Recognition System for one-handed ISL alphabets and numbers. We have used a hand landmark-based hand tracking module for detecting hands in real-time live video. The Euclidean Distance between hand landmarks is used as a feature to feed multilayer perceptron neural networks. We get 80% accuracy on training the model on the dataset. The prototype is tested by deploying the model in real-time SLR in live video, and it recognizing each one-handed gesture other than the gesture of alphabet āVā.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aloysius, N., Geetha, M.: Understanding vision-based continuous sign language recognition. Multimedia Tools Appl. 79, 22177ā22209 (2020)
Rathi, S., Gawande, U.: Development of full-duplex intelligent communication system for deaf and dumb people. In: 2017 7th International Conference on Cloud Computing, Data Science and Engineering-Confluence, pp. 733ā738, IEEE (2017)
Das, A., Gawde, S., Suratwala, K., Kalbande, D.: Sign language recognition using deep learning on custom processed static gesture images. In: 2018 International Conference on Smart City and Emerging Technology (ICSCET), pp. 1ā6, IEEE (2018)
Dutta, K.K., Bellary, S.A.S.: Machine learning techniques for Indian sign language recognition. In: 2017 International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC), pp. 333ā336, IEEE (2017)
Savant, R., Ajay, A.: Indian sign language recognition system for deaf and dumb using image processing and fingerspelling: a technical review. Nat. J. Syst. Inf. Technol. 11(1), 23 (2018)
Rao, G.A., Kishore, P.V.V.: Sign language recognition system simulated for video captured with smart phone front camera. Int. J. Electr. Comp. Eng. (2088ā8708) 6(5) (2016)
Chattoraj, S., Vishwakarma, K., Paul, T.: Assistive system for physically disabled people using gesture recognition. In: 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 60ā65, IEEE (2017)
Kishore, P.V.V., Kishore, S.R.C., Prasad, M.V.D.: Conglomeration of hand shapes and texture information for recognizing gestures of Indian sign language using feed-forward neural networks. Int. J. Eng. Technol. (IJET) 5(5), 3742ā3756 (2013)
Saqib, S., Kazmi, S.A.R.: Repository of static and dynamic signs. Int. J. Adv. Comput. Sci. Appl. 8, 101ā105 (2017)
Savant, R., Nasriwala, J.: Indian sign language recognition system: approaches and challenges. Adv. Innov. Res. 76 (2019)
Shaik, K.B., Ganesan, P., Kalist, V., Sathish, B.S., Jenitha, J.M.M.: Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Proc. Comp. Sci. 57, 41ā48 (2015)
Tripathi, K., Nandi, N.B.G.: Continuous Indian sign language gesture recognition and sentence formation. Proc. Comp. Sci. 54, 523ā531 (2015)
Reshna, S., Jayaraju, M.: Spotting and recognition of hand gesture for Indian sign language recognition system with skin segmentation and SVM. In: 2017 International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), pp. 386ā390, IEEE (2017)
Rautaray, S.S., Agrawal, A.: Vision based hand gesture recognition for human computer interaction: a survey. Artif. Intell. Rev. 43(1), 1ā54 (2015)
Sajanraj, T.D., Beena, M.V.: Indian sign language numeral recognition using region of interest convolutional neural network. In: 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), pp. 636ā640, IEEE (2018)
Kaur, G., Kaur, P.: Analysis of face recognition using YCbCr and CIElab skin color segmentation methods. Int. J. Adv. Res. Comp. Sci. 6(6) (2015)
Phung, S.L., Bouzerdoum, A., Chai, D.: A novel skin color model in YCbCr color space and its application to human face detection. In: Proceedings International Conference on Image Processing, vol. 1, pp. I-I, IEEE (2002)
Sikandar, T., Ghazali, K.H., Mohd, I.I., Rabbi, M.F.: Skin color pixel classification for face detection with hijab and niqab. In: Proceedings of the International Conference on Imaging, Signal Processing and Communication, pp. 1ā4 (2017)
Masood, S., Srivastava, A., Thuwal, H.C., Ahmad, M.: Real-time sign language gesture (word) recognition from video sequences using CNN and RNN. In: Intelligent Engineering Informatics, pp. 623ā632. Springer, Singapore (2018)
Ahire, P.G., Tilekar, K.B., Jawake, T.A., Warale, P.B.: Two way communicator between deaf and dumb people and normal people. In: 2015 International Conference on Computing Communication Control and Automation, pp. 641ā644, IEEE (2015)
Garcia, B., Viesca, S.A.: Real-time American sign language recognition with convolutional neural networks. Convolution. Neural Netw. Vis. Recogn. 2, 225ā232 (2016)
Hsieh, C.C., Liou, D.H.: Novel Haar features for real-time hand gesture recognition using SVM. J. Real-Time Image Proc. 10(2), 357ā370 (2015)
KƶpĆ¼klĆ¼, O., Gunduz, A., Kose, N., Rigoll, G.: Real-time hand gesture detection and classification using convolutional neural networks. In: 2019 14th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2019), pp. 1ā8, IEEE (2019)
Nagarajan, S., Subashini, T.S.: Static hand gesture recognition for sign language alphabets using edge oriented histogram and multi class SVM. Int. J. Comp. Appl. 82(4) (2013)
Zhang, F., Bazarevsky, V., Vakunov, A., Tkachenka, A., Sung, G., Chang, C.L., Grundmann, M.: Mediapipe hands: on-device real-time hand tracking. arXiv preprint arXiv:2006.10214 (2020)
āMediaPipe Hands.ā https://google.github.io/mediapipe/solutions/hands
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Savant, R.R., Nasriwala, J.V., Bhatt, P.P. (2022). Real-Time One-Hand Indian Sign Language Alphabets and Numbers Recognition in Live Video Using Fingertip Distance Feature. In: Das, B., Patgiri, R., Bandyopadhyay, S., Balas, V.E. (eds) Modeling, Simulation and Optimization. Smart Innovation, Systems and Technologies, vol 292. Springer, Singapore. https://doi.org/10.1007/978-981-19-0836-1_11
Download citation
DOI: https://doi.org/10.1007/978-981-19-0836-1_11
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-0835-4
Online ISBN: 978-981-19-0836-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)