Abstract
Fingerspelling is used in sign language to spell out names of people and places for which there is no sign or for which the sign is not known. In this work we describe a Turkish fingerspelling recognition system that recognizes all 29 letters of the Turkish alphabet. A single representative frame is extracted from the sign video, since that frame is enough for recognition purposes of the letters mentioned. Processing a single frame, instead of the whole video, increases speed considerably. The skin regions in the representative frame are extracted by color segmentation in YCrCb space before clearing noise regions by morphological opening. A novel fast alignment method that uses the angle of orientation between the axis of least inertia and y axis is applied to hand regions. This method compensates small orientation differences but increases big ones. This is desirable when differentiating the fingerspelling signs, some of which are close in shape but different in orientation. Also the use of minimum bounding square is advised, which helps in resizing without breaking the alignment. Binary values of this minimum bounding square are directly used as feature values, and that allowed experimenting with different classification schemes. Features like mean radial distance and circularity are also used for increasing success rate. Classifiers like kNN, SVM, Naïve Bayes, and RBF Network are experimented with, and 1NN and SVM are found to be the best two of them. The video database was created by 3 different signers, a set of 290 training videos, and a separate set of 174 testing videos are used in experiments. The best classifiers 1NN and SVM achieved a success rate of 99.43% and 98.83% respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
http://www.british-sign.co.uk/learnbslsignlanguage/whatisfingerspelling.htm
Starner, T., Weaver, J., Pentland, A.: Real-time American sign language recognition using desk and wearable computer based video. Ieee Transactions on Pattern Analysis and Machine Intelligence 20, 1371–1375 (1998)
Holden, E.J., Lee, G., Owens, R.: Australian sign language recognition. Machine Vision and Applications 16, 312–320 (2005)
Gao, W., Fang, G.L., Zhao, D.B., Chen, Y.Q.: A Chinese sign language recognition system based on SOFM/SRN/HMM. Pattern Recognition 37, 2389–2402 (2004)
Haberdar, H., Albayrak, S.: Real Time Isolated Turkish Sign Language Recognition From Video Using Hidden Markov Models With Global Features. In: Yolum, p., Güngör, T., Gürgen, F., Özturan, C. (eds.) ISCIS 2005. LNCS, vol. 3733, p. 677. Springer, Heidelberg (2005)
Haberdar, H., Albayrak, S.: Vision Based Real Time Isolated Turkish Sign Language Recognition. In: International Symposium on Methodologies for Intelligent Systems, Bari, Italy (2006)
Lamar, M., Bhuiyant, M.: Hand Alphabet Recognition Using Morphological PCA and Neural Networks. In: International Joint Conference on Neural Networks, Washington, USA, pp. 2839–2844 (1999)
Rebollar, J., Lindeman, R., Kyriakopoulos, N.: A Multi-Class Pattern Recognition System for Practical Fingerspelling Translation. In: International Conference on Multimodel Interfaces, Pittsburgh, USA (2000)
Isaacs, J., Foo, S.: Hand Pose Estimation for American Sign Language Recognition. In: IEEE System Theory on Thirty-Sixth Southeastern Symposium, pp. 132–136 (2004)
Feris, R., Turk, M., Raskar, R., Tan, K.: Exploiting Depth Discontinuities for Vision-Based Fingerspelling Recognition. In: 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2004) (2004)
Altun, O., Albayrak, S., Ekinci, A., Bükün, B.: Increasing the Effect of Fingers in Fingerspelling Hand Shapes by Thick Edge Detection and Correlation with Penalization. In: Chang, L.-W., Lie, W.-N. (eds.) PSIVT 2006. LNCS, vol. 4319, Springer, Heidelberg (2006)
Sazonov, V., Vezhnevetsi, V., Andreeva, A.: A survey on pixel vased skin color detection techniques. In: Graphicon 2003, pp. 85–92 (2003)
Chai, D., Bouzerdom, A.: A Bayesian Approach To Skin Colour Classification. In: TENCON 2000 (2000)
Umbaugh, S.E.: Computer Vision and Image Processing: A Practical Approach Using CVIP tools. Prentice-Hall, Englewood Cliffs (1998)
Aha, D.W., Kibler, D., Albert, M.K.: Instance-Based Learning Algorithms. Machine Learning 6, 37–66 (1991)
Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Computation 13, 637–649 (2001)
Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)
Fritzke, B.: Fast Learning with Incremental Rbf Networks. Neural Processing Letters 1, 2–5 (1994)
McCallum, A., Nigam, K.: A Comparison of Event Models for Naive Bayes Text Classification. In: AAAI 1998, Workshop on Learning for Text Categorization (1998)
John, G.H., Langley, P.: Estimating Continuous Distributions in Bayesian Classifiers. In: Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345. Morgan Kaufmann, San Mateo (1995)
Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Altun, O., Albayrak, S., Ekinci, A., Bükün, B. (2006). Turkish Fingerspelling Recognition System Using Axis of Least Inertia Based Fast Alignment. In: Sattar, A., Kang, Bh. (eds) AI 2006: Advances in Artificial Intelligence. AI 2006. Lecture Notes in Computer Science(), vol 4304. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11941439_51
Download citation
DOI: https://doi.org/10.1007/11941439_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-49787-5
Online ISBN: 978-3-540-49788-2
eBook Packages: Computer ScienceComputer Science (R0)