Abstract
Fingerspelling is a process of communicating letters of a spoken language alphabet using a person’s hand or hands. Portraying animations of fingerspelling has proved surprisingly resistant to automation because of the collisions that arise from conventional interpolation of keyframes of individual manual letters. Previous methods have not been able to provide convincingly realistic fingerspelling due to the absence of effective collision avoidance in the underlying animation algorithms. This paper reports on the development and evaluation of a new collision avoidance algorithm that aids fingerspelling. Instead of analyzing letter transitions, the algorithm capitalizes on the transitions of individual fingers. The new strategy is efficient enough to support real-time fingerspelling while still maintaining a high level of predictive accuracy. Utilizing this strategy in signing avatars is expected to improve the current resources for deaf children, hearing teachers, hearing parents, and interpreting students who want to improve their fingerspelling comprehension. Future work will include testing the strategy’s generality when applying it to other one-handed manual alphabets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akamatsu CT (1982) The acquisition of fingerspelling in pre-school children. University Microfilms, Ann Arbor
Antona M, Stephanidis C (2015) Universal access in human-computer in- teraction. In: Access to learning, health and well-being: 9th International Conference, UAHCI 2015, Held as Part of HCI International
Baowidan S, Guo N, Johnson S, Moncrief R, Berke L (2017) A New N- gram analytics tool in ELAN and its application to improve automatic fingerspelling generation. DePaul Universityss, Chicago
Calderon R (2000) Parental involvement in deaf children’s education programs as a predictor of child’s language, early reading, and social-emotional development. J Deaf Stud Deaf Educ 5(2):140–155. https://doi.org/10.1093/deafed/5.2.140
Davidson MJ (2000) Usability testing of computer animation of finger- spelling for American sign language. In: and others (ed) CTI Research Conference
Ebling S, Wolfe R, et al. (2015) Synthesizing the finger alphabet of Swiss German Sign Language and evaluating the comprehensibility of the resulting animations. Proceedings of SLPAT 2015: 6th Workshop on Speech and Language Processing for Assistive Technologies pp 10–16
Foley JD, van Dam A (1982) Fundamentals of interactive computer graphics, vol 2. Addison-Wesley Reading, MA
Guillory LM (1966) Claitors publishing division
Jaklic A, Vodopivec D, Komac V (1995) Learning sign language through multimedia. In: International Conference on Multimedia Computing and Systems, IEEE, pp 282–285
Mcdonald J, Alkoby K, Carter R, Christopher J, Davidson M, Ethridge D, Wolfe R (2001) An improved articulated model of the human hand. Vis Comput 17(3):158–166
Mckee RL (1992) What’s so hard about learning ASL? Students’and teachers’ perceptions. Sign Language Studies 75(1):129–157
Murdock K (2011) 3Ds Max 2012 bible, vol 783. John Wiley & Sons, New york
Padden C, Gunsauls DC (2003) How the alphabet came to be used in a sign language. Sign Language Studies 4(1):10–33. https://doi.org/10.1353/sls.2003.0026
Padden C, Ramsey C (1998) Reading ability in signing deaf children. Top Lang Disord 18:30–46
Patrie C, Johnson R (2011) Fingerspelled word recognition through rapid serial visual presentation. DawnSignPress, San Diego
Schick B (2005) Look who’s being left behind: educational interpreters and access to education for deaf and hard-of-hearing students. J Deaf Stud Deaf Educ 11(1):3–20. https://doi.org/10.1093/deafed/enj007
Shaffer L, Watson W (2004) Peer mentoring: what is that. Proceedings of the 15th National Convention Conference of Interpreter Trainers (CIT), Still shining after 25:77–92
Shipgood LE, Pring TR (1995) The difficulties of learning fingerspelling: an experimental investigation with hearing adult learners. Inter J Nal Lang Commun Disorders 30(4):401–416. https://doi.org/10.3109/13682829509087241
Stoll S, Camgoz NC, Hadfield S, Bowden R (2018) Sign language production using neural machine translation and generative adversarial networks. Proceedings of the 29th British Machine Vision Conference
Toro JA, Mcdonald JC, Wolfe R (2014) Fostering better deaf/hearing com- munication through a novel mobile app for fingerspelling . In: and others (ed) Computers helping people with special needs, Springer International, pp 559–564
Vicars B (2005) URL http://asl.ms
Wilcox S (1992) The phonetics of fingerspelling. John Benjamins Publishing, Amsterdam
Wolfe R, Alba N, Billups S, Davidson MJ, Dwyer C, Jamrozik D, (2006) An improved tool for practicing fingerspelling recognition. In: Conference 2006 international conference on technology and persons with disabilities, pp 17–22
Wolfe R, Mcdonald J, Toro J, Baowidan S, Moncrief R, Schnepp J (2015) Promoting better deaf/hearing communication through an improved interaction design for fingerspelling practice. International conference on universal access in human-computer interaction, pp 495–505
van Zijl L, Raitt L (2004) Implementation experience with collision avoidance in signing avatars. Proceedings of the 3rd international conference on Computer graphics, virtual reality, visualisation and interaction in Africa, pp 55–59
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Baowidan, S. Improving realism in automated fingerspelling of American sign language. Machine Translation 35, 387–404 (2021). https://doi.org/10.1007/s10590-021-09273-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10590-021-09273-1