Abstract
In Brazil, there are more than 9.3 million people with Hearing Impairment (PWHI) and deaf who face daily accessibility difficulties. On the other hand, there is the growth of the use of mobile devices and the application of the Internet of Things. The motivation for the development of this work lies in the absence of user interface systems based on sign language and customizable according to the profile that is applied to the prevention of risks external to the health of the deaf. This paper proposes the Apollo SignSound system, which promotes accessibility for PWHI and deaf people in a smart home environment, especially regarding safety. The scientific contribution of Apollo SignSound lies in the detection of ambient risks using neural networks. Besides this, SignSound also considers the user profile, mainly the degree of deafness, to generate accessible notifications represented in Brazilian Sign Language (LIBRAS in Portuguese). The notifications of risk sent to the smartphone of PWHI or deaf also can vibrate or turn on the light of the device. We implemented a prototype of a smart home that collects environmental sounds and notifies the deaf user. The scenario-based assessment included three activities of daily living events of a deaf user: a kettle boiling over the stove, a dog barking, and one person knocking on the door. The results indicate the means of the f score of 0.73 for accuracy evaluation. Usability and acceptance evaluations were performed by five deaf users and the results indicate the approval of 90% in the perceived ease of use and perceived usefulness.
Similar content being viewed by others
Notes
Available at: https://bit.ly/2s5AEZE
References
Abafire (2020) Sirenes pne. https://abafire.com.br/c/alarme-pne/sirenes-pne/. Accessed 4 Oct 2020
Abech M, Da Costa CA, Barbosa JLV, Rigo SJ, Da Rosa Righi R (2016) A model for learning objects adaptation in light of mobile and context-aware computing. Pers Ubiquit Comput 20:167–184. https://doi.org/10.1007/s00779-016-0902-3
Barbosa J (2015) Ubiquitous computing: Applications and research opportunities (invited talk). In: VI IEEE International conference on computational intelligence and computing research (ICCIC), Madurai, Índia pp 1–8. https://ieeexplore.ieee.org/document/7435625
Barbosa J, Hahn R, Barbosa D, Saccol A (2011) A ubiquitous learning model focused on learner interaction. Int J Learn Technol 6(1):62–83. https://doi.org/10.1504/IJLT.2011.040150
Barbosa DNF, Barbosa JLV, Bassani PBS, Rosa JH, Lewis M, Nino CP (2012) Content management in a ubiquitous learning environment. Int J Comput Appl Technol 46:24. https://doi.org/10.1504/IJCAT.2013.051385
Barbosa JLV, Barbosa DNF, Wagner A (2012) Learning in ubiquitous computing environments. Int J Inf Commun Technol Educ 8:64–77. https://doi.org/10.4018/jicte.2012070108
Barbosa J, Martins C, Franco L, Barbosa D (2016) Trailtrade: a model for trail-aware commerce support. Comput Ind 80:45–53. https://doi.org/10.1016/j.compind.2016.04.006
Barbosa J, Tavares J, Cardoso I, Mota B, Martini B (2018) Trailcare: an indoor and outdoor context-aware system to assist wheelchair users. Int J Hum Comput Stud 104:1–14. https://doi.org/10.1016/j.ijhcs.2018.04.001
Bavaresco R, Barbosa JLV, Vianna HD, Buttenbender PC, Dias LPS (2020) Design and evaluation of a context-aware model based on psychophysiology. Comput Methods Progr Biomed 189:105–299. https://doi.org/10.1016/j.cmpb.2019.105299
Bempong J, Stainslow J, Behm G (2020) Accessible smart home system for the deaf and hard-of-hearing. https://www.rit.edu/ntid/nyseta/sites/rit.edu.ntid.nyseta/files/docs/fullpapers_PDFs/StanislowJoeFullPaper.pdf
BrazilianMinistryOfHealth: Acessibilidade. http://portalms.saude.gov.br/acessibilidade (2020). Accessed 4 Oct 2020
Carneiro C, Barbosa J (2016) Hermes: um modelo para acessibilidade ubíqua dedicado à deficiência auditiva. Revista Brasileira de Computação Aplicada 8(3):19–33. https://doi.org/10.5335/rbca.v8i3.5816
Cho H, Kim H (2020) Home environmental sound alert system for deaf and hard-of-hearing users. http://spandh.dcs.shef.ac.uk/chat2017/papers/CHAT_2017_cho.pdf
da Surdez C (2020) Babá eletrônica vibratória para surdos. https://cronicasdasurdez.com/baba-eletronica-vibratoria-surdos/. Accessed 4 Oct 2020
da Surdez C (2020) Os perigos da surdez. http://cronicasdasurdez.com/os-perigos-da-surdez/. Accessed 4 Oct 2020
Damasceno VH, Victória Barbosa JL (2019) A scalable model for building context-aware applications for noncommunicable diseases prevention. Inf Process Lett 148:1–6. https://doi.org/10.1016/j.ipl.2019.03.010
Davis H, Silverman R (1970) Hearing and deafness. Holt, Rinehart and Winston, New York, p 522
Dey AK (2001) Understanding and using context. Pers Ubiquit Comput 5:4–7. https://doi.org/10.1007/s007790170019
Dias LPS, Barbosa JLV, Vianna HD (2018) Gamification and serious games in depression care: a systematic mapping study. Telemat Informat 35:213–224. https://doi.org/10.1016/j.tele.2017.11.002
Endsley MR (1995) Toward a theory of situation awareness in dynamic systems. Hum Fact 37(1):32–64. https://doi.org/10.1518/001872095779049543
Ferreira LGA, Matter VK, Barbosa DNF, Gluz JC, Barbosa JLV (2020) Using learners group profile for content recommendation in ubiquitous environments. Int J Inf Commun Technol Educ 16:1–19. https://doi.org/10.4018/IJICTE.2020100101
Franco LK, Rosa JH, Barbosa JLV, Costa CA, Yamin AC (2011) Mucs: a model for ubiquitous commerce support. Electron Commerce Res Appl 10:237–246. https://doi.org/10.1016/j.elerap.2010.08.006
Google (2020) Google forms. https://www.google.com/forms/about/. Accessed 4 Oct 2020
Gunnarsdóttir K, Arribas-Ayllon M (2020) Ambient intelligence: a narrative in search of users. https://www.academia.edu/1080720/Ambient_Intelligence_an_innovation_narrative. Accessed 4 Oct 2020
Handtalk (2020) Tradutor libras. https://www.handtalk.me/. Accessed 4 Oct 2020
Heart-it (2020) O que é deficiência. https://www.hear-it.org/pt/deficiencia-auditiva. Accessed 4 Oct 2020
Helfer GA, Barbosa JLV, Santos RB, Costa AB (2020) A computational model for soil fertility prediction in ubiquitous agriculture. Comput Eletron Agric 175:1–26. https://doi.org/10.1016/j.compag.2020.105602
IBGE (2020) Censo demográfico 2010. https://www.ibge.gov.br/estatisticas-novoportal/sociais/populacao/9662-censo-demografico-2010.html?edicao=9749&t=destaques. Accessed 4 Oct 2020
Kim G, Shin S, Kim J, Kim H (2018) Sound event detection and haptic vibration based home monitoring assistant system for the deaf and hard-of-hearing. In: Proceedings of the 2018 workshop on multimedia for accessible human computer interface (MAHCI’18), pp 1–7. ACM, New York, NY, USA. https://doi.org/10.1145/3264856.3264857
Kokar MM, Matheus CJ, Baclawski K (2009) Ontology-based situation awareness. Inf Fus Spec Issue High Level Inf Fus Situat Awareness 10(1):83–98
Larentis A, Barbosa DNF, Silva CR, Barbosa JL (2019) Applied computing to education on noncommunicable chronic diseases: a systematic mapping study. Telemed e-Health 1:1–10. https://doi.org/10.1089/tmj.2018.0282
Likert R (1932) A technique for the measurement of attitudes. Arch Psychol 22:1–55
Pittoli F, Vianna HD, Victória Barbosa JL, Butzen E, Gaedke MA, Dias da Costa JS, Scherer do Santos RB (2018) An intelligent system for prognosis of noncommunicable diseases’ risk factors. Telemat Informat 1:1–34. https://doi.org/10.1016/j.tele.2018.02.005
Planalto (2020) Libras law. http://www.planalto.gov.br/ccivil_03/leis/2002/l10436.htm. Accessed 4 Oct 2020
ProDeaf (2020) Tradutor libras. https://www.prodeaf.net/. Accessed 4 Oct 2020
Sadri F (2011) Ambient intelligence: a survey. ACM Comput Surv 43:36. https://doi.org/10.1145/1978802.1978815
Santos B (2020) Internet das coisas: da teoria à prática. http://homepages.dcc.ufmg.br/~mmvieira/cc/papers/internet-das-coisas.pdf. Accessed 4 Oct 2020
Satyanarayanan M (2001) Pervasive computing: vision and challenges. IEEE Pers Commun 8:10–17. https://doi.org/10.1109/98.943998
Scenihr (2020)Health risks from exposure to noise from personal music players. http://ec.europa.eu/health/ph_risk/committees/04_scenihr/docs/scenihr_o_018.pdf. Accessed 4 Oct 2020
SilenceProject (2020) Projeto silence. https://www.projetosilence.com/processo. Accessed 4 Oct 2020
SNPD (2020) Acessibilidade. https://www.mdh.gov.br/navegue-por-temas/pessoa-com-deficiencia/programas/acessibilidade. Accessed 4 Oct 2020
Tavares J, Barbosa J, Cardoso I, Costa C, Yamin A, Real R (2016) Hefestos: an intelligent system applied to ubiquitous accessibility. Univ Access Inf Soc 8:1–19. https://doi.org/10.1007/s10209-015-0423-2
Thomas T (2020) Smart assistant for deaf and dumb. https://www.mepits.com/project/288/wireless/smart-assistant-for-deaf-and-dumb. Accessed 4 Oct 2020
Vanderheiden G (2008) Ubiquitous accessibility, common technology core, and micro assistive technology: Commentary on “computers and people with disabilities. ACM Trans Access Comput 10:1–10
Vianna HD, Barbosa JLV (2020) Pompilos, a model for augmenting health assistant applications with social media content. J Univ Comput Sci 26:4–32. http://www.jucs.org/jucs_26_1/pompilos_a_model_for
Vianna HD, Barbosa JLV (2014) A model for ubiquitous care of noncommunicable diseases. IEEE J Biomed Health Informat 18:1597–1606. https://doi.org/10.1109/jbhi.2013.2292860
Wagner A, Barbosa JLV, Barbosa DNF (2014) A model for profile management applied to ubiquitous learning environments. Expert Syst Appl 41:2023–2034. https://doi.org/10.1016/j.eswa.2013.08.098
Weiser M (1991) The computer for the 21st century. Sci Am 265:94–104. https://doi.org/10.1145/329124.329126
Yoon C, Kim S (2007) Convenience and tam in a ubiquitous computing environment: the case of wireless lan. Electron Commer Res Appl 6:102–112. https://doi.org/10.1016/j.elerap.2006.06.009
Acknowledgements
The authors thank the Rio Grande do Sul State Research Support Foundation (Fundação de Amparo à Pesquisa do Estado do Rio Grande do Sul (FAPERGS)), the Coordination for the Improvement of Higher Education Personnel—Brazil (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES))—Financing Code 001, the National Council for Scientific and Technological Development (Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)), and the University of Vale do Rio dos Sinos (Universidade do Vale do Rio dos Sinos (Unisinos)) for supporting the development of this study. The authors especially acknowledge the support of the Applied Computing Graduate Program (Programa de Pós-Graduação em Computação Aplicada (PPGCA)) and the Mobile Computing Laboratory (Laboratório de Computação Móvel (Mobilab)) at Unisinos.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Declaration of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Ethics statement
This research did not require ethical approval in accordance with the regulations of the University of Vale do Rio dos Sinos (UNISINOS). The subjects who participated in the evaluation were not patients in treatment, but rather academic volunteers and teachers. They assessed the usability aspects of Apollo SignSound and not the effectiveness of its application. In addition, the participants agreed to participate in the evaluation of Apollo SignSound. Informed consent was obtained from all individual participants included in the study and the data were anonymized.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
da Rosa Tavares, J.E., Victória Barbosa, J.L. Apollo SignSound: an intelligent system applied to ubiquitous healthcare of deaf people. J Reliable Intell Environ 7, 157–170 (2021). https://doi.org/10.1007/s40860-020-00119-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40860-020-00119-w