Abstract
An experimental multimodal system, designed for polysensory diagnosis and stimulation of persons with impaired communication skills or even non-communicative subjects is presented. The user interface includes an eye tracking device and the EEG monitoring of the subject. Furthermore, the system consists of a device for objective hearing testing and an autostereoscopic projection system designed to stimulate subjects through their immersion in a virtual environment. Data analysis methods are described, and experiments associated with classification of mental states during listening exercises as well as audio-visual stimuli are presented and discussed. Feature extraction was based on discrete wavelet transformation and clustering employing the k-means algorithm was designed. All algorithms were implemented in the Python programming language with the use of Open Source libraries. Tests of the proposed system were performed in a Special School and Educational Center in Kościerzyna, Poland. Results and comparison with data gathered from the control group of healthy people are presented and discussed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Peeters, W., van den Brande, R., Polinder, S., Brazinova, A., Steyerberg, E.W., Lingsma, H.F., Maas, A.I.R.: Epidemiology of traumatic brain injury in Europe. Acta Neurochir. 157(1), 1683–1696 (2015)
Schnakers, C., Vanhaudenhuyse, A., Giacino, J., Ventura, M., Boly, M., Majerus, S., Moonen, G., Laureys, S.: Diagnostic accuracy of the vegetative and minimally conscious state: clinical consensus versus standardized neurobehavioral assessment. BMC Neurol. 9, 35 (2009)
Kunka, B., Sanner, T., Czyżewski, A., Kwiatkowska, A.: Consciousness study of subjects with unresponsive wakefulness syndrome employing multimodal interfaces. In: Proceedings of the International Conference on Brain Informatics and Health, Warsaw (2014)
Kunka, B., Czyżewski, A., Kwiatkowska, A.: Awarness evaluation of patients in vegetative state employing eye-gaze tracking system. Int. J. Artif. Intell. Tools 21(2), 1240007 (2012)
Rodrigues, R.A., Busssiere, M., Froeschl, M., Nathan, H.J.: Auditory-evoked potentials during coma: do they improve our prediction of awakening in comatose patients? J. Critic. Care 29, 93–100 (2014)
Skoe, E., Kraus, N.: Auditory brainstem response to complex sounds: a tutorial. Ear Hear. 31(3), 302–324 (2010)
Jones, E., Oliphant, E., Peterson, P., et al.: SciPy: Open Source Scientific Tools for Python (2001) http://www.scipy.org. Accessed 04 Feb 2017
http://www.numpy.org/. Accessed 04 Feb 2017
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. JMLR 12, 2825–2830 (2011)
https://pywavelets.readthedocs.io. Accessed 04 Feb 2017
Tobii – EyeX Controller technical specification. www.tobii.com/xperience/products/#Specification. Accessed 14 Sept 2016
Cichocki, A.: Blind signal processing methods for analyzing multichannel brain signals. Int. J. Bioelectromagnetism 6(1), 22–27 (2014)
Daud, S., Sudirman, R.: Butterworth bandpass and stationary wavelet transform filter comparison for electroencephalography signal. In: 6th International Conference on Intelligent Systems, Modelling and Simulation, Kuala Lumpur (2015)
Al-Quazz, N., Ali, S., Ahmad, S., Islam, M., Escudero, J.: Selection of mother wavelet functions for multi-channel EEG signal analysis during a working memory task. Sensors 15, 29015–29035 (2015). doi:10.3390/s151129015
Lin, S., Huang, Z.: Feature extraction of P300 s in EEG signal with discrete wavelet transform and fisher criterion. In: 2015 8th International Conference on Biomedical Engineering and Informatics, Shenyang (2015). doi:10.1109/BMEI.2015.7401500
Czyżewski, A., Łopatka, K., Kunka, B., Rybacki, R., Kostek, B.: Speech synthesis controlled by eye gazing. In: 129th Convention of the Audio Engineering Society, preprint 8165, San Francisco, USA (2010)
Kostek, B.: Music information retrieval in music repositories. In: Skowron, A., Suraj, Z. (eds.) Rough Sets and Intelligent Systems - Professor Zdzisław Pawlak in Memoriam. Intelligent Systems Reference Library, vol. 42, pp. 463–489. Springer, Heidelberg (2013)
Güneş, S., Polat, K., Yosunkaya, S., Efficient sleep stage recognition system based on EEG signal using k-means clustering based feature weighting. Expert Syst. Appl. 37, 7922–7928 (2010). DOI:http://doi.org/10.1016/j.eswa.2010.04.043
Acknowledgement
The project was funded by the National Science Centre on the basis of the decision number DEC-2013/11/B/ST8/04328.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kurowski, A. et al. (2017). Multimodal System for Diagnosis and Polysensory Stimulation of Subjects with Communication Disorders. In: Kryszkiewicz, M., Appice, A., Ślęzak, D., Rybinski, H., Skowron, A., Raś, Z. (eds) Foundations of Intelligent Systems. ISMIS 2017. Lecture Notes in Computer Science(), vol 10352. Springer, Cham. https://doi.org/10.1007/978-3-319-60438-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-60438-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60437-4
Online ISBN: 978-3-319-60438-1
eBook Packages: Computer ScienceComputer Science (R0)