Abstract
Brain–Computer Interfaces (BCI) are communication systems which can convey messages through brain activity alone. Recently BCIs were gaining interest among the virtual reality (VR) community since they have appeared as promising interaction devices for virtual environments (VEs). Especially these implicit interaction techniques are of great interest for the VR community, e.g., you are imaging the movement of your hand and the virtual hand is moving, or you can navigate through houses or museums by your thoughts alone or just by looking at some highlighted objects. Furthermore, VE can provide an excellent testing ground for procedures that could be adapted to real world scenarios, especially patients with disabilities can learn to control their movements or perform specific tasks in a VE. Several studies will highlight these interactions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
See, for instance, the work achieved as part of the BrainAble project: http://www.brainable.org/
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
References
Allison, B.Z., McFarland, D.J., Schalk, G., Zheng, S.D., Jackson, M.M., Wolpaw, J.R.: Towards an independent brain–computer interface using steady state visual evoked potentials. Clin. Neurophysiol. 119(2), 399–408 (2008)
Ammon, K., Gandevia, S.C.: Transcranial magnetic stimulation can influence the selection of motor programmes. J. Neurol. Neurosurg. Psychiatry 53(8), 705–707 (1990)
Bayliss, J.D.: Use of the evoked potential P3 component for control in a virtual apartment. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 113–116 (2003)
Bayliss, J.D., Ballard, D.H.: A virtual reality testbed for brain–computer interface research. IEEE Trans. Rehabil. Eng. 8(2), 188–90 (2000)
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., Flor, H.: A spelling device for the paralysed. Nature 398, 297–298 (1999)
Bowman, D., Kruijff, E., Jr, J.L., Poupyrev, I.: 3D User Interfaces: Theory and Practice. Addison-Wesley/Pearson Education, Redwood, USA (2005)
Burdea, G.: Force and touch feedback for virtual reality. Wiley, New York, USA (1996)
Burdea, G., Coiffet, P.: Virtual Reality Technology. Wiley, New York, USA (2003)
Clark, A.: Supersizing the mind: Embodiment, action, and cognitive extension. Oxford University Press, USA (2008)
Congedo, M., Goyat, M., Tarrin, N., Varnet, L., Rivet, B., Ionescu, G., Jrad, N., Phlypo, R., Acquadro, M., Jutten, C.: “Brain Invaders”: a prototype of an open-source P300-based video game working with the OpenViBE platform. In: 5th International BCI Conference (2011)
Cruz-Neira, C., Sandin, D., Defanti, T., Kentyon, R., Hart, J.: The CAVE : audio visual experience automatic virtual environment. Commun. ACM 35(6), 64–72 (1992)
Damen, E.J., Brunia, C.H.: Changes in heart rate and slow brain potentials related to motor preparation and stimulus anticipation in a time estimation task. Psychophysiology 24(6), 700–713 (1987)
Decety, J., Jeannerod, M., Germain, M., Pastene, J.: Vegetative response during imagined movement is proportional to mental effort. Behav. Brain Res. 42, 1–5 (1991)
Donchin, E., Spencer, K.M., Wijesinghe, R.: The mental prosthesis: assessing the speed of a P300-based brain–computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. 8, 174–179 (2000)
Edlinger, G., Holzner, C., Groenegress, C., Guger, C., Slater, M.: Goal-oriented control with brain–computer interface. In: Lecture Notes in Computer Science, Springer, Berlin/Heidelberg, vol. 5638, pp. 732–740 (2009)
Edlinger, G., Holzner, C., Guger, C.: A hybrid brain–computer interface for smart home control. In: Human Computer Interface Conference, Springer, Berlin/Heidelberg, 417–425 (2011)
Elshout, J., Molina, G.G.: Review of brain–computer interfaces based on the P300 evoked potential. Tech. Rep. PR-TN 2009/00066, Koninklijke Philips Electronics (2009)
Faller, J., Allison, B., Brunner, C., Schmalstieg, D., Pfurtscheller, G.: A software SSVEP BCI integrating stimuli within motivating and immersive virtual and augmented reality environments. In: Real Actions in Virtual Environments (RAVE) conference, Barcelona, Spain (2010a)
Faller, J., Leeb, R., Pfurtscheller, G., Scherer, R.: Avatar navigation in virtual and augmented reality environments using an SSVEP BCI. In: International Conference on Applied Bionics and Biomechanics (ICABB) 2010, Venice, Italy (2010b)
Faller, J., Müller-Putz, G.R., Schmalstieg, D., Pfurtscheller, G.: An application framework for controlling an avatar in a desktop based virtual environment via a software SSVEP brain–computer interface. Presence (Camb.) 19(1), 25–34 (2010c)
Fellner, D., Havemann, S., Hopp, A.: Dave - eine neue technologie zur preiswerten und hochqualitativen immersiven 3d-darstellung. In: Möller, R. (ed.) Proc. 8. Workshop: Sichtsysteme - Visualisierung in der Simulationstechnik, pp. 77–83. Shaker Verlag, Bremen (2003)
Friedman, D., Donenfeld, A., Zafran, E.: Neurophysiology-based art in immersive virtual reality. Int. J. Arts Technol. 2(4), 331–347 (2009)
George, L., Lécuyer, A.: An overview of research on passive brain–computer interfaces for implicit human–computer interaction. In: International Conference on Applied Bionics and Biomechanics (2010)
Groenegress, C., Holzner, C., Guger, C., Slater, M.: Effects of P300-based BCI use on reported presence in a virtual environment. Presence (Camb.) 19(1), 1–11 (2010)
Grychtol, B., Lakany, H., Valsan, G., Conway, B.A.: Human behavior integration improves classification rates in real-time BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 18(4), 362–368 (2010)
Guger, C., Schlögl, A., Neuper, C., Walterspacher, D., Strein, T., Pfurtscheller, G.: Rapid prototyping of an EEG-based brain–computer interface (BCI). IEEE Trans. Rehab. Eng. 9(1), 49–58 (2001)
Guger, C., Holzner, C., Groenegress, C., Edlinger, G., Slater, M.: Control of a smart home with a brain–computer interface. In: 4th International Brain–Computer Interface Workshop, pp. 339–342 (2008)
Guger, C., Daban, S., Sellers, E., Holzner, C., Krausz, G., Carabalona, R., Gramatica, F., Edlinger, G.: How many people are able to control a P300-based brain–computer interface (BCI)? Neurosci. Lett. 462(1), 94–98 (2009a)
Guger, C., Holzner, C., Groenegress, C., Edlinger, G., Slater, M.: Brain–computer interface for virtual reality control. In: Proceedings of ESANN 2009, pp. 443–448 (2009b)
Guger, C., Edlinger, G., Krausz, G.: Recent Advances in Brain–Computer Interface Systems, InTech, chap. Hardware/Software Components and Applications of BCIs, Rijeka, Croatia, 1–24 (2011)
Haggard, P.: Conscious intention and motor cognition. Trends Cogn. Sci. 9(6), 290–295 (2005)
Kronegg, J., Chanel, G., Voloshynovskiy, S., Pun, T.: EEG-based synchronized brain–computer interfaces: A model for optimizing the number of mental tasks. IEEE Trans. Neural Syst. Rehabil. Eng. 15(1), 50–58 (2007)
Krusienski, D., Sellers, E., Cabestaing, F., Bayoudh, S., McFarland, D., Vaughan, T., Wolpaw, J.: A comparison of classification techniques for the P300 speller. J. Neural Eng. 3, 299–305 (2006)
Kuebler, A., Nijboer, F., Mellinger, J., Vaughan, T.M., Pawelzik, H., Schalk, G., McFarland, D.J., Birbaumer, N., Wolpaw, J.R.: Patients with ALS can use sensorimotor rhythms to operate a brain–computer interface. Neurology 64(10), 1775–1777 (2005)
Lalor, E., Kelly, S., Finucane, C., Burke, R., Smith, R., Reilly, R.B., McDarby, G.: Steady-state vep-based brain computer interface control in an immersive 3-d gaming environment. EURASIP J. Appl. Signal Process. 19, 3156–3164 (2005)
Lécuyer, A.: Using eyes, hands, and brain for 3D interaction with virtual environments: A perception-based approach. Tech. rep., Habilitation thesis (2010)
Lécuyer, A., Lotte, F., Reilly, R., Leeb, R., Hirose, M., Slater, M.: Brain–computer interfaces, virtual reality and videogames. IEEE Computer 41(10), 66–72 (2008)
Leeb, R.: Brain-computer communication: the motivation, aim, and impact of virtual feedback. PhD thesis, Graz University of Technology (2008)
Leeb, R., Keinrath, C., Friedman, D., Guger, C., Scherer, R., Neuper, C., Garau, M., Antley, A., Steed, A., Slater, M., Pfurtscheller, G.: Walking by thinking: the brainwaves are crucial, not the muscles! Presence (Camb.) 15, 500–514 (2006)
Leeb, R., Friedman, D., Müller-Putz, G.R., Scherer, R., Slater, M., Pfurtscheller, G.: Self-paced (asynchronous) BCI control of a wheelchair in virtual environments: a case study with a tetraplegics. Comput. Intell. Neurosci. 2007, 79,642 (2007a)
Leeb, R., Lee, F., Keinrath, C., Scherer, R., Bischof, H., Pfurtscheller, G.: Brain-computer communication: motivation, aim and impact of exploring a virtual apartment. IEEE Trans. Neural Syst. Rehabil. Eng. 15, 473–482 (2007b)
Leeb, R., Scherer, R., Friedman, D., Lee, F.Y., Keinrath, C., Bischof, H., Slater, M., Pfurtscheller, G.: Combining BCI and virtual reality: scouting virtual worlds. In: Dornhege, G., Millán, J., Hinterberger, T., McFarland, D.J., Müller, K.R. (eds.) Toward brain–computer interfacing, chap 23, pp. 393–408. MIT Press, Cambridge/London (2007c)
Leeb, R., Settgast, V., Fellner, D.W., Pfurtscheller, G.: Self-paced exploring of the Austrian National Library through thoughts. Int. J. Bioelectromagn. 9, 237–244 (2007d)
Legény, J., Viciana-Abad, R., Lécuyer, A.: Navigating in virtual worlds using a self-paced SSVEP-based brain–computer interface with integrated stimulation and real-time feedback. Presence – Teleoperators and Virtual Environments, vol 20(6), 529–544, 2011
Lotte, F.: Brain–computer interfaces for 3D games: Hype or hope? In: Foundations of Digital Games, pp. 325–327 (2011)
Lotte, F., Renard, Y., Lécuyer, A.: Self-paced brain–computer interaction with virtual worlds: a qualitative and quantitative study “out-of-the-lab.” In: 4th International Brain–Computer Interface Workshop and Training Course, pp. 373–378 (2008)
Lotte, F., Langhenhove, A.V., Lamarche, F., Ernest, T., Renard, Y., Arnaldi, B., Lécuyer, A.: Exploring large virtual environments by thoughts using a brain–computer interface based on motor imagery and high-level commands. Presence (Camb.) 19(1), 54–70 (2010)
Mason, S., Kronegg, J., Huggins, J., Fatourechi, M., Schloegl, A.: Evaluating the performance of self-paced BCI technology. Tech. rep., Neil Squire Society (2006)
Millán JdR., Rupp, R., Müller-Putz, G., Murray-Smith, R., Giugliemma, C., Tangermann, M., Kübler, A., Leeb, R., Neuper, C., Müller, K.R., Mattia, D.: Combining brain-computer interfaces and assistive technologies: state-of-the-art and challenges. Front. Neuroprosthetics, 4(161), 1–15 (2010)
Neuper, C., Scherer, R., Wriessnegger, S., Pfurtscheller, G.: Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain–computer interface. Clin. Neurophysiol. 120(2), 239–247 (2009)
Nijholt, A., Tan, D., Pfurtscheller, G., Brunner, C., del R Millán, J., Allison, B., Graimann, B., Popescu, F., Blankertz, B., Müller, K.R.: Brain–computer interfacing for intelligent systems. IEEE Intell. Syst. 23, 72–79 (2008)
Nijholt, A., Bos, D.P.O., Reuderink, B.: Turning shortcomings into challenges: Brain–computer interfaces for games. Entertain. Comput. 1(2), 85–94 (2009)
Oishi, K., Kasai, T., Maeshima, T.: Autonomic response specificity during motor imagery. J. Physiol. Anthropol. Appl. Human Sci. 19(6), 255–261 (2000)
Pfurtscheller, G., Lopes da Silva, F.H.: Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin. Neurophysiol. 110, 1842–1857 (1999)
Pfurtscheller, G., Neuper, C.: Motor imagery and direct brain–computer communication. Proc. IEEE 89, 1123–1134 (2001)
Pfurtscheller, G., Leeb, R., Keinrath, C., Friedman, D., Neuper, C., Guger, C., Slater, M.: Walking from thought. Brain Res. 1071(1), 145–152 (2006a)
Pfurtscheller, G., Leeb, R., Slater, M.: Cardiac responses induced during thought-based control of a virtual environment. Int. J. Psychophysiol. 62, 134–140 (2006b)
Pfurtscheller, G., Müller-Putz, G.R., Schlögl, A., Graimann, B., Scherer, R., Leeb, R., Brunner, C., Keinrath, C., Lee, F., Townsend, G., Vidaurre, C., Neuper, C.: 15 years of BCI research at Graz University of Technology: current projects. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 205–210 (2006c)
Pfurtscheller, G., Leeb, R., Friedman, D., Slater, M.: Centrally controlled heart rate changes during mental practice in immersive virtual environment: a case study with a tetraplegic. Int. J. Psychophysiol. 68, 1–5 (2008)
Pfurtscheller, G., Allison, B., Bauernfeind, G., Brunner, C., Solis Escalante, T., Scherer, R., Zander, T., Müller-Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Front. Neurosci. 4, 42 (2010)
Pineda, J.A., Silverman, D.S., Vankov, A., Hestenes, J.: Learning to control brain rhythms: making a brain–computer interface possible. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 181–184 (2003)
Plass-Oude Bos, D., Duvinage, M., Oktay, O., Delgado Saa, J., Guruler, H., Istanbullu, A., Van Vliet, M., Van de Laar, B., Poel, M., Roijendijk, L., Tonin, L., Bahramisharif, A., Reuderink, B.: Looking around with your brain in a virtual world. In: IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain (SSCI’2011 CCMB) (2011)
Renard, Y., Lotte, F., Gibert, G., Congedo, M., Maby, E., Delannoy, V., Bertrand, O., Lécuyer, A.: OpenViBE: An open-source software platform to design, test and use brain–computer interfaces in real and virtual environments. Presence (Camb.) 19(1), 35–53 (2010)
Ron-Angevin, R., Diaz-Estrella, A.: Brain–computer interface: Changes in performance using virtual reality technique. Neurosci. Lett. 449(2), 123–127 (2009)
Ron-Angevin, R., Diaz-Estrella, A., Velasco-Alvarez, F.: A two-class brain computer interface to freely navigate through virtual worlds. Biomedizinische Biomed. Tech. (Berl.) 54(3), 126–133 (2009)
Scherer, R., Schlögl, A., Lee, F., Bischof, H., Jansa, J., Pfurtscheller, G.: The self-paced graz brain–computer interface: methods and applications. Comput. Intell. Neurosci. 2007, 1–9 (2007) (Article ID 79826)
Scherer, R., Lee, F., Schlögl, A., Leeb, R., Bischof, H., Pfurtscheller, G.: Towards self-paced brain–computer communication: Navigation through virtual worlds. IEEE Trans. Biomed. Eng. 55(2), 675–682 (2008)
Slater, M., Usoh, M., Steed, A.: Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans. Comput. Hum. Interact. 2(3), 201–219 (1995)
Taylor, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J., Helser, A.: VRPN: A device-independent, network-transparent VR peripheral system. In: VRST ’01, Proceedings of the ACM symposium on Virtual reality software and technology, pp. 55–61. ACM, New York, NY, USA (2001)
Touyama, H.: Advances in Human Computer Interaction, InTech Education and Publishing, chap Brain-CAVE Interface Based on Steady-State Visual Evoked Potential, pp. 437–450 (2008). No. 26 in ISBN 978-953-7619-15-2
Vaughan, T.M., Wolpaw, J.R., Donchin, E.: EEG-based communication: prospects and problems. IEEE Trans. Rehabil. Eng. 4, 425–430 (1996)
Velasco-Álvarez, F., Ron-Angevin, R.: Free virtual navigation using motor imagery through an asynchronous brain–computer interface. Presence (Camb.) 19(1), 71–81 (2010)
Vialatte, F., Maurice, M., Dauwels, J., Cichocki, A.: Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives. Prog. Neurobiol. 90, 418–438 (2010)
Williamson, J., Murray-Smith, R., Blankertz, B., Krauledat, M., Müller, K.: Designing for uncertain, asymmetric control: Interaction design for brain–computer Int. J. Hum. Comput. Stud. 67(10), 827–841 (2009)
Wolpaw, J., Birbaumer, N., McFarland, D., Pfurtscheller, G., Vaughan, T.: Brain–computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)
Zander, T., Kothe, C.: Towards passive brain–computer interfaces: applying brain–computer interface technology to human-machine systems in general. J. Neural Eng. 8(2), 025005 (2011)
Zickler, C., Donna, V.D., Kaiser, V., Al-Khodairy, A., Kleih, S., Kuebler, A., Malavasi, M., Mattia, D., Mongardi, S., Neuper, C., Rohm, M., Rupp, R.: Brain computer interaction applications for people with disabilities: Defining user needs and user requirements. In: AAATE (2009)
Acknowledgements
This work was supported by the European Union projects PRESENCIA (IST-2001-37927) and PRESENCCIA (IST-2006-27731), furthermore by the French National Research Agency projects OpenViBE (ANR-05-RNTL01601) and OpenViBE2 (ANR-09- CORD-017).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lotte, F. et al. (2012). Combining BCI with Virtual Reality: Towards New Applications and Improved BCI. In: Allison, B., Dunne, S., Leeb, R., Del R. Millán, J., Nijholt, A. (eds) Towards Practical Brain-Computer Interfaces. Biological and Medical Physics, Biomedical Engineering. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29746-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-29746-5_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29745-8
Online ISBN: 978-3-642-29746-5
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)