Skip to main content
SpringerLink
Log in

Brain-Driven Telepresence Robots: A Fusion of User’s Commands with Robot’s Intelligence

  • Conference paper
  • First Online:
AIxIA 2020 – Advances in Artificial Intelligence (AIxIA 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12414))

Abstract

This paper presents different methodologies to enhance the human-robot interaction during the control of brain-machine interface (BMI) driven telepresence robots. To overcome the limitations of BMIs, namely the low bit rate and the intrinsic uncertainty as a control channel, we hypothesize that the fusion of the user’s commands with the robot’s intelligence is essential to achieve robust and natural systems. Compared to most current neurorobotics works, we exploit the robot as an intelligent agent that contributes at different levels to choose the final action to perform. Furthermore, we present the first implementation of a BMI system inside the Robot Operating System (ROS) designed to facilitate the combination between BMI and robotics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://aixia2020.di.unito.it/awards/premio-pietro-torasso.

  2. 2.

    https://github.com/rosneuro.

  3. 3.

    NeuroFrame is a custom message defined according to the ROS’s standard.

  4. 4.

    NeuroEvent is a custom message defined according to the ROS’s standard.

  5. 5.

    NeuroOutput is a custom message defined according to the ROS’s standard.

  6. 6.

    https://cybathlon.ethz.ch/en.

References

  1. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)

    Article  Google Scholar 

  2. del Millán, J.R., et al.: Combining brain-computer interfaces and assistive technologies: state-of-the-art and challenges. Front. Neurosci. 4, 161 (2010)

    Google Scholar 

  3. Chaudhary, U., Birbaumer, N., Ramos-Murguialday, A.: Brain-computer interfaces for communication and rehabilitation. Nat. Rev. Neurol. 12(9), 513–525 (2016)

    Article  Google Scholar 

  4. Birbaumer, N., et al.: A spelling device for the paralysed. Nature 398(6725), 297–298 (1999)

    Article  Google Scholar 

  5. Lee, K., Liu, D., Perroud, L., Chavarriaga, R., del Millán, J.R.: A brain-controlled exoskeleton with cascaded event-related desynchronization classifiers. Robot. Auton. Syst. 90, 15–23 (2017)

    Article  Google Scholar 

  6. He, Y., Eguren, D., Azorín, J.M., Grossman, R.G., Luu, T.P., Contreras-Vidal, J.L.: Brain-machine interfaces for controlling lower-limb powered robotic systems. J. Neural Eng. 15(2), 1–16 (2018)

    Article  Google Scholar 

  7. Tonin, L., Leeb, R., Tavella, M., Perdikis, S., del Millán, J.R.: The role of shared-control in BCI-based telepresence. In: 2010 IEEE International Conference on Systems, Man and Cybernetics, pp. 1462–1466. IEEE (2010)

    Google Scholar 

  8. Iturrate, I., Antelis, J.M., Kubler, A., Minguez, J.: A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation. IEEE Trans. Robot. 25(3), 614–627 (2009)

    Article  Google Scholar 

  9. Beraldo, G., Antonello, M., Cimolato, A., Menegatti, E., Tonin, L.: Brain-Computer Interface meets ROS: a robotic approach to mentally drive telepresence robots. In: Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1–6. IEEE (2018)

    Google Scholar 

  10. Endsley, M.R.: Level of automation: integrating humans and automated systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Volume 41, SAGE Publications, Sage CA, Los Angeles, CA (1997)

    Google Scholar 

  11. Kaber, D.B., Endsley, M.R.: Out-of-the-loop performance problems and the use of intermediate levels of automation for improved control system functioning and safety. Process Saf. Progr. 16(3), 126–131 (1997)

    Article  Google Scholar 

  12. Murphy, R.R.: Introduction to Al Robotics (2000)

    Google Scholar 

  13. Sheridan, T.B.: Telerobotics. Automatica 25(4), 487–507 (1989)

    Article  Google Scholar 

  14. Goodrich, K., Schutte, P., Flemisch, F., Williams, R.: Application of the H-mode, a design and interaction concept for highly automated vehicles, to aircraft. In: Proceedings of the 25th IEEE Digital Avionics Systems Conference, pp. 1–13 (2006)

    Google Scholar 

  15. Flemisch, F., Abbink, D., Itoh, M., Pacaux-Lemoine, M.P., Weßel, G.: Shared control is the sharp end of cooperation: towards a common framework of joint action, shared control and human machine cooperation. IFAC-PapersOnLine 49(19), 72–77 (2016)

    Article  Google Scholar 

  16. Abbink, D.A., et al.: A topology of shared control systems-finding common ground in diversity. IEEE Trans. Hum.-Mach. Syst. 48(5), 509–525 (2018)

    Article  Google Scholar 

  17. Ferrell, W.R., Sheridan, T.B.: Supervisory control of remote manipulation. IEEE Spectr. 4(10), 81–88 (1967)

    Article  Google Scholar 

  18. Schilling, M., et al.: Towards a multidimensional perspective on shared autonomy. In: Proceedings of the AAAI Fall Symposium Series 2016, Stanford (USA) (2016)

    Google Scholar 

  19. Bradshaw, J.M., Feltovich, P.J., Jung, H., Kulkarni, S., Taysom, W., Uszok, A.: Dimensions of adjustable autonomy and mixed-initiative interaction. In: Nickles, M., Rovatsos, M., Weiss, G. (eds.) AUTONOMY 2003. LNCS (LNAI), vol. 2969, pp. 17–39. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-25928-2_3

    Chapter  MATH  Google Scholar 

  20. Allen, J., Guinn, C.I., Horvtz, E.: Mixed-initiative interaction. IEEE Intell. Syst. Appl. 14(5), 14–23 (1999)

    Article  Google Scholar 

  21. Finzi, A., Orlandini, A.: Human-robot interaction through mixed-initiative planning for rescue and search rovers. In: Bandini, S., Manzoni, S. (eds.) AI*IA 2005. LNCS (LNAI), vol. 3673, pp. 483–494. Springer, Heidelberg (2005). https://doi.org/10.1007/11558590_49

    Chapter  Google Scholar 

  22. Bevacqua, G., Cacace, J., Finzi, A., Lippiello, V.: Mixed-initiative planning and execution for multiple drones in search and rescue missions. In: Proceedings of the International Conference on Automated Planning and Scheduling, vol. 25 (2015)

    Google Scholar 

  23. Horvitz, E.: Principles of mixed-initiative user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 159–166 (1999)

    Google Scholar 

  24. Beraldo, G., Tonin, L., Cesta, A., Menegatti, E.: Shared-control, shared-autonomy and shared-intelligence in assistive technologies: three forms of cooperation between user and robot. In: Proceedings of the IEEE International Workshop Adaptive Behavioral Models of Robotic Models of Robotic Systems Based on Brain-Inspired AI Cognitive Architectures (APHRODITE). IEEE (2020)

    Google Scholar 

  25. Donges, E.: Aspekte der aktiven sicherheit bei der führung von personenkraftwagen. Automob-Ind 27(2) (1982)

    Google Scholar 

  26. Beraldo, G., Termine, E., Menegatti, E.: Shared-autonomy navigation for mobile robots driven by a door detection module. In: Alviano, M., Greco, G., Scarcello, F. (eds.) AI*IA 2019. LNCS (LNAI), vol. 11946, pp. 511–527. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35166-3_36

    Chapter  Google Scholar 

  27. Khatib, O.: Real-time obstacle avoidance for manipulators and mobile robots. In: Cox, I.J., Wilfong, G.T. (eds.) Autonomous Robot Vehicles, pp. 396–404. Springer, New York (1986). https://doi.org/10.1007/978-1-4613-8997-2_29

    Chapter  Google Scholar 

  28. Koren, Y., Borenstein, J.: Potential field methods and their inherent limitations for mobile robot navigation. In: Proceedings of the 1991 IEEE International Conference on Robotics and Automation, pp. 1398–1404. IEEE (1991)

    Google Scholar 

  29. Aigner, P., McCarragher, B.: Human integration into robot control utilising potential fields. In: Proceedings of the International Conference on Robotics and Automation, vol. 1, pp. 291–296. IEEE (1997)

    Google Scholar 

  30. Simpson, R.C., Levine, S.P., Bell, D.A., Jaros, L.A., Koren, Y., Borenstein, J.: NavChair: an assistive wheelchair navigation system with automatic adaptation. In: Mittal, V.O., Yanco, H.A., Aronis, J., Simpson, R. (eds.) Assistive Technology and Artificial Intelligence. LNCS, vol. 1458, pp. 235–255. Springer, Heidelberg (1998). https://doi.org/10.1007/BFb0055982

    Chapter  Google Scholar 

  31. Kim, B.K., Tanaka, H., Sumi, Y.: Robotic wheelchair using a high accuracy visual marker Lentibar and its application to door crossing navigation. In: Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), pp. 4478–4483. IEEE (2015)

    Google Scholar 

  32. Winiarski, T., Banachowicz, K., Seredyński, D.: Multi-sensory feedback control in door approaching and opening. In: Filev, D., et al. (eds.) Intelligent Systems 2014. AISC, vol. 323, pp. 57–70. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-11310-4_6

  33. Carlson, T., Demiris, Y.: Human-wheelchair collaboration through prediction of intention and adaptive assistance. In: Proceedings of the 2008 IEEE International Conference on Robotics and Automation, pp. 3926–3931. IEEE (2008)

    Google Scholar 

  34. Barber, R., Salichs, M.: Mobile robot navigation based on event maps. In: Proceedings of the Field and Service Robotics, pp. 61–66 (2001)

    Google Scholar 

  35. Joo, K., Lee, T.K., Baek, S., Oh, S.Y.: Generating topological map from occupancy grid-map using virtual door detection. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 1–6. IEEE (2010)

    Google Scholar 

  36. Levine, S.P., Bell, D.A., Jaros, L.A., Simpson, R.C., Koren, Y., Borenstein, J.: The NavChair assistive wheelchair navigation system. IEEE Trans. Rehabil. Eng. 7(4), 443–451 (1999)

    Article  Google Scholar 

  37. Zeng, Q., Burdet, E., Rebsamen, B., Teo, C.L.: Evaluation of the collaborative wheelchair assistant system. In: Proceedings of the 2007 IEEE 10th International Conference on Rehabilitation Robotics, pp. 601–608. IEEE (2007)

    Google Scholar 

  38. Hoshino, S., Maki, K.: Safe and efficient motion planning of multiple mobile robots based on artificial potential for human behavior and robot congestion. Adv. Robot. 29(17), 1095–1109 (2015)

    Article  Google Scholar 

  39. Hall, E.T.: The Hidden Dimension, vol. 609. Doubleday, Garden City (1910)

    Google Scholar 

  40. Mead, R., Matarić, M.J.: Autonomous human-robot proxemics: socially aware navigation based on interaction potential. Auton. Robots 41(5), 1189–1201 (2017). https://doi.org/10.1007/s10514-016-9572-2

    Article  Google Scholar 

  41. Rios-Martinez, J., Spalanzani, A., Laugier, C.: From proxemics theory to socially-aware navigation: a survey. Int. J. Soc. Robot. 7(2), 137–153 (2015)

    Article  Google Scholar 

  42. Scandolo, L., Fraichard, T.: An anthropomorphic navigation scheme for dynamic scenarios. In: 2011 IEEE International Conference on Robotics and Automation, pp. 809–814. IEEE (2011)

    Google Scholar 

  43. Zhang, B., Holloway, C., Carlson, T.: A hierarchical design for shared-control wheelchair navigation in dynamic environments. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2020)

    Google Scholar 

  44. Philips, J., et al.: Adaptive shared control of a brain-actuated simulated wheelchair. In: 2007 IEEE 10th International Conference on Rehabilitation Robotics, pp. 408–414. IEEE (2007)

    Google Scholar 

  45. Lopes, A.C., Pires, G., Vaz, L., Nunes, U.: Wheelchair navigation assisted by Human-Machine shared-control and a P300-based Brain Computer Interface. In: Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2438–2444. IEEE (2011)

    Google Scholar 

  46. Tonin, L., Beraldo, G., Tortora, S., Tagliapietra, L., del Millán, J.R., Menegatti, E.: ROS-Neuro: a common middleware for BMI and robotics. The acquisition and recorder packages. In: Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 2767–2772. IEEE (2019)

    Google Scholar 

  47. Beraldo, G., Tortora, S., Menegatti, E., Tonin, L.: ROS-Neuro: implementation of a closed-loop BMI based on motor imagery. In: Proceedings of the 2020 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE (2020)

    Google Scholar 

  48. Beraldo, G., et al.: ROS-Health: an open-source framework for neurorobotics. In: Proceedings of the 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), pp. 174–179. IEEE (2018)

    Google Scholar 

  49. Quigley, M., et al.: ROS: an open-source robot operating system (2009)

    Google Scholar 

Download references

Acknowledgments

This research was partially supported by Fondazione Ing. Aldo Gini, by MIUR (Italian Minister for Education) under the initiative “Departments of Excellence" (Law 232/2016) and by SI Robotics project (Invecchiamento sano e attivo attraverso SocIal ROBOTICS) project – PON 12 Aree Call 2017.

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Padova, Padua, Italy

    Gloria Beraldo, Luca Tonin & Emanuele Menegatti

  2. Institute of Cognitive Sciences and Technologies (ISTC), National Research Council, ISTC-CNR, Rome, Italy

    Gloria Beraldo & Amedeo Cesta

Authors
  1. Gloria Beraldo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Tonin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amedeo Cesta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emanuele Menegatti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gloria Beraldo .

Editor information

Editors and Affiliations

  1. Università degli Studi di Torino, Turin, Italy

    Matteo Baldoni

  2. Department of Informatics, Systems and C, University of Milano-Bicocca, Milan, Italy

    Stefania Bandini

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Beraldo, G., Tonin, L., Cesta, A., Menegatti, E. (2021). Brain-Driven Telepresence Robots: A Fusion of User’s Commands with Robot’s Intelligence. In: Baldoni, M., Bandini, S. (eds) AIxIA 2020 – Advances in Artificial Intelligence. AIxIA 2020. Lecture Notes in Computer Science(), vol 12414. Springer, Cham. https://doi.org/10.1007/978-3-030-77091-4_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77091-4_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77090-7

  • Online ISBN: 978-3-030-77091-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation