Skip to main content
SpringerLink
Log in

Investigating Embodied Music Expression Through the Leap Motion: Experimentations in Educational and Clinical Contexts

  • Conference paper
  • First Online:
Computer Supported Education (CSEDU 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1022))

Included in the following conference series:

Abstract

This paper describes a computer-based approach to foster embodied musical expression. The proposed system captures user’s hand gestures through a Leap Motion controller, and the signals thus generated are sent to a software tool that converts movements into music notes. When applied to an educational or a rehabilitation context, this solution may foster the development of communication and motor skills by using free-hand interaction. The conceptual framework was experimented in two contexts: technologically-augmented music lessons in a primary school, and music therapy sessions for the rehabilitation of physically and intellectually impaired people.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The experimentation took place at the Scuola Elementare Del Centro, Via Cesare Battisti 14, 22036 Erba (CO), Italy.

  2. 2.

    At first, only 3 pitches with clear tonal functions could be selected by users.

  3. 3.

    Cooperativa Sociale Sim-patia, Via G. Parini 180, 22070 Valmorea (CO), Italy, http://www.sim-patia.it/.

  4. 4.

    For further details, please refer to [1].

References

  1. Baratè, A., Elia, A., Ludovico, L.A., Oriolo, E.: The Leap Motion controller in clinical music therapy. A computer-based approach to intellectual and motor disabilities. In: McLaren, B.M., Reilly, R., Uhomoibhi, J., Zvacek, S. (eds.) Proceedings of the 10th International Conference on Computer Supported Education, CSEDU 2018, 15–17 March 2018, Funchal, Madeira, Portugal, pp. 461–469. SCITEPRESS - Science and Technology Publications, Lda., Setúbal (2018)

    Google Scholar 

  2. van Beurden, E., Zask, A., Barnett, L.M., Dietrich, U.: Fundamental movement skills: how do primary school children perform? The Move it Groove it program in rural Australia. J. Sci. Med. Sport 5(3), 244–252 (2002)

    Article  Google Scholar 

  3. Cattaneo, P.: La canzone come esperienza relazionale, educativa, terapeutica. Universal Music MGB Publications, Sesto Ulteriano, San Giuliano Milanese (2009)

    Google Scholar 

  4. Elia, A.: Con un fil di voce: dinamiche espressive tra parlato e canto. In: Cattaneo, P., Lopez, L. (eds.) Musicoterapia e Relazione. Franco Angeli, Milano (2017)

    Google Scholar 

  5. Gieser, S.N., Boisselle, A., Makedon, F.: Real-time static gesture recognition for upper extremity rehabilitation using the Leap Motion. In: Duffy, V.G. (ed.) DHM 2015. LNCS, vol. 9185, pp. 144–154. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21070-4_15

    Chapter  Google Scholar 

  6. Godøy, R.I., Leman, M.: Musical Gestures: Sound, Movement, and Meaning. Routledge, Abingdon (2010)

    Book  Google Scholar 

  7. Gray, C.H., Figueroa-Sarriera, H.J., Mentor, S.: The Cyborg Handbook. Routledge, Abingdon (1995)

    Google Scholar 

  8. Halberstam, J.M., Livingston, I.: Posthuman Bodies. Indiana University Press, Bloomington (1995)

    Google Scholar 

  9. Hansson, S.O.: The ethics of enabling technology. Camb. Q. Healthc. Ethics 16(3), 257–267 (2007)

    Article  MathSciNet  Google Scholar 

  10. Hantrakul, L., Kaczmarek, K.: Implementations of the Leap Motion device in sound synthesis and interactive live performance. In: Proceedings of the 2014 International Workshop on Movement and Computing, p. 142. ACM (2014)

    Google Scholar 

  11. Haraway, D.: Simians, Cyborgs, and Women: The Reinvention of Nature. Routledge, Abingdon (1991)

    Google Scholar 

  12. Hiebert, J., Morris, A.K., Berk, D., Jansen, A.: Preparing teachers to learn from teaching. J. Teach. Educ. 58(1), 47–61 (2007)

    Article  Google Scholar 

  13. Hmelo, C.E., Ferrari, M.: The problem-based learning tutorial: cultivating higher order thinking skills. J. Educ. Gifted 20(4), 401–422 (1997)

    Article  Google Scholar 

  14. Iosa, M., et al.: Leap Motion controlled videogame-based therapy for rehabilitation of elderly patients with subacute stroke: a feasibility pilot study. Top. Stroke Rehabil. 22(4), 306–316 (2015)

    Article  Google Scholar 

  15. Karageorghis, C.I., Terry, P.C.: The psychophysical effects of music in sport and exercise: a review. J. Sport Behav. 20(1), 54 (1997)

    Google Scholar 

  16. Khademi, M., Mousavi Hondori, H., McKenzie, A., Dodakian, L., Lopes, C.V., Cramer, S.C.: Free-hand interaction with Leap Motion controller for stroke rehabilitation. In: Proceedings of the Extended Abstracts of the 32nd Annual ACM Conference on Human Factors in Computing Systems, pp. 1663–1668. ACM (2014)

    Google Scholar 

  17. Lerner, N., Straus, J.: Sounding Off: Theorizing Disability in Music. Routledge, Abingdon (2006)

    Google Scholar 

  18. Liu, Z., Zhang, Y., Patrick Rau, P.-L., Choe, P., Gulrez, T.: Leap-Motion based online interactive system for hand rehabilitation. In: Rau, P.L.P. (ed.) CCD 2015. LNCS, vol. 9181, pp. 338–347. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20934-0_32

    Chapter  Google Scholar 

  19. Lubet, A.: Music, Disability, and Society. Temple University Press, Philadelphia (2011)

    Google Scholar 

  20. Ludovico, L.A., Mangione, G.R.: Self-regulation competence in music education. In: McPherson, M., Nunes, M.B. (eds.) Proceedings of the International Conference e-Learning 2014, EL 2014, pp. 46–54. IADIS Press, Lisbon (2014)

    Google Scholar 

  21. Lupton, D., Seymour, W.: Technology, selfhood and physical disability. Soc. Sci. Med. 50(12), 1851–1862 (2000)

    Article  Google Scholar 

  22. Martin, A.J.: Motivation and engagement in music and sport: testing a multidimensional framework in diverse performance settings. J. Pers. 76(1), 135–170 (2008)

    Article  Google Scholar 

  23. McPherson, G.E., Renwick, J.M.: A longitudinal study of self-regulation in children’s musical practice. Music Educ. Res. 3(2), 169–186 (2001)

    Article  Google Scholar 

  24. Penley, C., Ross, A.: Technoculture, vol. 3. University of Minnesota Press, Minneapolis (1991)

    Google Scholar 

  25. Ritter, M., Aska, A.: Leap Motion as expressive gestural interface. In: Proceedings ICMC—SMC—2014, pp. 659–662 (2014)

    Google Scholar 

  26. Riva, G., Baños, R.M., Botella, C., Mantovani, F., Gaggioli, A.: Transforming experience: the potential of augmented reality and virtual reality for enhancing personal and clinical change. Front. Psychiatry 7, 164 (2016)

    Article  Google Scholar 

  27. Santagata, R., Zannoni, C., Stigler, J.W.: The role of lesson analysis in pre-service teacher education: an empirical investigation of teacher learning from a virtual video-based field experience. J. Math. Teach. Educ. 10(2), 123–140 (2007)

    Article  Google Scholar 

  28. Scheufele, P.M.: Effects of progressive relaxation and classical music on measurements of attention, relaxation, and stress responses. J. Behav. Med. 23(2), 207–228 (2000)

    Article  Google Scholar 

  29. Schlaug, G., Norton, A., Overy, K., Winner, E.: Effects of music training on the child’s brain and cognitive development. Ann. N.Y. Acad. Sci. 1060(1), 219–230 (2005)

    Article  Google Scholar 

  30. Straus, J.N.: Extraordinary Measures: Disability in Music. Oxford University Press, Oxford (2011)

    Book  Google Scholar 

  31. Taylor, J., Curran, K.: Using Leap Motion and gamification to facilitate and encourage rehabilitation for hand injuries: Leap Motion for rehabilitation. In: Handbook of Research on Holistic Perspectives in Gamification for Clinical Practice, pp. 183–192 (2015)

    Google Scholar 

  32. Weichert, F., Bachmann, D., Rudak, B., Fisseler, D.: Analysis of the accuracy and robustness of the Leap Motion controller. Sensors 13(5), 6380–6393 (2013)

    Article  Google Scholar 

  33. Zhu, G., Cai, S., Ma, Y., Liu, E.: A series of Leap Motion-based matching games for enhancing the fine motor skills of children with autism. In: 2015 IEEE 15th International Conference on Advanced Learning Technologies (ICALT), pp. 430–431. IEEE (2015)

    Google Scholar 

Download references

Acknowledgements

The authors wish to acknowledge the music therapist Antonio Elia, staff members and patients of the Cooperativa Sociale Sim-patia, Valmorea, Italy for experimenting the Leap Motion in a music therapy context. Besides, the authors would like to thank Monica Mengacci, Daniela Pina, third-grade students attending section B and the entire staff of the Scuola Elementare Del Centro, Erba, Italy for their support in classroom experimentation. Finally, we are grateful to Veronica Curioni, who coordinated and operatively performed the observation phase on primary school students.

Author information

Authors and Affiliations

  1. Laboratorio di Informatica Musicale (LIM), Dipartimento di Informatica “Giovanni Degli Antoni”, Università degli Studi di Milano, Via Celoria 18, 20133, Milano, Italy

    Adriano Baratè, Luca A. Ludovico & Eleonora Oriolo

Authors
  1. Adriano Baratè
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca A. Ludovico
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eleonora Oriolo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca A. Ludovico .

Editor information

Editors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, PA, USA

    Bruce M. McLaren

  2. MIT, Cambridge, MA, USA

    Rob Reilly

  3. University of Denver, Denver, CO, USA

    Susan Zvacek

  4. University of Ulster, Newtownabbey, UK

    James Uhomoibhi

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Baratè, A., Ludovico, L.A., Oriolo, E. (2019). Investigating Embodied Music Expression Through the Leap Motion: Experimentations in Educational and Clinical Contexts. In: McLaren, B., Reilly, R., Zvacek, S., Uhomoibhi, J. (eds) Computer Supported Education. CSEDU 2018. Communications in Computer and Information Science, vol 1022. Springer, Cham. https://doi.org/10.1007/978-3-030-21151-6_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-21151-6_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-21150-9

  • Online ISBN: 978-3-030-21151-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation