Abstract
A new genre of learning technologies is emerging that integrates computer simulations with physical or “embodied” interactions such as hand gestures. While this genre presents new opportunities for innovative digital environments that physically engage learners, there is very little guidance on how to design these environments to optimize learning. This chapter presents considerations specifically for the design of gesture-augmented learning environments. Design considerations are discussed in three main areas related to (1) what gestural interactions are used, (2) constraints of the learning environment, and (3) what social and contextual supports are offered. The term considerations is used rather than principles or guidelines to highlight the real tradeoffs and legitimate decisions to be made when designing gesture-based technologies for learning. These considerations are illustrated with detailed examples from a project that implements students’ gestures as the primary method of interaction with digital science simulations. Although the examples specifically pertain to learning in science, the considerations are framed such that they can be applied to a broad range of domains.
References
Abrahamson, D., & Lindgren, R. (2014). Embodiment and embodied design. In Cambridge handbook of the Learning Sciences (2nd ed., pp. 358–376). Cambridge, UK: Cambridge University Press. 10.1017/CBO9781139519526.022.
Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. The Journal of the Learning Sciences, 21, 247–286. 10.1080/10508406.2011.611446.
Black, J. B., Segal, A., Vitale, J., & Fadjo, C. (2012). Embodied cognition and learning environment design. In D. Jonassen & S. Lamb (Eds.), Theoretical foundations of student-centered learning environments (2nd ed., pp. 198–223). New York: Routledge.
Chase, C. C., Chin, D. B., Oppezzo, M. A., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18, 334–352.
Clement, J. (2013). Roles for explanatory models and analogies in conceptual change. In S. Vosniadou (Ed.), International handbook of research on conceptual change (2nd ed., pp. 412–446). New York: Routledge.
Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning and instruction: Essays in honour of Robert Glaser (pp. 453–494). Hillsdale, NJ: Lawrence Erlbaum & Associates.
Dourish, P. (2001). Where the action is: The foundations of embodied interaction. Cambridge, MA: MIT Press.
Engelkamp, J., & Zimmer, H. D. (1994). The human memory: A multi-modal approach. Seattle, WA: Hogrefe & Huber.
Flood, V. J., Amar, F. G., Nemirovsky, R., Harrer, B. W., Bruce, M. R. M., & Wittmann, M. C. (2014). Paying attention to gesture when students talk chemistry: Interactional resources for responsive teaching. Journal of Chemical Education, 92, 11–22. 10.1021/ed400477b.
Gallagher, S. (2005). How the body shapes the mind. Oxford: Oxford University Press.
Gibson, J. J. (1962). Observations on active touch. Psychological Review, 69, 477–491.
Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
Goldin-Meadow, S. (2005). Hearing gesture: How our hands help us think. Cambridge, MA: Harvard University Press.
Goldin-Meadow, S., Cook, S. W., & Mitchell, Z. a. (2009). Gesturing gives children new ideas about math. Psychological Science, 20, 267–272. 10.1111/j.1467-9280.2009.02297.x.
Han, I., & Black, J. B. (2011). Incorporating haptic feedback in simulation for learning physics. Computers & Education, 57, 2281–2290.
Hannafin, M., & Peck, K. (1988). The design, development, and evaluation of instructional software. New York: Macmillan.
Hartman, B. A., Miller, B. K., & Nelson, D. L. (2000). The effects of hands-on occupation versus demonstration on children’s recall memory. American Journal of Occupational Therapy, 54, 477–483.
Hostetter, A. B., & Alibali, M. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15, 495–514. 10.3758/PBR.15.3.495.
Isbister, K., & Mueller, F. F. (2015). Guidelines for the design of movement-based games and their relevance to HCI. Human Computer Interaction, 30, 366–399.
Johnson-Glenberg, M. C., Birchfield, D. A., Tolentino, L., & Koziupa, T. (2014a). Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. Journal of Educational Psychology, 106, 86–104. 10.1037/a0034008.
Johnson-Glenberg, M. C., Savio-Ramos, C., & Henry, H. (2014b). “Alien Health”: A nutrition instruction exergame using the kinect sensor. Games for Health: Research, Development, and Clinical Applications, 3, 241–251.
Johnson-Glenberg, M. C., Megowan-Romanowicz, C., Birchfield, D. A., & Savio-Ramos, C. (2016). Effects of embodied learning and digital platform on the retention of physics content: Centripetal force. Frontiers in Psychology, 7, 1–22. 10.3389/fpsyg.2016.01819.
Kim, M., Roth, W. M., & Thom, J. (2011). Children’s gestures and the embodied knowledge of geometry. International Journal of Science and Mathematics Education, 9, 207–238. 10.1007/s10763-010-9240-5.
Lindgren, R. (2015). Getting into the cue: Embracing technology-facilitated body movements as a starting point for learning. In V. R. Lee (Ed.), Learning technologies and the body: Integration and implementation in formal and informal learning environments (pp. 39–54). New York: Routledge.
Lindgren, R., & Johnson-Glenberg, M. C. (2013). Emboldened by embodiment: Six precepts for research on embodied learning and mixed reality. Educational Researcher, 42, 445–452. 10.3102/0013189X13511661.
Lindgren, R., Wallon, R. C., Brown, D. E., Mathayas, N., & Kimball, N. (2016). “Show me” what you mean: Learning and design implications of eliciting gesture in student explanations. In C. Looi, J. Polman, U. Cress, & P. Reimann (Eds.), Proceedings of the Twelfth International Conference of the Learning Sciences (pp. 1014–1017). Singapore: National Institute of Education.
Mathayas, N., Brown, D. E., & Lindgren, R. (2016). Exploring middle school students’ sense making of a computer simulation about thermal conduction. In C. Looi, J. Polman, U. Cress, & P. Reimann (Eds.), Proceedings of the Twelfth International Conference of the Learning Sciences (pp. 1267–1268). Singapore: National Institute of Education.
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
Nielsen, M., Störring, M., Moeslund, T. B., & Granum, E. (2003). A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In International gesture workshop (pp. 409–420). Springer: Heidelberg.
Paek, S. (2012). The impact of multimodal virtual manipulatives on young children’s mathematics learning (doctoral dissertation). Retrieved from ProQuest dissertations & theses full text (3554708). Ann Arbor, MI.
Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction design: Beyond human-computer interaction. New York: Wiley.
Radford, L. (2009). Why do gestures matter? Sensuous cognition and the palpability of mathematical meanings. Educational Studies in Mathematics, 70, 111–126. 10.1007/s10649-008-9127-3.
Resnick, M. (2002). Rethinking learning in the digital age. In G. S. Kirkman, P. K. Cornelius, J. D. Sachs, & K. Schwab (Eds.), The global information technology report 2001–2002: Readiness for the networked world. New York: Oxford University Press.
Roth, W.-M. (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71, 365–392. 10.3102/00346543071003365.
Schwartz, R. N. (2010). Considering the activity in interactivity: A multimodal perspective (doctoral dissertation). Retrieved from ProQuest dissertations & theses full text (3404551). Ann Arbor, MI.
Segal, A. (2011). Do gestural interfaces promote thinking? Embodied interaction: Congruent gestures and direct touch promote performance in math (doctoral dissertation). Retrieved from ProQuest dissertations & theses full text (3453956). Ann Arbor, MI.
Shapiro, L. (2010). Embodied cognition. New York: Routledge.
Singer, M., Radinsky, J., & Goldman, S. R. (2008). The role of gesture in meaning construction. Discourse Processes, 45, 365–386. 10.1080/01638530802145601.
Wallon, R. C., Brown, D. E., & Lindgren, R. (2016). Student gestures during shifts from descriptions to explanations of gas pressure. Paper presented at the annual meeting of the National Association for Research in Science Teaching, Baltimore, MD.
Wilson, M. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9, 625–636. 10.3758/BF03196322.
Yoon, C., Thomas, M. O., & Dreyfus, T. (2011). Gestures and insight in advanced mathematical thinking. International Journal of Mathematical Education in Science and Technology, 42, 891–901.
Acknowledgments
This material is based upon work supported by the National Science Foundation under Grant No. DUE-1432424. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this entry
Cite this entry
Wallon, R.C., Lindgren, R. (2017). Considerations for the Design of Gesture-Augmented Learning Environments. In: Spector, M., Lockee, B., Childress, M. (eds) Learning, Design, and Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-17727-4_75-1
Download citation
DOI: https://doi.org/10.1007/978-3-319-17727-4_75-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-17727-4
Online ISBN: 978-3-319-17727-4
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education