Abstract
Physical manipulation is a key part of externalizing representations of knowledge and the creative process. However, contemporary tools for remote collaboration ignore physical manipulation and the haptic modality. We are interested in exploring remote physical manipulation in the context of ideation and brainstorming. Augmented Reality provides much of the benefits of spatial representation of remote participants, yet AR does not allow for rich physical manipulation and haptic feedback. Thus, we propose to use pairs of multi-robot system to provide synchronized haptic proxies in conjunction with the AR system. These small, tangible robots can be used directly as handles for digital models. We share insights gathered during experimentation to help design platforms combining AR and actuated tangibles, and present several application scenarios to illustrate their potential for remote collaboration.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Africano, D., Berg, S., Lindbergh, K., Lundholm, P., Nilbrink, F., & Persson, A. (2004). Designing tangible interfaces for children’s collaboration. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’04 (pp. 853–868). ACM, New York, NY, USA. https://doi.org/10.1145/985921.985945.
Amano, K., & Yamamoto, A. (2012). Tangible interactions on a flat panel display using actuated paper sheets. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces, ITS ’12 (pp. 351–354). ACM, New York, NY, USA. https://doi.org/10.1145/2396636.2396698.
Antle, A. N. (2007). The cti framework: Informing the design of tangible systems for children. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 195–202). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1227010.
Billinghurst, M., Kato, H., & Poupyrev, I. (2001). The magicbook–Moving seamlessly between reality and virtuality. IEEE Computer Graphics and Applications, 21(3), 6–8. https://doi.org/10.1109/38.920621.
Brave, S., Ishii, H., & Dahley, A. (1998). Tangible interfaces for remote collaboration and communication. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work, CSCW ’98 (pp. 169–178). ACM, New York, NY, USA. https://doi.org/10.1145/289444.289491.
Brewer, J., Williams, A., & Dourish, P. (2007). A handle on what’s going on: Combining tangible interfaces and ambient displays for collaborative groups. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 3–10). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1226971.
Buxton, B. (2009). Mediaspace–meaningspace–meetingspace. In Media space 20+ years of mediated life (pp. 217–231). Springer.
Clark, H. H., Brennan, S. E., et al. (1991). Grounding in communication. Perspectives on Socially Shared Cognition, 13, 127–149.
Elvezio, C., Sukan, M., Oda, O., Feiner, S., & Tversky, B. (2017). Remote collaboration in ar and vr using virtual replicas. In ACM SIGGRAPH 2017 VR Village (p. 13). ACM.
Everitt, K. M., Klemmer, S. R., Lee, R., & Landay, J. A. (2003). Two worlds apart: Bridging the gap between physical and virtual media for distributed design collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 553–560). ACM
Furió, D., Fleck, S., Bousquet, B., Guillet, J. P., Canioni, L., & Hachet, M. (2017). Hobit: Hybrid optical bench for innovative teaching. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17 (pp. 949–959). ACM, New York, NY, USA. https://doi.org/10.1145/3025453.3025789.
He, Z., Zhu, F., & Perlin, K. (2017) Physhare: Sharing physical interaction in virtual reality. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology (pp. 17–19). ACM
Henderson, S., & Feiner, S. (2011). Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics, 17(10), 1355–1368.
Holmquist, L. E., Falk, J., & Wigström, J. (1999). Supporting group collaboration with interpersonal awareness devices. Personal Technologies, 3(1), 13–21. https://doi.org/10.1007/BF01305316.
Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’06 (pp. 437–446). ACM, New York, NY, USA. https://doi.org/10.1145/1124772.1124838.
Ishii, H., & Kobayashi, M. (1992). Clearboard: A seamless medium for shared drawing and conversation with eye contact. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 525–532). ACM.
Ishii, H., Lakatos, D., Bonanni, L., & Labrune, J. B. (2012). Radical atoms: Beyond tangible bits, toward transformable materials. Interactions, 19(1), 38–51. https://doi.org/10.1145/2065327.2065337.
Ishii, H., & Miyake, N. (1991). Toward an open shared workspace: Computer and video fusion approach of teamworkstation. Communications of the ACM, 34(12), 37–50.
Jordà , S., Geiger, G., Alonso, M., & Kaltenbrunner, M. (2007). The reactable: Exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 139–146). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1226998.
Kasahara, S., Niiyama, R., Heun, V., & Ishii, H. (2013). Extouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, TEI ’13 (pp. 223–228). ACM, New York, NY, USA. https://doi.org/10.1145/2460625.2460661.
Kato, H., Billinghurst, M., Poupyrev, I., Tetsutani, N., & Tachibana, K. (2001). Tangible augmented reality for human computer interaction. In Proceedings of Nicograph 2001.
Kojima, M., Sugimoto, M., Nakamura, A., Tomita, M., Nii, H., & Inami, M. (2006). Augmented coliseum: An augmented game environment with small vehicles. In First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP ’06) (pp. 6 pp). https://doi.org/10.1109/TABLETOP.2006.3.
Le Goc, M., Kim, L. H., Parsaei, A., Fekete, J. D., Dragicevic, P., & Follmer, S. (2016). Zooids: Building blocks for swarm user interfaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST ’16 (pp. 97–109). ACM, New York, NY, USA (2016). https://doi.org/10.1145/2984511.2984547.
Leap Motion Inc. (2018). Leap motion. Online. https://www.leapmotion.com/.
Lee, G. A., Nelles, C., Billinghurst, M., & Kim, G. J. (2004). Immersive authoring of tangible augmented reality applications. In Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality (pp. 172–181). IEEE Computer Society.
Lee, M., Norouzi, N., Bruder, G., Wisniewski, P. J., & Welch, G. F. (2018). The physical-virtual table: exploring the effects of a virtual human’s physical influence on social interaction. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (p. 25). ACM.
Leithinger, D., Follmer, S., Olwal, A., & Ishii, H. (2014). Physical telepresence: Shape capture and display for embodied, computer-mediated remote collaboration. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14 (pp. 461–470). ACM, New York, NY, USA. https://doi.org/10.1145/2642918.2647377.
Mi, H., & Sugimoto, M. (2011). Hats: Interact using height-adjustable tangibles in tabletop interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS ’11 (pp. 71–74). ACM, New York, NY, USA. https://doi.org/10.1145/2076354.2076368.
Microsoft Corporation. (2019). Microsoft hololens | the leader in mixed reality technology. Online. https://www.microsoft.com/en-us/hololens.
Mueller, F. F., Cole, L., O’Brien, S., & Walmink, W. (2006). Airhockey over a distance: A networked physical game to support social interactions. In Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, ACE ’06. ACM, New York, NY, USA. https://doi.org/10.1145/1178823.1178906.
Nowacka, D., Ladha, K., Hammerla, N. Y., Jackson, D., Ladha, C., Rukzio, E., & Olivier, P. (2013). Touchbugs: Actuated tangibles on multi-touch tables. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13 (pp. 759–762). ACM, New York, NY, USA. https://doi.org/10.1145/2470654.2470761.
Oda, O., Elvezio, C., Sukan, M., Feiner, S., & Tversky, B. (2015). Virtual replicas for remote assistance in virtual and augmented reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (pp. 405–415). ACM.
Özgür, A., Johal, W., Mondada, F., & Dillenbourg, P. (2017). Windfield: Demonstrating wind meteorology with handheld haptic robots. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (pp. 48–49). ACM.
Pangaro, G., Maynes-Aminzade, D., & Ishii, H. (2002). The actuated workbench: Computer-controlled actuation in tabletop tangible interfaces. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology, UIST ’02 (pp. 181–190). ACM, New York, NY, USA. https://doi.org/10.1145/571985.572011.
Patten, J., & Ishii, H. (2007). Mechanical constraints as computational constraints in tabletop tangible interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’07 (pp. 809–818). ACM, New York, NY, USA. https://doi.org/10.1145/1240624.1240746.
Patten Studio. (2014). Thumbles: Expanding the vocabulary of design. Online. http://www.pattenstudio.com/works/thumbles/.
Pedersen, E. W., & Hornbæk, K. (2011). Tangible bots: Interaction with active tangibles in tabletop interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11 (pp. 2975–2984). ACM, New York, NY, USA. https://doi.org/10.1145/1978942.1979384.
Poupyrev, I., Nashida, T., & Okabe, M. (2007). Actuation and tangible user interfaces: The vaucanson duck, robots, and shape displays. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 205–212). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1227012.
Richter, J., Thomas, B. H., Sugimoto, M., & Inami, M. (2007). Remote active tangible interactions. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 39–42). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1226977.
Riedenklau, E., Hermann, T., & Ritter, H. (2012). An integrated multi-modal actuated tangible user interface for distributed collaborative planning. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, TEI ’12 (pp. 169–174). ACM, New York, NY, USA. https://doi.org/10.1145/2148131.2148167.
Roo, J. S., Gervais, R., Frey, J., & Hachet, M. (2017). Inner garden: Connecting inner states to a mixed reality sandbox for mindfulness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17 (pp. 1459–1470). ACM, New York, NY, USA. https://doi.org/10.1145/3025453.3025743.
Roo, J. S., & Hachet, M. (2017). One reality: Augmenting how the physical world is experienced by combining multiple mixed reality modalities. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, UIST ’17 (pp. 787–795). ACM, New York, NY, USA. https://doi.org/10.1145/3126594.3126638.
Rosenfeld, D., Zawadzki, M., Sudol, J., & Perlin, K. (2004). Physical objects as bidirectional user interface elements. IEEE Computer Graphics and Applications, 24(1), 44–49. https://doi.org/10.1109/MCG.2004.1255808.
Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. (2016). Using mobile eye-trackers to unpack the perceptual benefits of a tangible user interface for collaborative learning. ACM Transactions on Computer-Human Interaction (TOCHI), 23(6), 39.
Tang, J. C., & Minneman, S. (1991). Videowhiteboard: Video shadows to support remote collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 315–322). ACM.
Underkoffler, J., & Ishii, H. (1999). Urp: A luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’99 (pp. 386–393). ACM, New York, NY, USA. https://doi.org/10.1145/302979.303114.
Weiss, M., Schwarz, F., Jakubowski, S., & Borchers, J. (2010). Madgets: Actuating widgets on interactive tabletops. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology, UIST ’10 (pp. 293–302). ACM, New York, NY, USA. https://doi.org/10.1145/1866029.1866075.
Wilson, A. D., & Robbins, D. C. (2007). Playtogether: Playing games across multiple interactive tabletops. In IUI Workshop on Tangible Play: Research and Design for Tangible and Tabletop Games.
Xie, L., Antle, A. N., & Motamedi, N. (2008). Are tangibles more fun? Comparing children’s enjoyment and engagement using physical, graphical and tangible user interfaces. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (pp. 191–198). ACM.
Yarosh, S., Tang, A., Mokashi, S., & Abowd, G. D. (2013). Almost touching: parent-child remote communication using the sharetable system. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (pp. 181–192). ACM.
Zhao, Y., Kim, L.H., Wang, Y., Le Goc, M., & Follmer, S. (2017). Robotic assembly of haptic proxy objects for tangible interaction and virtual reality. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (pp. 82–91). ACM.
Zhou, Z., Cheok, A. D., Chan, T., Pan, J. H., & Li, Y. (2004). Interactive entertainment systems using tangible cubes. In IE2004 Proceedings of the First Australian Workshop on Interactive Entertainment (p. 19).
Acknowledgements
This work is supported in part by the Hasso Plattner Design Thinking Research Program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Le Goc, M., Zhao, A., Wang, Y., Dietz, G., Semmens, R., Follmer, S. (2020). Investigating Active Tangibles and Augmented Reality for Creativity Support in Remote Collaboration. In: Meinel, C., Leifer, L. (eds) Design Thinking Research . Understanding Innovation. Springer, Cham. https://doi.org/10.1007/978-3-030-28960-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-28960-7_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-28959-1
Online ISBN: 978-3-030-28960-7
eBook Packages: Business and ManagementBusiness and Management (R0)