Abstract
Emerging ubiquitous environments raise the need to support multiple interaction modalities in diverse types of devices. Designing multimodal interfaces for ubiquitous environments using development tools creates challenges since target platforms support different resources and interfaces. Model-based approaches have been recognized as useful for managing the increasing complexity consequent to the many available interaction platforms. However, they have usually focused on graphical and/or vocal modalities. This paper presents a solution for enabling the development of tilt-based hand gesture and graphical modalities for mobile devices in a multimodal user interface development tool. The challenges related to developing gesture-based applications for various types of devices involving mobile devices are discussed in detail. The possible solution presented is based on a logical description language for hand-gesture user interfaces. Such language allows us to obtain a user interface implementation on the target mobile platform. The solution is illustrated with an example application that can be accessed from both the desktop and mobile device supporting tilt-based gesture interaction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abrams, M., et al.: UIML: An Appliance-Independent XML User Interface Language. In: Proceedings of the 8th WWW conference (1994)
Baldonado, M., et al.: The Stanford Digital Library Metadata Architecture. Int. J. Digit. Libr. 1, 108–121 (1997)
Bastide, R., et al.: A Model-Based Approach for Real-Time Embedded Multimodal Systems in Military Aircrafts. In: Proceedings ICMI 2004, pp. 243–250. ACM Press, New York (2004)
Berti, S., et al.: The TERESA XML Language for the Description of Interactive Systems at Multiple Abstraction Levels. In: Workshop Organized at Advanced Visual Interfaces, pp. 103–110 (2004)
Bouchet, J., Nigay, L., Ganille, T.: ICARE software components for rapidly developing multimodal interfaces. In: Proceedings ICMI 2004, pp. 251–258. ACM Press, New York (2004)
Calvary, G., et al.: A Unifying Reference Framework for Multi-Target User Interfaces. Interacting with Computers 15(3), 289–308 (2003)
Coutaz, J., et al.: Four Easy Pieces for Assessing the Usability of Multmodal Interaction: the CARE properties. In: Proceedings INTERACT 1995, pp. 115–120 (1995)
Eslambolchilar, P., Murray-Smith, R.: Tilt-based Automatic Zooming and Scaling in Mo-bile Devices - a state-space implementation. In: Brewster, S.A., Dunlop, M.D. (eds.) Mobile HCI 2004. LNCS, vol. 3160, pp. 120–131. Springer, Heidelberg (2004)
Hinckley, K., et al.: Sensing Techniques for Mobile Interaction. CHI Letters (ACM UIST 2000 Symposium on User Interface Software & Technology) 2(2), 91–100 (2000)
Kela, J., et al.: Accelerometer-based gesture control for a design environment. Personal and Ubiquitous Computing 10(5), 285–299 (2005)
Kim, S.-H., et al.: An interaction and product design of gesture based TV remote control. In: CHI ’04 extended abstracts, Vienna, Austria, pp. 1548–1548 (2004)
Linjama, J., Kaaresoja, T.: Novel, minimalist haptic gesture interaction for mobile devices. In: Proc. Nordic-CHI, 2004, Tampere, Finland, pp. 457–458 (2004)
Long, A.C., Landay, J.A., Rowe, L.: Implications for a gesture design tool. In: Human Factors in Computing Systems (SIGCHI Proc.), pp. 40–47. ACM Press, New York (1999)
Mäntyjärvi, J., et al.: Gesture interaction for small handheld devices to support multimedia applications. Journal of Mobile Multimedia 1(2), 92–112 (2005)
Mori, G., Paternò, F., Santoro, C.: Design and Development of Multi-Device User Interfaces through Multiple Logical Descriptions. IEEE Transactions on Software Engineering 30(8), 507–520 (2004)
Obrenovic, Z., Starcevic, D., Selic, B.: A Model-Driven Approach to Content Repurposing. IEEE Mutimedia, 62–71 (Januray/March 2004)
Paternò, F.: Model-Based Design and Evaluation of Interactive Application. Springer, Heidelberg (1999)
Paternò, F., Giammarino, F.: Authoring Interfaces with Combined Use of Graphics and Voice for both Stationary and Mobile Devices. In: Proceedings AVI’06, May 2006, ACM Press, New York (2006)
Rekimoto, J.: Tilting operations for small screen interfaces. In: ACM UIST 1996, pp. 167–168 (1996)
Stanciulescu, A., et al.: A Transformational Approach for Multimodal Web User Interfaces based on USIXML. In: Proceedings ICMI 2005, pp. 259–266. ACM Press, New York (2005)
Szekely, P.: Retrospective and Challenges for Model-Based Interface Development. In: 2nd International Workshop on Computer-Aided Design of User Interfaces, Namur University Press, Namur (1996)
Wilson, A., Shafer, S.: Between u and i: XWand: UI for intelligent spaces. In: Proceedings of the conference on Human factors in computing systems, CHI 2003, April 2003, pp. 545–552 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mäntyjärvi, J., Paternò, F., Santoro, C. (2007). Incorporating Tilt-Based Interaction in Multimodal User Interfaces for Mobile Devices. In: Coninx, K., Luyten, K., Schneider, K.A. (eds) Task Models and Diagrams for Users Interface Design. TAMODIA 2006. Lecture Notes in Computer Science, vol 4385. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70816-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-540-70816-2_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70815-5
Online ISBN: 978-3-540-70816-2
eBook Packages: Computer ScienceComputer Science (R0)