Abstract
Multimodal interfaces are becoming more common, even in the field of safety critical interactive software, mainly due to the naturalness of the interaction that increases the bandwidth between the user and the system they are interacting with. However, the specificities of multimodal interactive systems make it difficult to gather information from the use of modalities and to extract from this information recommendations for improving the multimodal user interfaces. This chapter aims at presenting how abstract information described in models can be fruitfully exploited to improve the quality of evaluations of multimodal interfaces. The approach presented in this chapter combines model-based verification (based on simulation scenario extraction generated from models) and empirical methods for usability evaluation. Our aim is to try to bring together two separated (and often opposite) issues, such as usability and reliability, into the development of safety critical systems. This approach is illustrated via a Space Ground System of a satellite control room, whose multimodal interaction technique is fully described by the means of formal models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bach, C., & Scapin, D. (2003). Ergonomic criteria adapted to human virtual environment interaction. In: Rauterberg, M., Menozzi, M. & Wesson, J. (eds.) IFIP conference on human-computer interaction (INTERACT’2003) (pp. 880–883), Zurich, Switzerland. IOS Press.
Baille, L., & Schatz, R. (2005). Exploring Multimodality in the Laboratory and the Field. In Lazzari, G. & Pianesi, P. (Eds.), ACM international conference on multimodal interfaces (ICMI’2005) (pp. 100–107). New York: ACM Press.
Balbo, S., Coutaz, J., & Salber, D. (2003) Towards automatic evaluation of multimodal user interfaces. Intelligent User Interfaces, Knowledge-Based Systems, 6(4), pp. 267–274.
Bastide, R., Navarre, D., Palanque, P., Schyn, A., & Dragicevic, P. (2004). Model-based approach for real-time embedded multimodal systems in military aircrafts. In ACM International Conference on Multimodal Interfaces (ICMI’2004). New York: ACM Press.
Bolt, R. A. (1980). Put-that-there: Voice and gesture at the graphics interface. In Proceedings of the 7th International Conference on Computer Graphics and Interactive Techniques (pp. 262–270), Seattle, USA.
Bolt, R. E., & Herranz, E. (1992). Two-handed gesture in multi-modal natural dialog. In J. Mackinlay & M. Green (Eds.), Symposium on User Interface Software and Technology (UIST’92) (pp. 7–14). New York: ACM Press.
Bowman, D., Gabbard, J., & Hix, D. (2002). A survey of usability evaluation in virtual environments: Classification and comparison of methods. Presence: Teleoperators and Virtual Environments, 11(4), 404–424.
Brewster, S. A., Wright, P. C., & Edwards, A. D. N. (1994). The design and evaluation of an auditory-enhanced scrollbar. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI ’94) (pp. 173–179). New York: ACM Press.
Buxton, W., & Myers, B.A. (1986) A study in two-handed input. In: Mantei M., & Orbeton, P. (Eds.), ACM Conference on Human Factors in Computing Systems (CHI’86) (pp. 321–326), Boston, Massachusetts. ACM Press.
Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., & Young, R. (1995). Four easy pieces for assessing the usability of multimodal in interaction: the CARE Properties. In S. A. Arnesen & S. Gilmore (Eds.), The IFIP Conference on Human-Computer Interaction (INTERACT’95) (pp. 115–120). Lillehammer, Norway: Chapman & Hall.
Dillon, R.F., Edey, J.D., & Tombaugh, J.W. (1990). Measuring the true cost of command selection: techniques and results. In J.C. Chew & J. Whiteside (Eds.), ACM Conference on Human Factors in Computing Systems (CHI’90) (pp 19–25), Seattle, Washington. ACM Press.
Dragicevic P., & Fekete J-D. (2001). Input device selection and interaction configuration with ICON. In A. Blandford, J. Vanderdonckt & P. Gray (Eds.), People and Computer XV – Interaction without Frontiers: Joint Proceedings of HCI 2001 and IHM 2001 (pp. 448–543). London: Springer.
Dragicevic, P., Navarre, D., Palanque, P., Schyn, A., & Bastide, R. (2004). Very-high-fidelity prototyping for both presentation and dialogue parts of multimodal interactive systems. In: DSVIS/EHCI 2004 Joint Conference 11th Workshop on Design Specification and Verification of Interactive Systems and Engineering for HCI, Tremsbüttel Castle, Hamburg, Germany, Lecture Notes in Computer Science n^ˆ 3425 (pp. 179–199).
Dybkjær, L., Bernsen, N. O., & Minker, W. (2004). New challenges in usability evaluation – Beyond task-oriented spoken dialogue systems. In Proceedings of ICSLP, (vol. III)(pp. 2261–2264).
Jöst, M., Haubler, J., Merdes, M., & Malaka, R. (2005). Multimodal interaction for pedestrians: an evaluation study. In J. Riedl, A. Jameson, D. Billsus & T. Lau (Eds.), ACM International Conference on Intelligent User Interfaces (IUI’2005) (pp. 59–66), San Diego. ACM Press.
Hinckley, K., Pausch, R., Proffitt, D., & Kassel, N. F. (1998). Two-handed virtual manipulation. ACM Transactions on Computer-Human Interaction, 5(3), 260–302.
Holzapfel, H., Nickler, K., & Stiefelhagen, R. (2004) Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3D pointing gesture. In R. Sharma & T. Darrell, T. (Eds.), ACM International Conference on Multimodal Interfaces (ICMI’2004) (pp. 175–182). New York: ACM Press.
Kaster, T., Pfeiffer, M., & Bauckhage, C. (2003). Combining speech and haptics for intuitive and efficient navigation through image database. In S. Oviatt (Ed.), ACM International Conference on Multimodal interfaces (ICMI’2003) (pp. 180–187). New York: ACM Press.
Kabbash, P., Buxton, W., & Sellen, A. (1994). Two-handed input in a compound task. In C. Plaisant (Ed.), ACM Conference on Human Factors in Computing System (CHI’94) (pp. 417–423), Boston, Massachusetts. ACM Press.
Kjeldskov, J., & Stage, J. (2004). New techniques for usability evaluation of mobile systems. International Journal on Human-Computer Studies, 60(5), 599–220.
Lewis, C., Polson, P., & Wharton, R. (1990). Testing a walkthrough methodology for theory-based design of walk-up-and-us interfaces. In J. C. Chew & J. Whiteside (Eds.), ACM Conference on Human Factors in Computing Systems (CHI’90) (pp. 235–241), Seattle, Washington. ACM Press.
Memon A., Pollack M., & Soffa M-L. (2000). Automated test oracles for GUIs. ACM SIGSOFT Software Engineering Notes: Proceedings of the 8th ACM SIGSOFT International Symposium on Foundations of Software Engineering: twenty-first century applications (SIGSOFT ’00/FSE-8), Volume 25, Issue 6 (pp. 30–39). ACM Press.
Navarre D., Palanque P., Dragicevic P., & Bastide R. (2006). An approach integrating two complementary model-based environments for the construction of multimodal interactive applications. Interacting with Computers, 18(5), 910–941.
Navarre, D., Palanque, P., Bastide, R., Schyn, A., Winckler, M., Nedel, L., & Freitas, C.M.D.S. (2005). A formal description of multimodal interaction techniques for immersive virtual reality applications. In M. F. Costabile & F. Paterno (Eds.), IFIP Conference on Human-Computer Interaction (INTERACT’2005)(pp. 170–183). Berlin: Springer-Verlag.
Nedel, L., Freitas, C.M.D.S., Jacob, L., & Pimenta, M. (2003). Testing the use of egocentric interactive techniques in immersive virtual environments. In: M. Rauterberg, M. Menozzi & J. Wesson (Eds.), IFIP Conference on Human-Computer Interaction (INTERACT’2003) (pp. 471–478), Zurich, Switzerland. IOS Press.
Nielsen, J., & Mack, R. (1994). Usability inspection methods. New York: Wiley.
Ould, M., Bastide, R., Navarre, D., Palanque, P., Rubio, F., & Schyn, A. (2004). Multimodal and 3D graphic man-machine interfaces to improve operations. In Proceedings of Eighth International Conference on Space Operations (pp. 435–450), Montréal, Canada.
Oviatt, S. (1999). Ten myths of multimodal interaction. Communications of the ACM, Vol. 42( 11), 74 –81.
Palanque, P., Bastide, R., & Paterno, F. (1997). Formal specification as a tool for objective assessment of safety-critical interactive systems. In Proceedings of IFIP TC 13 INTERACT’97 conference (pp. 323–330), Sydney, Australia, 14–18 July 1997. Chapman & Hall.
Paternò, F., & Santos, I. (2006). Designing and developing multi-user, multi-device web interfaces. In G. Calvary & J. Vanderdonckt (Eds.), Conference on Computer-Aided Design of User Interfaces (CADUI2006). Dordrecht, the Netherlands: Kluwer Academics Publishers.
Poupyrev, I, Weghorst, S., Billinghurst, M., & Ichikawa, T. (1998). Egocentric object manipulation in virtual environments: empirical evaluation of interaction techniques. In N. Ferreira & M. Göbel (Eds.), Proceedings of Computer Graphics Forum (EUROGRAPHICS’98) (pp. 41–52). Malden, MA: Blackwell Publishers.
Polson, P. G., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, 36(5), 741–773.
Reason J. (1990). Human Error. Cambridge University Press.
Suhm, B., Myers, B., & Waibel, A. (1999). Model-based and empirical evaluation of multimodal interactive error correction. In M.E. Atwood (Eds.), ACM Conference on Human Factors in Computing Systems (CHI’99) (pp. 584–591). New York: ACM Press.
Trevisan, D., Vanderdonckt, J., Macq, B., & Raftopoulous, C. (2003). Modeling interaction for image-guided procedures. In Proceedings of International Conference on Medical Imaging SPIE’2003(pp. 108–118). San Diego: International Society for Optical Engineering (v. 5029).
Trevisan, D. G., Nedel, L. P., Macq, B., & Vanderdonckt, J. (2006). Detecting interaction variables in a mixed reality system for maxillofacial-guided surgery. In: SBC Symposium on Virtual Reality (SVR’2006) (pp. 39–50). Belém do Pará, Brazil: SBC Press.
Zhai, S., Barton, A.S., & Selker, T. (1997). Improving browsing performance: a study of four input devices for scrolling and pointing tasks. In S. Howard, J. Hammond & G. Lindgaard (Eds.), The IFIP Conference on Human-Computer Interaction (INTERACT’97)(pp. 286–292). Sydney: Chapman & Hall.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag London Limited
About this chapter
Cite this chapter
Bernhaupt, R., Navarre, D., Palanque, P., Winckler, M. (2008). Model-Based Evaluation: A New Way to Support Usability Evaluation of Multimodal Interactive Applications. In: Law, E.LC., Hvannberg, E.T., Cockton, G. (eds) Maturing Usability. Human-Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84628-941-5_5
Download citation
DOI: https://doi.org/10.1007/978-1-84628-941-5_5
Publisher Name: Springer, London
Print ISBN: 978-1-84628-940-8
Online ISBN: 978-1-84628-941-5
eBook Packages: Computer ScienceComputer Science (R0)