Skip to main content
SpringerLink
Log in

Visualizing Eye Movements in Formal Cognitive Models

  • Conference paper
  • First Online:
Eye Tracking and Visualization (ETVIS 2015)

Part of the book series: Mathematics and Visualization ((MATHVISUAL))

Included in the following conference series:

  • 2031 Accesses

Abstract

We present two visualization approaches illustrating the value of formal cognitive models for predicting, capturing, and understanding eye tracking as a manifestation of underlying cognitive processes and strategies. Computational cognitive models are formal theories of cognition which can provide predictions for human eye movements in visual decision-making tasks. Visualizing the internal dynamics of a model provides insights into how the interplay of cognitive mechanisms influences the observable eye movements. Animation of those model behaviors in virtual human agents gives explicit, high fidelity visualizations of model behavior, providing the analyst with an understanding of the simulated human’s behavior. Both can be compared to human data for insight about cognitive mechanisms engaged in visual tasks and how eye movements are affected by changes in internal cognitive strategies, external interface properties, and task demands. We illustrate the visualizations on two models of visual multitasking and juxtapose model performance against a human operator performing the same task.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The four tasks used herein constitute the standard implementation of the mMAT-B for multitasking research [6]. The number of tasks can be flexibly increased or decreased and the nature of the tasks can be changed to accommodate the research questions of interest. For more on the mMAT-B software, see http://sai.mindmodeling.org/mmatb.

References

  1. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111 (4), 1036–1060 (2004)

    Article  Google Scholar 

  2. Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., Ertl. T.: State-of-the-art of visualization for eye tracking data. In: Proceedings of EuroVis, Swansea, vol. 2014 (2014)

    Google Scholar 

  3. Bojko, A.A.: Informative or misleading? Heatmaps deconstructed. In: Jacko, J.A. (eds.) Human-Computer Interaction. New Trends, pp. 30–39. Springer, Berlin/Heidelberg (2009)

    Chapter  Google Scholar 

  4. Bostock, M., Ogievetsky, V., Heer, J.: D3: Data driven documents. IEEE Trans. Vis. Comput. Gr. 17 (12), 2301–2309 (2011)

    Article  Google Scholar 

  5. Busemeyer, J.R., Diederich, A.: Cognitive Modeling. Sage, Los Angeles (2010)

    Google Scholar 

  6. Cline, J., Arendt, D.L., Geiselman, E.E., Blaha, L.M.: Web-based implementation of the modified multi-attribute task battery. In: 4th Annual Midwestern Cognitive Science Conference, Dayton (2014)

    Google Scholar 

  7. Courgeon, M., Rautureau, G., Martin, J.-C., Grynszpan, O.: Joint attention simulation using eye-tracking and virtual humans. IEEE Trans. Affect. Comput. 5 (3), 238–250 (2014)

    Article  Google Scholar 

  8. Dam, E.B., Koch, M., Lillholm, M.: Quaternions, interpolation and animation. Technical report DIKU-TR-98/5, University of Copenhagen, Universitetsparken 1, DK-2100 Kbh (1998)

    Google Scholar 

  9. Halverson, T., Hornof, A.J.: A computational model of “Active Vision” for visual search in human-computer interaction. Hum. Comput. Interact. 26 (4), 285–314 (2011)

    Article  Google Scholar 

  10. Halverson, T., Reynolds, B., Blaha, L.M.: SIMCog-JS: simplified interfacing for modeling cognition – JavaScript. In: Proceedings of the International Conference on Cognitive Modeling, Groningen, pp. 39–44 (2015)

    Google Scholar 

  11. Itti, L., Dhavale, N., Pighin, F.: Photorealistic attention-based gaze animation. In: 2006 IEEE International Conference on Multimedia and Expo, Toronto, pp. 521–524 (2006)

    Google Scholar 

  12. Kieras, D.E., Meyer, D.E.: An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Hum. Comput. Interact. 12 (4), 391–438 (1997)

    Article  Google Scholar 

  13. Laird, J.E.: The Soar Cognitive Architecture. MIT Press, Cambridge (2012)

    Google Scholar 

  14. Nyamsuren, E., Taatgen, N.A.: Pre-attentive and Attentive Vision Module. In: Proceedings of the 2012 International Conference on Cognitive Modeling, Berlin, pp. 211–216 (2012)

    Google Scholar 

  15. Patterson, R.E., Blaha, L.M., Grinstein, G.G., Liggett, K.K., Kaveney, D.E., Sheldon, K.C., Havig, P.R., Moore, J.A.: A human cognition framework for information visualization. Comput. Gr. 42, 42–58 (2014)

    Article  Google Scholar 

  16. Peebles, D.: A cognitive architecture-based model of graph comprehension. In: Rußwinkel, N., Drewitz, U., van Rijn, H. (eds.) 11th International Conference on Cognitive Modeling, Berlin, pp. 37–42 (2012)

    Google Scholar 

  17. Peebles, D.: Strategy and pattern recognition in expert comprehension of 2 × 2 interaction graphs. Cognit. Syst. Res. 24, 43–51 (2013)

    Article  Google Scholar 

  18. Peebles, D., Cheng, P.C.-H.: Modeling the effect of task and graphical representation on response latency in a graph reading task. Human Factors: J. Hum. Fact. Ergon. Soc. 45 (1), 28–46 (2003)

    Article  Google Scholar 

  19. Raschke, M., Blascheck, T., Richter, M., Agapkin, T., Ertl, T.: Visual analysis of perceptual and cognitive processes. In: 2014 International Conference on Information Visualization Theory and Applications (IVAPP), Lisbon, pp. 284–291 (2014)

    Google Scholar 

  20. Raschke, M., Engelhardt, S., Ertl, T.: A framework for simulating visual search strategies. In: Proceedings of the 11th International Conference on Cognitive Modeling, Ottawa, pp. 221–226 (2013)

    Google Scholar 

  21. Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29 (6), 1045–1060 (2005)

    Article  Google Scholar 

  22. Salvucci, D.D.: An integrated model of eye movements and visual encoding. Cogn. Syst. Res. 1 (4), 201–220 (2001)

    Article  Google Scholar 

  23. Salvucci, D.D., Taatgen, N.A.: Threaded cognition: an integrated theory of concurrent multitasking. Psychol. Rev. 115 (1), 101–130 (2008)

    Article  Google Scholar 

  24. Schoelles, M., Gray, W.D.: Speculations on model tracing for visual analytics. In: Proceedings of the 12th International Conference on Cognitive Modeling, Ottawa, pp. 406–407 (2013)

    Google Scholar 

  25. Teo, L.-H., John, B.E., Blackmon, M.H.: CogTool-Explorer: a model of goal-directed user exploration that considers information layout. In: Proceedings of the Conference on Human Factors in Computing Systems, Texas, pp. 2479–2488 (2012)

    Google Scholar 

  26. Wang, L.-C., Chen, C.: A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Trans. Robot. Autom. 7 (4), 489–499 (1991)

    Article  Google Scholar 

  27. Wolfe, J.M.: Guided Search 4.0. In: Gray, W.D. (ed.) Integrated Models of Cognitive Systems, pp. 99–119. Oxford University Press, Oxford/New York(2007)

    Chapter  Google Scholar 

  28. Zhang, Y., Hornof, A.J.: Understanding multitasking through parallelized strategy exploration and individualized cognitive modeling. In: Conference on Human Factors in Computing Systems, New York, pp. 3885–3894 (2014)

    Google Scholar 

Download references

Acknowledgements

The authors thank Dr. Megan Morris and Mr. Jacob Kern for their assistance in processing the Tobii Glasses data, and Dr. Dustin Arendt for recurrence plot inspiration. The views expressed in this paper are those of the authors and do not reflect the official policy or position of the Department of Defense or the U.S. Government. This work was supported by AFOSR LRIR to L.M.B. Distribution A: Approved for public release; distribution unlimited. 88ABW Cleared 08/26/2015; 88ABW-2015-4021.

Author information

Authors and Affiliations

  1. George Mason University, Fairfax, Virginia, USA

    J. Timothy Balint

  2. Wright State University, Dayton, Ohio, USA

    Brad Reynolds

  3. Air Force Research Laboratory, Wright-Patterson AFB, Dayton, Ohio, USA

    Leslie M. Blaha

  4. Oregon Research in Cognitive Applications, LLC, Springfield, Oregon, USA

    Tim Halverson

Authors
  1. J. Timothy Balint
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brad Reynolds
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leslie M. Blaha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tim Halverson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Timothy Balint .

Editor information

Editors and Affiliations

  1. VISUS, University of Stuttgart, Stuttgart, Germany

    Michael Burch

  2. Max Planck Institute for Biological Cybernetics, TĂĽbingen, Germany

    Lewis Chuang

  3. SIAT, Simon Fraser University, Surrey, British Columbia, Canada

    Brian Fisher

  4. VIS, University of Stuttgart, Stuttgart, Germany

    Albrecht Schmidt

  5. VISUS, University of Stuttgart, Stuttgart, Germany

    Daniel Weiskopf

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Balint, J.T., Reynolds, B., Blaha, L.M., Halverson, T. (2017). Visualizing Eye Movements in Formal Cognitive Models. In: Burch, M., Chuang, L., Fisher, B., Schmidt, A., Weiskopf, D. (eds) Eye Tracking and Visualization. ETVIS 2015. Mathematics and Visualization. Springer, Cham. https://doi.org/10.1007/978-3-319-47024-5_6

Download citation

Publish with us

Policies and ethics

Search

Navigation