Skip to main content
SpringerLink
Log in

Predictive eye movements in natural vision

  • Research Article
  • Published:
Experimental Brain Research Aims and scope Submit manuscript

Abstract

In the natural world, the brain must handle inherent delays in visual processing. This is a problem particularly during dynamic tasks. A possible solution to visuo-motor delays is prediction of a future state of the environment based on the current state and properties of the environment learned from experience. Prediction is well known to occur in both saccades and pursuit movements and is likely to depend on some kind of internal visual model as the basis for this prediction. However, most evidence comes from controlled laboratory studies using simple paradigms. In this study, we examine eye movements made in the context of demanding natural behavior, while playing squash. We show that prediction is a pervasive component of gaze behavior in this context. We show in addition that these predictive movements are extraordinarily precise and operate continuously in time across multiple trajectories and multiple movements. This suggests that prediction is based on complex dynamic visual models of the way that balls move, accumulated over extensive experience. Since eye, head, arm, and body movements all co-occur, it seems likely that a common internal model of predicted visual state is shared by different effectors to allow flexible coordination patterns. It is generally agreed that internal models are responsible for predicting future sensory state for control of body movements. The present work suggests that model-based prediction is likely to be a pervasive component in natural gaze control as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Abernethy B (1990) Expertise, visual search, and information pick-up in squash. Perception 19:63–77

    Article  PubMed  CAS  Google Scholar 

  • Aivar P, Hayhoe M, Mruczek R (2005) Role of spatial memory in saccadic targeting in natural tasks. J Vis 5:177–193

    Article  PubMed  Google Scholar 

  • Assad JA, Maunsell JH (1995) Neuronal correlates of inferred motion in primate posterior parietal cortex. Nature 373(6514):518–521

    Article  PubMed  CAS  Google Scholar 

  • Babcock JS, Pelz JB (2004) Building a lightweight eyetracking headgear. In: ACM SIGCHI eye tracking research & applications symposium, pp 109–114

  • Bahill AT, LaRitz T (1984) Why can’t batters keep their eye on the ball? Am Sci 72:249–253

    Google Scholar 

  • Bahill AT, Baldwin DG, Venkateswaran J (2005) Predicting a baseball’s path. Am Sci 93:218–225

    Google Scholar 

  • Barborica A, Ferrera VP (2003) Estimating invisible target speed from neuronal activity in monkey frontal eye field. Nat Neurosci 6:66–74

    Article  PubMed  CAS  Google Scholar 

  • Barnes G, Collins C (2008) Evidence for a link between the extra-retinal component of random-onset pursuit and the anticipatory pursuit of predictable object motion. J Neurophysiol 100:1135–1146

    Article  PubMed  CAS  Google Scholar 

  • Battaglia PW, Schrater P (2007) Humans trade off viewing time and movement duration to improve visuomotor accuracy in a fast reaching task. J Neurosci 27(26):6984–6994

    Article  PubMed  CAS  Google Scholar 

  • Battaglia PW, Schrater P, Kersten D (2005) Auxiliary object knowledge influences visually-guided interception behavior. In: Proceedings of 2nd symposium APGV, pp 145–152

  • Becker W, Fuchs A (1985) Prediction in the oculomotor system: smooth pursuit during the transient disappearance of a visual target. Exp Brain Res 57:562–575

    Article  PubMed  CAS  Google Scholar 

  • Blakemore SJ, Frith CD, Wolpert DW (2001) The cerebellum is involved in predicting the sensory consequences of action. Neuroreport 12(9):1879–1884

    Article  PubMed  CAS  Google Scholar 

  • Brouwer A, Knill DC (2007) The role of memory in visually guided reaching. J Vis 7(5):6 (1–12)

    Google Scholar 

  • Chen-Harris H, Joiner WM, Ethier V, Zee DS, Shadmehr R (2008) Adaptive control of saccades via internal feedback. J Neurosci 28(11):2804–2813

    Google Scholar 

  • Churchland M, Chou I, Lisberger S (2003) Evidence for object permanence in the smooth pursuit movements of monkeys. J Neurophysiol 90:2205–2218

    Article  PubMed  Google Scholar 

  • Collins C, Barnes GR (1999) Independent control of head and gaze movements during head-free pursuit in humans. J Physiol 515.1:299–314

    Google Scholar 

  • Ferrera V, Barborica A (2010) Internally generated error signals in monkey frontal eye field during an inferred motion task. J Neurosci 30(35):11612–11623

    Article  PubMed  CAS  Google Scholar 

  • Findlay JM (1981) Spatial and temporal factors in the predictive generation of saccadic eye movements. Vision Res 21(3):347–354

    Google Scholar 

  • Flanagan J, Wing A (1997) The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads. J Neurosci 17:1519–1528

    PubMed  CAS  Google Scholar 

  • Fukushima K, Akao T, Shichinohe N, Nitta T, Kurkin S, Fukushima J (2008) Predictive signals in the pursuit area of the monkey frontal eye fields. Prog Brain Res 171:433–440

    Article  PubMed  CAS  Google Scholar 

  • Fukushima K, Kasahara S, Akao T, Kurkin S, Fukushima J, Peterson B (2009) Eye-pursuit and reafferent head movement signals carried by pursuit neurons in the caudal part of the frontal eye fields during head-free pursuit. Cereb Cortex 19:263–275

    Article  PubMed  Google Scholar 

  • Hayhoe M (2009) Visual memory in motor planning and action. In: Brockmole J (ed) Memory for the visual world. Psychology Press, UK, pp 117–139

    Google Scholar 

  • Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9(4):188–193

    Article  PubMed  Google Scholar 

  • Hayhoe M, Mennie N, Sullivan B, Gorgos K (2005) The role of internal models and prediction in catching balls. In: Proceedings of AAAI fall symposium series

  • Isotalo E, Lasker AG, Zee DS (2005) Cognitive influences on predictive saccadic tracking. Exp Brain Res 165:461–469

    Article  PubMed  CAS  Google Scholar 

  • Itti L, Koch C (2001) Computational modeling of visual attention. Nat Rev Neurosci 2:194–203

    Article  PubMed  CAS  Google Scholar 

  • Johansson RS (1996) Sensory control of dextrous manipulation in humans. In: Wing A, Haggard P, Flanagan J (eds) Hand and brain: the neurophysiology and psychology of hand movements. Academic Press, San Diego, pp 381–414

    Google Scholar 

  • Joiner W, Shelhamer M (2006) Pursuit and saccadic tracking exhibit a similar dependence on movement preparation time. Exp Brain Res 173:572–586

    Article  PubMed  Google Scholar 

  • Jovancevic-Misic J, Hayhoe M (2009) Adaptive gaze control in natural environments. J Neurosci 29(19):6234–6238

    Article  PubMed  CAS  Google Scholar 

  • Kawato M (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9(6):718–727

    Google Scholar 

  • Koerding K, Wolpert D (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247

    Article  CAS  Google Scholar 

  • Koerding K, Wolpert D (2006) Bayesian decision theory in sensorimotor control. Trends Cogn Sci 10:319–326

    Article  Google Scholar 

  • Kowler E, Martins A, Pavel M (1984) The effect of expectations on slow oculomotor control- IV Anticipatory smooth eye movements depend on prior target motions. Vis Res 24:197–210

    Article  PubMed  CAS  Google Scholar 

  • Krauzlis R (2005) The control of voluntary eye movements: new perspectives. Neuroscientist 11(2):124–137

    Article  PubMed  Google Scholar 

  • Land M (2004) Eye movements in daily life. In: Chalupa L, Werner J (eds) The visual neurosciences, vol 2. MIT Press, Cambridge, pp 1357–1368

    Google Scholar 

  • Land MF, Furneaux S (1997) The knowledge base of the oculomotor system. Philos Trans R Soc Lond B Biol Sci 352:1231–1239

    Article  PubMed  CAS  Google Scholar 

  • Land MF, McLeod P (2000) From eye movements to actions: how batsmen hit the ball. Nat Neurosci 3:1340–1345

    Article  PubMed  CAS  Google Scholar 

  • Lopez-Moliner J, Field DT, Wann JP (2007) Interceptive timing: prior knowledge matters. J Vis 7:1–8

    Google Scholar 

  • Madelain L, Krauzlis RJ (2003) Effects of learning on smooth pursuit during transient disappearance of a visual target. J Neurophysiol 90:972–982

    Article  PubMed  Google Scholar 

  • Mulliken G, Andersen R (2009) Forward modles and state estimation in parietal cortex. In: Gazzaniga MS (ed) The cognitive neurosciences IV. MIT Press, Cambridge, pp 599–611

    Google Scholar 

  • Nyffeler T, Rivaud-Pechoux S, Wattiez N, Gaymard N (2008) Involvement of the supplementary eye field in oculomotor predictive behavior. J Cog Neurosci 20(9):1583–1594

    Article  Google Scholar 

  • Pierrot-Deseilligny Ch, Müri A, Nyffeler B, Milea D (2005) The role of the human dorsolateral prefrontal cortex in ocular motor behavior. Ann NY Acad Sci 1039:239–251

    Article  PubMed  Google Scholar 

  • Saunders JA, Knill DC (2003) Humans use continuous visual feedback from the hand to control fast reaching movements. Exp Brain Res 152:341–352

    Article  PubMed  Google Scholar 

  • Schlicht E, Schrater P (2007) Reach-to-grasp trajectories adjust for uncertainty in the location of visual targets. Exp Brain Res 182:47–57

    Article  PubMed  Google Scholar 

  • Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14:3208–3224

    PubMed  CAS  Google Scholar 

  • Shelhamer M, Joiner WM (2003) Saccades exhibit abrupt transition between reactive and predictive, predictive saccade sequence have long-term correlations. J Neurophysiol 90:2763–2769

    Article  PubMed  Google Scholar 

  • Shichinohe N, Akao T, Kurkin S, Fukushima J, Kaneko C, Fukushima K (2009) Memory and decision making in the frontal cortex during visual motion processing for smooth pursuit eye movements. Neuron 62:717–732

    Article  PubMed  CAS  Google Scholar 

  • Simo LS, Krisky CM, Sweeney JA (2005) Functional neuroanatomy of anticipatory behavior: dissociation between sensory-driven and memory-driven systems. Cereb Cortex 15:1982–1991

    Article  PubMed  Google Scholar 

  • Sprague N, Ballard D, Robinson A (2007) Modeling embodied visual behaviors. ACM Trans Appl Percept 4(2):11

    Google Scholar 

  • Tabata H, Muira K, Kawano K (2008) Trial-by-trial updating of the gain in preparation for smooth pursuit eye movement based on past experience in humans. J Neurophysiol 99:747–758

    Article  PubMed  Google Scholar 

  • Tassinari H, Hudson TE, Landy MS (2006) Combining priors and noisy visual cues in a rapid pointing task. J Neurosci 26:10154–10163

    Article  PubMed  CAS  Google Scholar 

  • Tresilian JR, Houseman JH (2005) Systematic variation in performance of an interceptive action with changes in the temporal constraints. Q J Exp Psychol 58A:447–466

    Google Scholar 

  • Wolpert D (2007) Probabilistic models in human sensorimotor control. Hum Mov Sci 26:511–524

    Article  PubMed  Google Scholar 

  • Wolpert D, Miall C, Kawato M (1998) Internal models in the cerebellum. Trends Cogn Sci 2:338–347

    Article  PubMed  CAS  Google Scholar 

  • Xaio Q, Barborica A, Ferrera V (2007) Modulation of visual responses in macaque frontal eye field during covert tracking of invisible targets. Cereb Cortex 17(4):918–928

    Article  Google Scholar 

  • Zago M, Bosco G, Maffei V, Iosa M, Ivanenko Y, Lacquaniti F (2004) Internal models of target motion: expected dynamics overrides measured kinematics in timing manual interceptions. J Neurophysiol 91:1620–1634

    Article  PubMed  Google Scholar 

  • Zago M, McIntyre J, Patrice Senot P, Lacquaniti F (2008) Internal models and prediction of visual gravitational motion. Vis Res 48:1532–1538

    Article  PubMed  Google Scholar 

  • Zago M, McIntyre J, Patrice Senot P, Lacquaniti F (2009) Visuo-motor coordination and internal models for object interception. Exp Brain Res 192(4):571–604

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This work was supported by NIH Grant EY05729. The authors wish to acknowledge the assistance in data collection of Jason Droll and also players Eric Hernady and Mithun Mukherjee, as well as Martin Heath and members of the University of Rochester Squash Team.

Author information

Authors and Affiliations

  1. Center for Perceptual Systems, University of Texas at Austin, 1 University Station, #A8000, Austin, TX, 78712-0187, USA

    Mary M. Hayhoe & Travis McKinney

  2. School of Optometry, SUNY, New York, NY, USA

    Kelly Chajka

  3. Center for Imaging Science, Rochester Institute of Technology, Rochester, NY, USA

    Jeff B. Pelz

Authors
  1. Mary M. Hayhoe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Travis McKinney
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kelly Chajka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeff B. Pelz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mary M. Hayhoe.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (AVI 263 kb)

Supplementary material 2 (AVI 137 kb)

Supplementary material 3 (AVI 219 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hayhoe, M.M., McKinney, T., Chajka, K. et al. Predictive eye movements in natural vision. Exp Brain Res 217, 125–136 (2012). https://doi.org/10.1007/s00221-011-2979-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00221-011-2979-2

Keywords

Search

Navigation