Skip to main content
SpringerLink
Log in

Hearing in a “Moving” Visual World: Coordinate Transformations Along the Auditory Pathway

  • Chapter
  • First Online:
Multisensory Processes

Part of the book series: Springer Handbook of Auditory Research ((SHAR,volume 68))

Abstract

This chapter reviews the literature on how auditory signals are transformed into a coordinate system that facilitates interactions with the visual system. Sound location is deduced from cues that depend on the position of the sound with respect to the head, but visual location is deduced from the pattern of light illuminating the retina, yielding an eye-centered code. Connecting sights and sounds originating from the same position in the physical world requires the brain to incorporate information about the position of the eyes with respect to the head. Eye position has been found to interact with auditory signals at all levels of the auditory pathway that have been tested but usually yields a code that is in a hybrid reference frame: neither head nor eye centered. Computing a coordinate transformation, in principle, may be easy, which could suggest that the looseness of the computational constraints may permit hybrid coding. A review of the behavioral literature addressing the effects of eye gaze on auditory spatial perception and a discussion of its consistency with physiological observations concludes the chapter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Andersen, R. A., & Buneo, C. A. (2002). Intentional maps in posterior parietal cortex. Annual Review of Neuroscience, 25(1), 189–220.

    CAS  PubMed  Google Scholar 

  • Andersen, R. A., & Mountcastle, V. B. (1983). The influence of the angle of gaze upon the excitability of the light-sensitive neurons of the posterior parietal cortex. The Journal of Neuroscience, 3(3), 532–548.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Andersen, R. A., Essick, G. K., & Siegel, R. M. (1985). Encoding of spatial location by posterior parietal neurons. Science, 230(4724), 456–458.

    CAS  PubMed  Google Scholar 

  • Best, V., Ozmeral, E. J., & Shinn-Cunningham, B. G. (2007). Visually-guided attention enhances target identification in a complex auditory scene. Journal of the Association for Research in Otolaryngology, 8(2), 294–304.

    PubMed  PubMed Central  Google Scholar 

  • Bohlander, R. W. (1984). Eye position and visual attention influence perceived auditory direction. Perceptual and Motor Skills, 59(2), 483–510.

    CAS  PubMed  Google Scholar 

  • Caruso, V., Pages, D. S., Sommer, M., & Groh, J. M. (2017). Beyond the labeled line: Variation in visual reference frames from intraparietal cortex to frontal eye fields and the superior colliculus. Journal of Neurophysiology, 119(4), 1411–1421.

    PubMed  PubMed Central  Google Scholar 

  • Chalupa, L. M., & Rhoades, R. W. (1977). Responses of visual, somatosensory, and auditory neurones in the golden hamster’s superior colliculus. The Journal of Physiology, 270(3), 595–626.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Cui, Q. N., O’Neill, W. E., & Paige, G. D. (2010a). Advancing age alters the influence of eye position on sound localization. Experimental Brain Research, 206(4), 371–379.

    PubMed  PubMed Central  Google Scholar 

  • Cui, Q. N., Razavi, B., O’Neill, W. E., & Paige, G. D. (2010b). Perception of auditory, visual, and egocentric spatial alignment adapts differently to changes in eye position. Journal of Neurophysiology, 103(2), 1020–1035.

    PubMed  Google Scholar 

  • Deneve, S., Latham, P. E., & Pouget, A. (2001). Efficient computation and cue integration with noisy population codes. Nature Neuroscience, 4(8), 826–831.

    CAS  PubMed  Google Scholar 

  • Drager, U. C., & Hubel, D. H. (1975). Responses to visual stimulation and relationship between visual, auditory, and somatosensory inputs in mouse superior colliculus. Journal of Neurophysiology, 38(3), 690–713.

    CAS  PubMed  Google Scholar 

  • Fu, K.-M. G., Shah, A. S., O’Connell, M. N., McGinnis, T., Eckholdt, H., Lakatos, P., Smiley, J., & Schroeder, C. E. (2004). Timing and laminar profile of eye-position effects on auditory responses in primate auditory cortex. Journal of Neurophysiology, 92(6), 3522–3531.

    PubMed  Google Scholar 

  • Fuchs, A. F., & Luschei, E. S. (1970). Firing patterns of abducens neurons of alert monkeys in relationship to horizontal eye movement. Journal of Neurophysiology, 33(3), 382–392.

    CAS  PubMed  Google Scholar 

  • Getzmann, S. (2002). The effect of eye position and background noise on vertical sound localization. Hearing Research, 169(1–2), 130–139.

    PubMed  Google Scholar 

  • Groh, J. M., & Sparks, D. L. (1992). Two models for transforming auditory signals from head-centered to eye-centered coordinates. Biological Cybernetics, 67(4), 291–302.

    CAS  PubMed  Google Scholar 

  • Groh, J. M., & Sparks, D. L. (1996). Saccades to somatosensory targets. I. Behavioral characteristics. Journal of Neurophysiology, 75(1), 412–427.

    CAS  PubMed  Google Scholar 

  • Groh, J. M., Trause, A. S., Underhill, A. M., Clark, K. R., & Inati, S. (2001). Eye position influences auditory responses in primate inferior colliculus. Neuron, 29(2), 509–518.

    CAS  PubMed  Google Scholar 

  • Groh, J. M., Kelly, K. A., & Underhill, A. M. (2003). A monotonic code for sound azimuth in primate inferior colliculus. Journal of Cognitive Neuroscience, 15(8), 1217–1231.

    PubMed  Google Scholar 

  • Grothe, B., & Pecka, M. (2014). The natural history of sound localization in mammals—A story of neuronal inhibition. Frontiers in Neural Circuits, 8(116), 1–19.

    Google Scholar 

  • Guthrie, B. L., Porter, J. D., & Sparks, D. L. (1983). Corollary discharge provides accurate eye position information to the oculomotor system. Science, 221(4616), 1193–1195.

    CAS  PubMed  Google Scholar 

  • Hafter, E. R., & Maio, J. D. (1975). Difference thresholds for interaural delay. The Journal of the Acoustical Society of America, 57(1), 181–187.

    CAS  PubMed  Google Scholar 

  • Jay, M. F., & Sparks, D. L. (1984). Auditory receptive fields in primate superior colliculus shift with changes in eye position. Nature, 309(5966), 345–347.

    CAS  PubMed  Google Scholar 

  • Jay, M. F., & Sparks, D. L. (1987a). Sensorimotor integration in the primate superior colliculus. I. Motor convergence. Journal of Neurophysiology, 57(1), 22–34.

    CAS  PubMed  Google Scholar 

  • Jay, M. F., & Sparks, D. L. (1987b). Sensorimotor integration in the primate superior colliculus. II. Coordinates of auditory signals. Journal of Neurophysiology, 57(1), 35–55.

    CAS  PubMed  Google Scholar 

  • Lee, J., & Groh, J. M. (2012). Auditory signals evolve from hybrid- to eye-centered coordinates in the primate superior colliculus. Journal of Neurophysiology, 108(1), 227–242.

    PubMed  PubMed Central  Google Scholar 

  • Lee, J., & Groh, J. M. (2014). Different stimuli, different spatial codes: A visual map and an auditory rate code for oculomotor space in the primate superior colliculus. PLoS One, 9(1), e85017.

    PubMed  PubMed Central  Google Scholar 

  • Lewald, J. (1997). Eye-position effects in directional hearing. Behavioural Brain Research, 87(1), 35–48.

    CAS  PubMed  Google Scholar 

  • Lewald, J. (1998). The effect of gaze eccentricity on perceived sound direction and its relation to visual localization. Hearing Research, 115(1–2), 206–216.

    CAS  PubMed  Google Scholar 

  • Lewald, J., & Ehrenstein, W. H. (1996). The effect of eye position on auditory lateralization. Experimental Brain Research, 108(3), 473–485.

    CAS  PubMed  Google Scholar 

  • Lewald, J., & Ehrenstein, W. H. (2001). Effect of gaze direction on sound localization in rear space. Neuroscience Research, 39(2), 253–257.

    CAS  PubMed  Google Scholar 

  • Lewald, J., & Getzmann, S. (2006). Horizontal and vertical effects of eye-position on sound localization. Hearing Research, 213(1–2), 99–106.

    PubMed  Google Scholar 

  • Linden, J. F., Grunewald, A., & Andersen, R. A. (1999). Responses to auditory stimuli in macaque lateral intraparietal area II. Behavioral modulation. Journal of Neurophysiology, 82(1), 343–358.

    CAS  PubMed  Google Scholar 

  • Luschei, E. S., & Fuchs, A. F. (1972). Activity of brain stem neurons during eye movements of alert monkeys. Journal of Neurophysiology, 35(4), 445–461.

    CAS  PubMed  Google Scholar 

  • Maddox, R. K., Pospisil, D. A., Stecker, G. C., & Lee, A. K. C. (2014). Directing eye gaze enhances auditory spatial cue discrimination. Current Biology, 24(7), 748–752.

    CAS  PubMed  Google Scholar 

  • Maier, J. X., & Groh, J. M. (2009). Multisensory guidance of orienting behavior. Hearing Research, 258(1–2), 106–112.

    PubMed  PubMed Central  Google Scholar 

  • Maier, J. X., & Groh, J. M. (2010). Comparison of gain-like properties of eye position signals in inferior colliculus versus auditory cortex of primates. Frontiers in Integrative Neuroscience, 4, 121.

    PubMed  PubMed Central  Google Scholar 

  • Marrone, N., Mason, C. R., & Kidd, G. (2008). Tuning in the spatial dimension: Evidence from a masked speech identification task. The Journal of the Acoustical Society of America, 124(2), 1146–1158.

    PubMed  PubMed Central  Google Scholar 

  • Mays, L. E., & Sparks, D. L. (1980). Dissociation of visual and saccade-related responses in superior colliculus neurons. Journal of Neurophysiology, 43(1), 207–232.

    CAS  PubMed  Google Scholar 

  • McAlpine, D., & Grothe, B. (2003). Sound localization and delay lines—Do mammals fit the model? Trends in Neurosciences, 26(7), 347–350.

    CAS  PubMed  Google Scholar 

  • McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746–748.

    CAS  PubMed  Google Scholar 

  • Meredith, A. M., & Stein, B. E. (1986a). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Research, 365(2), 350–354.

    CAS  PubMed  Google Scholar 

  • Meredith, M. A., & Stein, B. E. (1986b). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662.

    CAS  PubMed  Google Scholar 

  • Metzger, R. R., Mullette-Gillman, O. D. A., Underhill, A. M., Cohen, Y. E., & Groh, J. M. (2004). Auditory saccades from different eye positions in the monkey: Implications for coordinate transformations. Journal of Neurophysiology, 92(4), 2622–2627.

    PubMed  Google Scholar 

  • Middlebrooks, J. C., & Onsan, Z. A. (2012). Stream segregation with high spatial acuity. The Journal of the Acoustical Society of America, 132(6), 3896–3911.

    PubMed  PubMed Central  Google Scholar 

  • Middlebrooks, J. C., Clock, A. E., Xu, L., & Green, D. M. (1994). A panoramic code for sound location by cortical neurons. Science, 264(5160), 842–843.

    CAS  PubMed  Google Scholar 

  • Middlebrooks, J. C., Xu, L., Eddins, A. C., & Green, D. M. (1998). Codes for sound-source location in nontonotopic auditory cortex. Journal of Neurophysiology, 80(2), 863–881.

    CAS  PubMed  Google Scholar 

  • Mills, A. W. (1958). On the minimum audible angle. The Journal of the Acoustical Society of America, 30(4), 237–246.

    Google Scholar 

  • Mohler, C. W., Goldberg, M. E., & Wurtz, R. H. (1973). Visual receptive fields of frontal eye field neurons. Brain Research, 61, 385–389.

    CAS  PubMed  Google Scholar 

  • Mullette-Gillman, O. D. A., Cohen, Y. E., & Groh, J. M. (2005). Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus. Journal of Neurophysiology, 94(4), 2331–2352.

    PubMed  Google Scholar 

  • Mullette-Gillman, O. D. A., Cohen, Y. E., & Groh, J. M. (2009). Motor-related signals in the intraparietal cortex encode locations in a hybrid, rather than eye-centered reference frame. Cerebral Cortex, 19(8), 1761–1775.

    PubMed  Google Scholar 

  • Populin, L. C., & Yin, T. C. T. (1998). Sensitivity of auditory cells in the superior colliculus to eye position in the behaving cat. In A. R. Palmer, A. Q. Summerfield, & R. Meddis (Eds.), Psychophysical and physiological advances in hearing (pp. 441–448). London: Whurr.

    Google Scholar 

  • Populin, L. C., Tollin, D. J., & Yin, T. C. T. (2004). Effect of eye position on saccades and neuronal responses to acoustic stimuli in the superior colliculus of the behaving cat. Journal of Neurophysiology, 92(4), 2151–2167.

    PubMed  Google Scholar 

  • Porter, K. K., Metzger, R. R., & Groh, J. M. (2006). Representation of eye position in primate inferior colliculus. Journal of Neurophysiology, 95(3), 1826–1842.

    PubMed  Google Scholar 

  • Porter, K. K., Metzger, R. R., & Groh, J. M. (2007). Visual- and saccade-related signals in the primate inferior colliculus. Proceedings of the National Academy of Sciences of the United States of America, 104(45), 17855–17860.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Pouget, A., & Sejnowski, T. J. (1997). Spatial transformations in the parietal cortex using basis functions. Journal of Cognitive Neuroscience, 9(2), 222–237.

    CAS  PubMed  Google Scholar 

  • Razavi, B., O’Neill, W. E., & Paige, G. D. (2007). Auditory Spatial perception dynamically realigns with changing eye position. The Journal of Neuroscience, 27(38), 10249–10258.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Robinson, D. A. (1972). Eye movements evoked by collicular stimulation in the alert monkey. Vision Research, 12(11), 1795–1808.

    CAS  PubMed  Google Scholar 

  • Robinson, D. A., & Fuchs, A. F. (1969). Eye movements evoked by stimulation of frontal eye fields. Journal of Neurophysiology, 32(5), 637–648.

    CAS  PubMed  Google Scholar 

  • Russo, G. S., & Bruce, C. J. (1994). Frontal eye field activity preceding aurally guided saccades. Journal of Neurophysiology, 71(3), 1250–1253.

    CAS  PubMed  Google Scholar 

  • Sajad, A., Sadeh, M., Keith, G. P., Yan, X., Wang, H., & Crawford, J. D. (2015). Visual-motor transformations within frontal eye fields during head-unrestrained gaze shifts in the monkey. Cerebral Cortex, 25(10), 3932–3952.

    PubMed  Google Scholar 

  • Schiller, P. H., True, S. D., & Conway, J. L. (1979). Effects of frontal eye field and superior colliculus ablations on eye movements. Science, 206(4418), 590–592.

    CAS  PubMed  Google Scholar 

  • Schiller, P. H., True, S. D., & Conway, J. L. (1980). Deficits in eye movements following frontal eye-field and superior colliculus ablations. Journal of Neurophysiology, 44(6), 1175–1189.

    CAS  PubMed  Google Scholar 

  • Sommer, M. A., & Wurtz, R. H. (2008). Brain circuits for the internal monitoring of movements. Annual Review of Neuroscience, 31, 317–338.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Sparks, D. L. (1975). Response properties of eye movement-related neurons in the monkey superior colliculus. Brain Research, 90(1), 147–152.

    CAS  PubMed  Google Scholar 

  • Sparks, D. L. (1978). Functional properties of neurons in the monkey superior colliculus: Coupling of neuronal activity and saccade onset. Brain Research, 156(1), 1–16.

    CAS  PubMed  Google Scholar 

  • Sparks, D. L., & Hartwich-Young, R. (1989). The deep layers of the superior colliculus. Reviews of Oculomotor Research, 3, 213–255.

    CAS  PubMed  Google Scholar 

  • Stricanne, B., Andersen, R. A., & Mazzoni, P. (1996). Eye-centered, head-centered, and intermediate coding of remembered sound locations in area LIP. Journal of Neurophysiology, 76(3), 2071–2076.

    CAS  PubMed  Google Scholar 

  • Wann, J. P., & Ibrahim, S. F. (1992). Does limb proprioception drift? Experimental Brain Research, 91(1), 162–166.

    CAS  PubMed  Google Scholar 

  • Weerts, T. C., & Thurlow, W. R. (1971). The effects of eye position and expectation on sound localization. Perception & Psychophysics, 9(1), 35–39.

    Google Scholar 

  • Werner-Reiss, U., & Groh, J. M. (2008). A rate code for sound azimuth in monkey auditory cortex: Implications for human neuroimaging studies. The Journal of Neuroscience, 28(14), 3747–3758.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Werner-Reiss, U., Kelly, K. A., Trause, A. S., Underhill, A. M., & Groh, J. M. (2003). Eye position affects activity in primary auditory cortex of primates. Current Biology, 13(7), 554–562.

    CAS  PubMed  Google Scholar 

  • Wood, K. C., & Bizley, J. K. (2015). Relative sound localisation abilities in human listeners. The Journal of the Acoustical Society of America, 138(2), 674–686.

    PubMed  PubMed Central  Google Scholar 

  • Woods, T. M., Lopez, S. E., Long, J. H., Rahman, J. E., & Recanzone, G. H. (2006). Effects of stimulus azimuth and intensity on the single-neuron activity in the auditory cortex of the alert macaque monkey. Journal of Neurophysiology, 96(6), 3323–3337.

    PubMed  Google Scholar 

  • Wurtz, R. H., & Goldberg, M. E. (1972). Activity of superior colliculus in behaving monkey. III. Cells discharging before eye movements. Journal of Neurophysiology, 35(4), 575–586.

    CAS  PubMed  Google Scholar 

  • Zahn, J. R., Abel, L. A., Dell’Osso, L. F., & Daroff, R. B. (1979). The audioocular response: Intersensory delay. Sensory Processes, 3(1), 60.

    CAS  PubMed  Google Scholar 

  • Zambarbieri, D., Schmid, R., Magenes, G., & Prablanc, C. (1982). Saccadic responses evoked by presentation of visual and auditory targets. Experimental Brain Research, 47(3), 417–427.

    CAS  PubMed  Google Scholar 

  • Zwiers, M. P., Versnel, H., & Van Opstal, A. J. (2004). Involvement of monkey inferior colliculus in spatial hearing. The Journal of Neuroscience, 24(17), 4145–4156.

    CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Neurobiology, Center for Cognitive Neuroscience, Duke University, Durham, NC, USA

    Shawn M. Willett & Jennifer M. Groh

  2. Department of Psychology and Neuroscience, Center for Cognitive Neuroscience, Duke University, Durham, NC, USA

    Jennifer M. Groh

  3. Department of Biomedical Engineering, University of Rochester, Rochester, NY, USA

    Ross K. Maddox

  4. Department of Neuroscience, University of Rochester, Rochester, NY, USA

    Ross K. Maddox

  5. Del Monte Institute for Neuroscience, University of Rochester, Rochester, NY, USA

    Ross K. Maddox

  6. Center for Visual Science, University of Rochester, Rochester, NY, USA

    Ross K. Maddox

Authors
  1. Shawn M. Willett
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer M. Groh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ross K. Maddox
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shawn M. Willett .

Editor information

Editors and Affiliations

  1. Department of Speech and Hearing Sciences, Institute for Learning and Brain Sciences, University of Washington, Seattle, WA, USA

    Adrian K. C. Lee

  2. Departments of Hearing and Speech Sciences, Psychiatry, Psychology and Vanderbilt Brain Institute, Vanderbilt University, Nashville, TN, USA

    Mark T. Wallace

  3. Department of Integrative Physiology and Neuroscience, Washington State University Vancouver, Vancouver, WA, USA

    Allison B. Coffin

  4. Department of Biology, University of Maryland, College Park, MD, USA

    Arthur N. Popper

  5. Department of Psychology, Loyola University Chicago, Chicago, IL, USA

    Richard R. Fay

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Willett, S.M., Groh, J.M., Maddox, R.K. (2019). Hearing in a “Moving” Visual World: Coordinate Transformations Along the Auditory Pathway. In: Lee, A., Wallace, M., Coffin, A., Popper, A., Fay, R. (eds) Multisensory Processes. Springer Handbook of Auditory Research, vol 68. Springer, Cham. https://doi.org/10.1007/978-3-030-10461-0_5

Download citation

Publish with us

Policies and ethics

Search

Navigation