Abstract
This paper focuses on the localization of footstep sounds interactively generated during walking and provided through headphones. Three distinct experiments were conducted in a laboratory involving a pair of sandals enhanced with pressure sensors and a footstep synthesizer capable of simulating two typologies of surface materials: solid (e.g., wood) and aggregate (e.g., gravel). Different sound delivery methods (mono, stereo, binaural) as well as several surface materials, in the presence or absence of concurrent contextual auditory information provided as soundscapes, were evaluated in a vertical localization task. Results showed that solid surfaces were localized significantly farther from the walker’s feet than the aggregate ones. This effect was independent of the used rendering technique, of the presence of soundscapes, and of merely temporal or spectral attributes of sound. The effect is hypothesized to be due to a semantic conflict between auditory and haptic information such that the higher the semantic incongruence the greater the distance of the perceived sound source from the feet. The presented results contribute to the development of further knowledge toward a basis for the design of continuous multimodal feedback in virtual reality applications .
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Audio examples of the involved stimuli can be found at http://www.ahws-project.net/audio.html. A video of an apparatus similar to that involved in the experiment can be found at http://www.youtube.com/watch?v=kRKcKgYCPCY.
Such values were chosen according to the results of a previous experiment whose goal was to find the appropriate level of amplitude for those synthesized sounds (Turchet and Serafin 2013). Measurements were conducted by placing the microphone of an SPL meter inside one of the two headphones: Such microphone was inserted in a hole, having its same diameter, created in a piece of hardwood which was subsequently sealed against one of the two headphones. The amplitude peak value of the footstep sound was considered.
The mean ILDs were extracted from the CIPIC HRTF database (Algazi et al. 2001).
References
Algazi VR, Duda RO, Duraiswami R, Gumerov NA, Tang Z (2002) Approximating the head-related transfer function using simple geometric models of the head and torso. J Acoust Soc Am 112(5):2053–2064
Algazi VR, Duda RO, Thompson DM (2002) The use of head-and-torso models for improved spatial sound synthesis. In: Proceedings of 113th convention audio engineering society, Los Angeles, pp 1–18
Algazi VR, Duda RO, Thompson DM, Avendano C (2001) The CIPIC HRTF database. In: Proceedings of IEEE workshop on applications signal processing, audio and acoustic. New Paltz, New York, pp 1–4
Avanzini F, Rocchesso D (2001) Modeling collision sounds: non-linear contact force. In: Proceedings of digital audio effects conference, pp 61–66
Begault DR, Wenzel EM, Anderson MR (2001) Direct comparison of the impact of head tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. J Audio Eng Soc 49(10):904–916
Blauert J (1983) Spatial hearing: the psychophysics of human sound localization. MIT Press, Cambridge
Boren BB, Geronazzo M, Majdak P, Choueiri E (2014) PHOnA: a public dataset of measured headphone transfer functions. In: Proceedings of 137th audio engineering society convention
Burkhard MD, Sachs RM (1975) Anthropometric manikin for acoustic research. J Acoust Soc Am 58(1):214–222
Caclin A, Soto-Faraco S, Kingstone A, Spence C (2002) Tactile“capture” of audition. Percept Psychophys 64(4):616–630
Cheng CI, Wakefield GH (2001) Introduction to head-related transfer functions (HRTFs): representations of HRTFs in time, frequency, and space. J Audio Eng Soc 49(4):231–249
Cook P (1997) Physically informed sonic modeling (phism): synthesis of percussive sounds. Comput Music J 21(3):38–49
Geronazzo M, Spagnol S, Avanzini F (2013) Mixed structural modeling of head-related transfer functions for customized binaural audio delivery. In Proceedings of 18th international conference on digital signal processing (DSP 2013). Santorini, Greece
Geronazzo M, Spagnol S, Avanzini F (2013) A modular framework for the analysis and synthesis of head-related transfer functions. In: Proceedings of 134th audio engineering society convention, Rome, Italy
Geronazzo M, Spagnol S, Bedin A, Avanzini F (2014) Enhancing vertical localization with image-guided selection of non-individual head-related transfer functions. In: Proceedings of IEEE international conference on acoustics, speech, and signal processing (ICASSP 2014), Firenze, Italy, pp 4496–4500
Giordano B, Visell Y, Yao HY, Hayward V, Cooperstock J, McAdams S (2012) Identification of walked-upon materials in auditory, kinesthetic, haptic and audio-haptic conditions. J Acoust Soc Am 131:4002–4012
Hebrank J, Wright D (1974) Spectral cues used in the localization of sound sources on the median plane. J Acoust Soc Am 56(6):1829–1834
Howard IP, Templeton WB (1966) Human spatial orientation. Wiley, New York
Hunt KH, Crossley FRE (1975) Coefficient of restitution interpreted as damping in vibroimpact. ASME J Appl Mech 42(2):440–445
Hwang S, Park Y, Park Y (2008) Modeling and customization of head-related impulse responses based on general basis functions in time domain. Acta Acust United Acust 94(6):965–980
Jousmäki V, Hari R (1998) Parchment-skin illusion: sound-biased touch. Curr Biol 8(6):R190–R191
Kitagawa N, Spence C (2006) Audiotactile multisensory interactions in human information processing. Jpn Psychol Res 48(3):158–173
Kobayashi Y, Osaka R, Hara T, Fujimoto H (2008) How accurately people can discriminate the differences of floor materials with various elasticities. IEEE Trans Neural Rehab Syst Eng 16(1):99–105
Laurienti P, Kraft R, Maldjian J, Burdette J, Wallace M (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158(4):405–414
Middlebrooks JC (1999) Individual differences in external-ear transfer functions reduced by scaling in frequency. J Acoust Soc Am 106(3):1480–1492
Møller H, Sørensen MF, Jensen CB, Hammershøi D (1996) Binaural technique: Do we need individual recordings? J Audio Eng Soc 44(6):451–469
Nordahl R, Berrezag A, Dimitrov S, Turchet L, Hayward V, Serafin S (2010) Preliminary experiment combining virtual reality haptic shoes and audio synthesis. In: Haptics: generating and perceiving tangible sensations, lecture notes in computer science, Springer, Berlin, vol 6192, pp 123–129
Nordahl R, Serafin S, Turchet L (2010) Sound synthesis and evaluation of interactive footsteps for virtual reality applications. In: Proceedings of the IEEE virtual reality conference. IEEE Press, pp 147–153
Papetti S, Civolani M, Fontana F (2011) Rhythm’n’shoes: a wearable foot tapping interface with audio-tactile feedback. In: Proceedings of the international conference on new interfaces for musical expression, pp 473–476
Papetti S, Fontana F, Civolani M, Berrezag A, Hayward V (2010) Audio-tactile display of ground properties using interactive shoes. In: Haptic and audio interaction design, Lecture notes in computer science, Springer, Berlin, vol 6306, pp 117–128
Serafin S, Turchet L, Nordahl R, Dimitrov S, Berrezag A, Hayward V (2010) Identification of virtual grounds using virtual reality haptic shoes and sound synthesis. In: Proceedings of eurohaptics symposium on haptic and audio-visual stimuli: enhancing experiences and interaction, pp 61–70
Slater M, Lotto B, Arnold MM, Sanchez-Vives MV (2009) How we experience immersive virtual environments: the concept of presence and its measurement. Anuario de Psicologia 40(2):193–210
Spagnol S, Geronazzo M, Avanzini F (2013) On the relation between pinna reflection patterns and head-related transfer function features. IEEE Trans Audio Speech Lang Process 21(3):508–519
Spagnol S, Geronazzo M, Rocchesso D, Avanzini F (2014) Synthetic individual binaural audio delivery by pinna image processing. Int J Pervasive Comput Commun 10(3):239–254
Stein B, Meredith M (1993) The merging of the senses. MIT Press, Cambridge
Steinicke F, Visell Y, Campos J, Lécuyer A (2013) Human walking in virtual environments: perception, technology, and applications. Springer, Berlin
Strutt JW (1904) On the acoustic shadow of a sphere. Philos Trans R Soc Lond 203:87–110
Thomas JP, Shiffrar M (2010) I can see you better if i can hear you coming: action-consistent sounds facilitate the visual detection of human gait. J Vis 10(12):14. doi:10.1167/10.12.14
Turchet L (2015) Designing presence for real locomotion in immersive virtual environments: an affordance-based experiential approach. Virtual Real (accepted)
Turchet L (2015) Footstep sounds synthesis: design, implementation, and evaluation of foot-floor interactions, surface materials, shoe types, and walkers’ features. Appl Acoust (in press)
Turchet L, Camponogara I, Cesari P (2015) Interactive footstep sounds modulate the perceptual-motor aftereffect of treadmill walking. Exp Brain Res 233:205–214
Turchet L, Nordahl R, Berrezag A, Dimitrov S, Hayward V, Serafin S (2010) Audio-haptic physically based simulation of walking on different grounds. In: Proceedings of IEEE international workshop on multimedia signal processing, IEEE Press, pp 269–273
Turchet L, Serafin S (2011) A preliminary study on sound delivery methods for footstep sounds. In: Proceedings of digital audio effects conference, pp 53–58
Turchet L, Serafin S (2013) Investigating the amplitude of interactive footstep sounds and soundscape reproduction. Appl Acoust 74(4):566–574
Turchet L, Serafin S (2014) Semantic congruence in audio-haptic simulation of footsteps. Appl Acoust 75(1):59–66
Turchet L, Serafin S, Cesari P (2013) Walking pace affected by interactive sounds simulating stepping on different terrains. ACM Trans Appl Percept 10(4):23:1–23:14
Turchet L, Serafin S, Nordahl R (2010) Examining the role of context in the recognition of walking sounds. In: Proceedings of sound and music computing conference
Visell Y, Cooperstock J, Giordano B, Franinovic K, Law A, McAdams S, Jathal K, Fontana F (2008) A vibrotactile device for display of virtual ground materials in walking. Lect Notes Comput Sci 5024:420–426
Visell Y, Fontana F, Giordano B, Nordahl R, Serafin S, Bresin R (2009) Sound design and perception in walking interactions. Int J Hum Comput Stud 67(11):947–959
Vliegen J, Van Opstal AJ (2004) The influence of duration and level on human sound localization. J Acoust Soc Am 115(4):1705–1713
Wenzel EM, Arruda M, Kistler DJ, Wightman FL (1993) Localization using nonindividualized head-related transfer functions. J Acoust Soc Am 94(1):111–123
Zanotto D, Turchet L, Boggs E, Agrawal S (2014) Solesound: Towards a novel portable system for audio-tactile underfoot feedback. In: Proceedings of the 5th IEEE international conference on biomedical robotics and biomechatronics, pp 193–198
Acknowledgments
The work of the first author was supported by the Danish Council for Independent Research, Grant No. 12-131985.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Turchet, L., Spagnol, S., Geronazzo, M. et al. Localization of self-generated synthetic footstep sounds on different walked-upon materials through headphones. Virtual Reality 20, 1–16 (2016). https://doi.org/10.1007/s10055-015-0272-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-015-0272-6