Abstract
Language and action have been thought of as closely related. Comprehending words or phrases that are related to actions commonly activates motor and premotor areas, and this comprehension process interacts with action preparation and/or execution. However, it remains unclear whether comprehending action-related language interacts with action observation. In the current study, we examined whether the observation of tool-use gesture subjects to interaction with language. In an electroencephalography (EEG) study (n = 20), participants were presented with video clips of an actor performing tool-use (TU, e.g., hammering with a fist) and emblematic (EM, e.g., the thumb up sign for ‘good job’) gestures accompanied by either comprehensible German (G) or incomprehensible Russian sentences (R). Participants performed a semantic judging task, evaluating whether the co-speech gestures were object- or socially-related. Behavioral results from the semantic task showed faster response for the TU versus EM gestures only in the German condition. For EEG, we found that TU elicited beta power decrease (~ 20 Hz) when compared to EM gestures, however this effect was reduced when gestures were accompanied by German instead of Russian sentences. We concluded that the processing of action-related sentences might facilitate gesture observation, in the sense that motor simulation required for TU gestures, as indexed by reduced beta power, was modulated when accompanied by comprehensible German speech. Our results corroborate the functional role of the beta oscillations during perception of hand gestures, and provide novel evidence concerning language–motor interaction.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agnew ZK, Wise RJ, Leech R (2012) Dissociating object directed and non-object directed action in the human mirror system; implications for theories of motor simulation. PLoS ONE 7:e32517
Andric M, Solodkin A, Buccino G, Goldin-Meadow S, Rizzolatti G, Small SL (2013) Brain function overlaps when people observe emblems, speech, and grasping. Neuropsychologia 51:1619–1629
Arnstein D, Cui F, Keysers C, Maurits NM, Gazzola V (2011) µ-suppression during action observation and execution correlates with BOLD in dorsal premotor, inferior parietal, and SI cortices. J Neurosci 31:14243–14249
Avanzini P, Fabbri-Destro M, Volta D, Daprati R, Rizzolatti E, Cantalupo G, G (2012) The dynamics of sensorimotor cortical oscillations during the observation of hand movements: an EEG study. PLoS ONE 7:e37534
Barsalou LW (2008) Grounded cognition. Annu Rev Psychol 59:617–645
Biau E, Soto-Faraco S (2013) Beat gestures modulate auditory integration in speech perception. Brain Lang 124:143–152
Boulenger V, Roy AC, Paulignan Y, Deprez V, Jeannerod M, Nazir TA (2006) Cross-talk between language processes and overt motor behavior in the first 200 msec of processing. J Cogn Neurosci 18:1607–1615
Boulenger V, Silber BY, Roy AC, Paulignan Y, Jeannerod M, Nazir TA (2008) Subliminal display of action words interferes with motor planning: a combined EEG and kinematic study. J Physiol (Paris) 102:130–136
Brinkman L, Stolk A, Dijkerman HC, de Lange FP, Toni I (2014) Distinct roles for alpha-and beta-band oscillations during mental simulation of goal-directed actions. J Neurosci 34:14783–14792
Buccino G, Binkofski F, Fink GR, Fadiga L, Fogassi L, Gallese V, Seitz RJ, Zilles K, Rizzolatti G, Freund HJ (2001) Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur J Neurosci 13:400–404
Buccino G, Riggio L, Melli G, Binkofski F, Gallese V, Rizzolatti G (2005) Listening to action-related sentences modulates the activity of the motor system: a combined TMS and behavioral study. Cogn Brain Res 24:355–363
Chersi F, Thill S, Ziemke T, Borghi AM (2010) Sentence processing: linking language to motor chains. Frontiers in neurorobotics 4:4
Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119:593–609
Goldin-Meadow S (2005) Hearing gesture: how our hands help us think. Harvard University Press, Cambridge
Green A, Straube B, Weis S, Jansen A, Willmes K, Konrad K, Kircher T (2009) Neural integration of iconic and unrelated coverbal gestures: a functional MRI study. Hum Brain Mapp 30:3309–3324
Groppe DM, Urbach TP, Kutas M (2011) Mass univariate analysis of event-related brain potentials/fields I: a critical tutorial review. Psychophysiology 48:1711–1725
Hari R, Forss N, Avikainen S, Kirveskari E, Salenius S, Rizzolatti G (1998) Activation of human primary motor cortex during action observation: a neuromagnetic study. Proc Natl Acad Sci 95:15061–15065
Hauk O, Johnsrude I, Pulvermüller F (2004) Somatotopic representation of action words in human motor and premotor cortex. Neuron 41:301–307
He Y, Gebhardt H, Steines M, Sammer G, Kircher T, Nagels A, Straube B (2015) The EEG and fMRI signatures of neural integration: an investigation of meaningful gestures and corresponding speech. Neuropsychologia 72:27–42
Higuchi S, Imamizu H, Kawato M (2007) Cerebellar activity evoked by common tool-use execution and imagery tasks: an fMRI study. Cortex 43:350–358
Holle H, Obermeier C, Schmidt-Kassow M, Friederici AD, Ward J, Gunter TC (2012) Gesture facilitates the syntactic analysis of speech. Front Psychol 3:74
Järveläinen J, Schuermann M, Hari R (2004) Activation of the human primary motor cortex during observation of tool use. Neuroimage 23:187–192
Johnson-Frey SH (2004) The neural bases of complex tool use in humans. Trends Cogn Sci 8:71–78
Johnson-Frey SH, Newman-Norlund R, Grafton ST (2005) A distributed left hemisphere network active during planning of everyday tool use skills. Cereb Cortex 15:681–695
Kendon A (1980) Gesticulation and speech: Two aspects of the process of utterance. Relat Verbal Nonverbal Commun 25:207–227
Kircher T, Straube B, Leube D, Weis S, Sachs O, Willmes K, Konrad K, Green A (2009) Neural interaction of speech and gesture: differential activations of metaphoric co-verbal gestures. Neuropsychologia 47:169–179
Kita S, Özyürek A (2003) What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. J Mem Lang 48:16–32
Klepp A, Niccolai V, Buccino G, Schnitzler A, Biermann-Ruben K (2015) Language–motor interference reflected in MEG beta oscillations. Neuroimage 109:438–448
Krahmer E, Swerts M (2007) The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception. J Mem Lang 57:396–414
Lindenberg R, Uhlig M, Scherfeld D, Schlaug G, Seitz RJ (2012) Communication with emblematic gestures: shared and distinct neural correlates of expression and reception. Hum Brain Mapp 33:812–823
Maris E, Oostenveld R (2007) Nonparametric statistical testing of EEG- and MEG-data. J Neurosci Methods 164:177–190
McNeill D (1992) Hand and mind: What gestures reveal about thought. University of Chicago Press, Chicago
McNeill D (2006) Gesture: a psycholinguistic approach. In: Brown K (ed) The encyclopedia of language linguistics. Elsevier, Boston, pp 58–66
McNeill D (2008) Gesture and thought. University of Chicago Press, Chicago
Moreno I, De Vega M, León I (2013) Understanding action language modulates oscillatory mu and beta rhythms in the same way as observing actions. Brain Cogn 82:236–242
Moreno I, De Vega M, León I, Bastiaansen M, Lewis AG, Magyari L (2015) Brain dynamics in the comprehension of action-related language. A time-frequency analysis of mu rhythms. Neuroimage 109:50–62
Muthukumaraswamy SD, Johnson BW, McNair NA (2004) Mu rhythm modulation during observation of an object-directed grasp. Cogn Brain Res 19:195–201
Newman-Norlund R, van Schie HT, van Hoek ME, Cuijpers RH, Bekkering H (2010) The role of inferior frontal and parietal areas in differentiating meaningful and meaningless object-directed actions. Brain Res 1315:63–74
Oldfield RC (1971) The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9:97–113
Perry A, Bentin S (2009) Mirror activity in the human brain while observing hand movements: a comparison between EEG desynchronization in the µ-range and previous fMRI results. Brain Res 1282:126–132
Pulvermuller F (2005) Brain mechanisms linking language and action. Nat Rev Neurosci 6:576–582
Pulvermüller F, Hauk O, Nikulin VV, Ilmoniemi RJ (2005) Functional links between motor and language systems. Eur J Neurosci 21:793–797
Quandt LC, Marshall PJ, Shipley TF, Beilock SL, Goldin-Meadow S (2012) Sensitivity of alpha and beta oscillations to sensorimotor characteristics of action: An EEG study of action production and gesture observation. Neuropsychologia 50:2745–2751
Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci 2:661–670
Salmelin R, Hámáaláinen M, Kajola M, Hari R (1995) Functional segregation of movement-related rhythmic activity in the human brain. Neuroimage 2:237–243
Sato M, Mengarelli M, Riggio L, Gallese V, Buccino G (2008) Task related modulation of the motor system during language processing. Brain Lang 105:83–90
Schaller F, Weiss S, Müller HM (2017) EEG beta-power changes reflect motor involvement in abstract action language processing. Brain Lang 168:95–105
Scorolli C, Borghi AM (2007) Sentence comprehension and action: Effector specific modulation of the motor system. Brain Res 1130:119–124
Skipper JI, Goldin-Meadow S, Nusbaum HC, Small SL (2009) Gestures orchestrate brain networks for language understanding. Curr Biol 19:661–667
Straube B, Green A, Bromberger B, Kircher T (2011) The differentiation of iconic and metaphoric gestures: Common and unique integration processes. Hum Brain Mapp 32:520–533
Straube B, Green A, Weis S, Kircher T (2012) A supramodal neural network for speech and gesture semantics: an fMRI study. PLoS ONE 7:e51207
Villarreal M, Fridman EA, Amengual A, Falasco G, Gerscovich ER, Ulloa ER, Leiguarda RC (2008) The neural substrate of gesture recognition. Neuropsychologia 46:2371–2382
Wang L, Chu M (2013) The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia 51:2847–2855
Willems RM, Hagoort P (2007) Neural evidence for the interplay between language, gesture, and action: a review. Brain Lang 101:278–289
Willems RM, Labruna L, D’Esposito M, Ivry R, Casasanto D (2011) A Functional role for the motor system in language understanding evidence from theta-burst transcranial magnetic stimulation. Psychol Sci 22:849–854
Wriessnegger S, Leeb R, Kaiser V, Neuper C, Müller-Putz G (2013) Watching object related movements modulates mirror-like activity in parietal brain regions. Clin Neurophysiol 124:1596–1604
Xu J, Gannon PJ, Emmorey K, Smith JF, Braun AR (2009) Symbolic gestures and spoken language are processed by a common neural system. Proc Natl Acad Sci 106:20664–20669
Acknowledgements
This research project is supported by a grant from the ‘Von Behring-Röntgen-Stiftung’ (Project No. 59-0002; 64-0001) and by the ‘Deutsche Forschungsgemeinschaft’ (Project No. DFG: STR 1146/9-1 and SFB/TRR135 project A3). M.S. is supported by the DFG (Project No. STR 1146/4-1). B.S. is supported by the DFG (Project No. STR 1146/8-1). Part of the work of this paper has been presented at the Tilburg Gesture Research meeting (TiGER 2013).
Author information
Authors and Affiliations
Corresponding author
Additional information
Handling Editor: Micah M. Murray.
Rights and permissions
About this article
Cite this article
He, Y., Steines, M., Sammer, G. et al. Action-Related Speech Modulates Beta Oscillations During Observation of Tool-Use Gestures. Brain Topogr 31, 838–847 (2018). https://doi.org/10.1007/s10548-018-0641-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10548-018-0641-z