How Sensorimotor Interactions Enable Sentence Imitation

Abstract

Despite intensive debates regarding action imitation and sentence imitation, few studies have examined their relationship. In this paper, we argue that the mechanism of action imitation is necessary and in some cases sufficient to describe sentence imitation. We first develop a framework for action imitation in which key ideas of Hurley’s shared circuits model are integrated with Wolpert et al.’s motor selection mechanism and its extensions. We then explain how this action-based framework clarifies sentence imitation without a language-specific faculty. Finally, we discuss the empirical support for and philosophical significance of this perspective.

Appendix

The Convertor

The convertor receives the actor’s intention d _t and actual state x _t at time t, and it outputs desired state x ^* _t+1 at time t + 1. We use p _t  = f (x _t , d _t ) and  x ^* _t+1 = g(d _t , p _t ) to describe the parameter generated by motor control processing and a desired motor state generated by motor planning, where f and g are functions with inverse relationship $x_{t + 1}^{*} = g\left({d_{t},f\left({x_{t},d_{t}} \right)} \right).$

A Combined Forward and Inverse Model

Suppose each model activates multiple predictors 1, 2, 3,…, n at t, and select among their next state predictions $\hat{x}_{t + 1}^{1},\hat{x}_{t + 1}^{2},\hat{x}_{t + 1}^{3}, \ldots,\hat{x}_{t + 1}^{n}$ through testing (see Fig. 3). Each predictor receives actual feedback x _t and the efference copy of motor command u _t to generate a prediction. The prediction of the i-th predictor is $\hat{x}_{t + 1}^{i} = \varPhi \left({w_{t}^{i}, x_{t},u_{t}} \right)$, where $w_{t}^{i}$ represents the parameters of the function approximator $\varPhi$. This predictive next state is compared with the actual next state. If an error occurs, then the wrong prediction is sent to a responsibility estimator to generate responsibility signal $\lambda_{t}^{i}$, which can be calculated by using the softmax activation function.

$$\lambda_{t}^{i} = \frac{{e^{{{{- \left| {x_{t} + - \hat{x}_{t}^{i}} \right|^{2}} \mathord{\left/{\vphantom {{- \left| {x_{t} + - \hat{x}_{t}^{i}} \right|^{2}} {\sigma^{2}}}} \right. \kern-0pt} {\sigma^{2}}}}}}}{{\mathop \sum \nolimits_{j = 1}^{n} e^{{{{- \left| {x_{t} + - \hat{x}_{t}^{i}} \right|^{2}} \mathord{\left/{\vphantom {{- \left| {x_{t} + - \hat{x}_{t}^{i}} \right|^{2}} {\sigma^{2}}}} \right. \kern-0pt} {\sigma^{2}}}}}}}$$

(1)

In Eq. (1), $x_{t}$ is the framework’s actual voice output, and $\sigma$ is a scaling constant. The softmax activation function calculates the error signals and normalizes them into probability values between 0 and 1. Predictors with few errors receive higher responsibilities. Thus, responsibility signals can regulate predictor learning in a competitive manner. Moreover, a paired controller exists for each predictor, and it receives the desire next state x ^* _t+1 and outputs motor commands. Suppose that the framework activates controllers 1, 2, 3,…, n and generates $u_{t}^{1},u_{t}^{2},u_{t}^{3}, \ldots, u_{t}^{n}$. The motor command of the i-th controllers is $u_{t}^{i} = \psi \left({\alpha_{t}^{i}, x_{t + 1}^{*}} \right)$, where $\alpha_{t}^{i}$ is the parameter of a function approximator $\psi$. The summation of the motor commands generated by controllers 1, 2, 3,…, n is represented by Eq. (2).

$$u_{t} = \mathop \sum \limits_{i = 1}^{n} \lambda_{t}^{i} u_{t}^{i} = \mathop \sum \limits_{i = 1}^{n} \lambda_{t}^{i} \psi \left({\alpha_{t}^{i}, x_{t + 1}^{*}} \right)$$

(2)

The Hierarchy of Models

For simplicity, we describe only a two-level hierarchy, although it is extendable to an arbitrary number of levels. Suppose that the predictor of the i-th higher-level model receives actual state X _t and the efference copy of U _t at t, and suppose that it outputs the approximate prediction $\hat{X}_{t + 1}^{i}$ without activating subordinate controllers:

$$\hat{X}_{t + 1}^{i} = \varPhi \left({W_{t}^{i},X_{t},U_{t}} \right) = \left({P\left({1|W_{t}^{i},X_{t},U_{t}} \right), \ldots,P\left({n|W_{t}^{i},X_{t},U_{t}} \right)} \right)$$

(3)

In Eq. (3), $\varPhi$ refers to a vector-valued and nonlinear function approximator; $W_{t}^{i}$ is the synaptic weight of the higher-level j-th pair; X _t is the current state (posterior probability); U _t is the higher-level command; and $P\left({j|W_{t}^{i},X_{t},U_{t}} \right)$ refers to the posterior probability in which the j-th pair is selected under $W_{t}^{i},X_{t},{\text{and}}\;U_{t}$. Likewise, the i-th higher-level prediction $\hat{X}_{t + 1}^{i}$ is compared with actual state X _t from the subordinate level to generate higher-level responsibility (i.e., prior probability) $\lambda_{i}^{H} \left(t \right)$ via the estimator

$$\lambda_{i}^{H} \left(t \right) = \frac{{\hat{\lambda}_{i}^{H} \left(t \right)e^{{- {{\left| {x_{t} - \hat{x}_{t}^{i}} \right|^{2}} \mathord{\left/{\vphantom {{\left| {x_{t} - \hat{x}_{t}^{i}} \right|^{2}} {\sigma^{2}}}} \right. \kern-0pt} {\sigma^{2}}}}}}}{{\mathop \sum \nolimits_{j = 1}^{N} \hat{\lambda}_{i}^{H} \left(t \right)e^{{- {{\left| {x_{t} - \hat{x}_{t}^{i}} \right|^{2}} \mathord{\left/{\vphantom {{\left| {x_{t} - \hat{x}_{t}^{i}} \right|^{2}} {\sigma^{2}}}} \right. \kern-0pt} {\sigma^{2}}}}}}}$$

(4)

$\lambda_{i}^{H} \left(t \right)$ can regulate the subordinate level in a competitive manner. Moreover, each higher-v predictor has a paired controller that receives the desired next state X ^* _t+1 and current state X _t from the subordinate level as input. X ^* _t+1 is an abstract representation that determines the selection and activation order of subordinate controllers. Each higher-level controller generates commands to the subordinate level, and the command of the i-th higher-level controller is $U_{t}^{i} = \varPsi \left({\varLambda_{t}^{i}, X_{t + 1}^{*},X_{t}} \right)$, where $\varLambda_{t}^{i}$ is the parameter of a function approximator $\varPsi$. Then, $U_{t}$, the summation of (prior probability) commands for the lowest pairs, is weighted by $\lambda_{i}^{H} \left(t \right)$:

$$U_{t} = \left({\hat{\lambda}_{1}^{L} \left(t \right), \ldots,\hat{\lambda}_{n}^{L} \left(t \right)} \right) = \mathop \sum \limits_{i = 1}^{N} \lambda_{i}^{H} \left(t \right)U_{t}^{i} = \mathop \sum \limits_{i = 1}^{N} \lambda_{t}^{H} \left(t \right)\varPsi \left({\varLambda_{t}^{i}, X_{t + 1}^{*},X_{t}} \right)$$

(5)

Abstraction of the Sequence of Constituents

Suppose that the k-th higher-level model receives a sequence of actual input [$x_{0},x_{1}, \ldots,x_{t}$] (represented by X _t ) and generates a prediction regarding a sequence of output [$\hat{x}_{1}^{\text{k}}, \ldots,\hat{x}_{t}^{\text{k}}$] (represented by $\hat{X}_{t + 1}^{\text{k}}$). The task of this higher level is to determine the prediction that has the least mismatch with the next actual sequence of input. The comparison result can be represented as Eq. (4). The responsibility signal can be used to revise higher-level commands (see Eq. (5)), which determines the behavior of the subordinate level. The efference copy of the commands can also be used for further prediction (and revision). We also use a recurrent network to describe the higher-level prediction of sequence $\hat{X}_{t + 1}^{\text{k}} = f\left({W_{t}^{k}, X_{t}} \right)$, in which f is a nonlinear function that can use weights $W_{t}^{k}$ to predict a vector of posterior probabilities. The network dynamics can be described as:

$$\tau \frac{d}{{d_{t}}}a_{i} \left(t \right) = - a_{i} \left(t \right) + \mathop \sum \limits_{j = 1} W_{ij}^{K} b_{j} \left(t \right)$$

$$b_{i} \left(t \right) = \left\{{\begin{array}{*{20}l} {g\left({a_{i} \left(t \right)} \right)} \hfill & {\left({\text{output}} \right)} \hfill \\ {X_{t}^{i}} \hfill & {\left({\text{input}} \right)} \hfill \\ \end{array}} \right.$$

In the above differential equation, g(X) is the sigmoid function with derivative g(X)(1 − g(X)), $a_{i}$ is the activation, and $b_{i}$ is the output at the i-th node. In Haruno et al.’s (2003) simulation, their models successfully learned two sequences and determined the one that should be reproduced under a given context, even when 5 % noise was added.

Control Variable Processing for Estimated Intention

Suppose that the highest models generate predictive control parameters $\hat{p}_{t}^{1},\hat{p}_{t}^{2},\hat{p}_{t}^{3}, \ldots, \hat{p}_{t}^{n}$ at time t. Each predictive parameter will be compared with actually observed parameter p _t from the control variable encoding (Fig. 4). The comparison result is represented by the responsibility signal:

Fig. 4

Control variable processing in action comprehension mode

$$\lambda_{t}^{i} = \frac{{e^{{{{- \left| {p_{t} + - \hat{p}_{t}^{i}} \right|^{2}} \mathord{\left/{\vphantom {{- \left| {p_{t} + - \hat{p}_{t}^{i}} \right|^{2}} {\sigma^{2}}}} \right. \kern-0pt} {\sigma^{2}}}}}}}{{\mathop \sum \nolimits_{j = 1}^{n} e^{{{{- \left| {p_{t} + - \hat{p}_{t}^{i}} \right|^{2}} \mathord{\left/{\vphantom {{- \left| {p_{t} + - \hat{p}_{t}^{i}} \right|^{2}} {\sigma^{2}}}} \right. \kern-0pt} {\sigma^{2}}}}}}}$$

(6)

The mechanism of intention/parameter matching in control variable processing generates simulated intention $\hat{d}_{t}^{1},\hat{d}_{t}^{2},\hat{d}_{t}^{3}, \ldots, \hat{d}_{t}^{n}$. The final estimated intention is represented by:

$$d_{\text{t}} = \mathop \sum \limits_{i = 1}^{n} \lambda_{t}^{i} d_{t}^{i}$$

(7)

Searching for Meaning

Here, we use Oztop et al.’s (2005) algorithm of mental state search to infer the meaning that the speaker intends to convey. To initialize the algorithm, we also set T _k and S _k to an empty sequence (T _k  = S _k  = []). T _k and S _k represent sequences of observed and mentally simulated vectors of control variables extracted under the mental state k. Next, repeat steps (1)–(5) from speech onset to speech end.

• 1. Pick next possible mental state (j) (which can be thought of as an index for the possible referent to which the speaker is referring).

• 2. Observe: Extract the relevant control variables based on the hypothesized mental state (j), $x_{j}^{i}$, and add them to T _j (T _j  = [T _j , $x_{j}^{i}$]). Here, i indicates that the collected data were placed in ith position in the visual control variable sequence.

• 3. Simulate: Mentally simulate speech with mental state j while storing the simulated control variables x _j in S _j (S _j  = [$x_{j}^{0}, x_{j}^{1}, \ldots, x_{j}^{N}$], where N is the number of control variables collected during observation).

• 4. Compare: Compute the discounted difference between T _j and S _j , where N is the length of T _j and S _j . $D_{N} = \frac{{\left({1 - \gamma} \right)}}{{\left({1 = \gamma^{N + 1}} \right)}}\mathop \sum \limits_{i = 0}^{N} \left({x_{\text{sim}}^{i} - x^{0}} \right)^{T} {\mathbf{W}}\left({x_{\text{sim}}^{i} - x^{i}} \right)\gamma^{N - i}$, where $x_{\text{sim}}^{i}$ ∈ Sj and $x^{i}$ ∈ Tj and W is a diagonal matrix normalizing components of $x^{i}$ and γ is the discount factor.

• 5. If DN is smallest so far, set j _min = j.

Return: j _min (the observer infers that j _min is the actor’s intended meaning).