Jarzynski’s Equality, Fluctuation Theorems, and Variance Reduction: Mathematical Analysis and Numerical Algorithms

Abstract

In this paper, we study Jarzynski’s equality and fluctuation theorems for diffusion processes. While some of the results considered in the current work are known in the (mainly physics) literature, we review and generalize these nonequilibrium theorems using mathematical arguments, therefore enabling further investigations in the mathematical community. On the numerical side, variance reduction approaches such as importance sampling method are studied in order to compute free energy differences based on Jarzynski’s equality.

Appendices

Appendix A: Connections with Thermodynamic Integration and Adiabatic Switching: Alchemical Transition Case

In this appendix, we study two (essentially equivalent) asymptotic regimes of nonequilibrium processes using formal arguments. In particular, we will derive the thermodynamic integration identity from Jarzynski’s identity, therefore bridging these two different free energy calculation methods. Let us point out that such a connection is indeed known in physics community [14], although we are not aware of its mathematical derivation in the literature. For simplicity, we only consider the alchemical transition case studied in Sect. 2 and assume the protocol $\lambda (\cdot )$ is deterministic with $\epsilon = 0$.

From Jarzynski’s equality to thermodynamic integration Thermodynamic integration is a well known method and has been widely used to compute free energy differences [23]. From the definition of the normalization constant $Z(\cdot )$ in (10), we can derive the thermodynamic integration identity by the simple argument

$$\begin{aligned} \Delta F(T)&= F(\lambda (T)) - F(\lambda (0))\nonumber \\&= -\beta ^{-1} \ln \frac{Z(\lambda (T))}{Z(\lambda (0))} \nonumber \\&= -\beta ^{-1} \int _0^T \frac{d}{ds}\Big ( \ln \frac{Z(\lambda (s))}{Z(\lambda (0))}\Big )\, ds \nonumber \\&= \int _0^T \Big ( \frac{\int _{{\mathbb {R}}^n} e^{-\beta V(x,\lambda (s))} \nabla _\lambda V(x,\lambda (s))\, dx)}{Z(\lambda (s))}\Big )\,\cdot f(\lambda (s), s)\, ds \nonumber \\&= \int _0^T \big ({\mathbf {E}}_{\mu _{\lambda (s)}} (\nabla _{\lambda } V)\big ) \cdot f(\lambda (s), s)\, ds\,. \end{aligned}$$

(144)

In the following, using a formal argument, we show that the identity (144) corresponds to the Jarzynski’s equality (29) in certain asymptotic limit. For this purpose, we consider the dynamics

$$\begin{aligned} \begin{aligned} d x(s)&= \frac{1}{\tau } b(x(s), \lambda (s))\,ds + \sqrt{\frac{2\beta ^{-1}}{\tau }} \sigma (x(s), \lambda (s)) \,dw^{(1)}(s)\,, \end{aligned} \end{aligned}$$

(145)

on $s \in [0, T]$, where $0 < \tau \ll 1$ and $\lambda (s)$ satisfies the ODE

$$\begin{aligned} {\dot{\lambda }}(s) = f(\lambda (s), s)\,. \end{aligned}$$

(146)

Clearly, dynamics (145) is related to (1) by rescaling time with the parameter $0 < \tau \ll 1$, and its infinitesimal generator is $\frac{1}{\tau } {\mathcal {L}}_1$, where ${\mathcal {L}}_1$ is defined in (6) with $\lambda (\cdot )$ being time dependent. The main observation is that, repeating the argument from Sect. 2.2, the Jarzynski’s equality (29) holds for (145) and (146) for any $\tau >0$. As a consequence,

$$\begin{aligned} e^{-\beta \Delta F(T)} = {\mathbf {E}}_{\mu (\lambda (0))} \Big (g(\cdot , \lambda (0), 0)\Big ) \,, \end{aligned}$$

(147)

where the function g now satisfies

$$\begin{aligned} \begin{aligned}&\partial _t g + \frac{1}{\tau }{\mathcal {L}}_{1} g + f\cdot \nabla _\lambda g - \beta \big (f\cdot \nabla _\lambda V \big ) g = 0\,, \quad 0 \le t < T\,,\\&g(\cdot ,\cdot , T) = 1\,. \end{aligned} \end{aligned}$$

(148)

To show that (147) reduces to the thermodynamic integration identity (144) as $\tau \rightarrow 0$, it is enough to study the asymptotic limit of (148). To this end, we consider the formal asymptotic expansion

$$\begin{aligned} g = g_0 + \tau g_1+\tau ^2g_2 + \cdots \end{aligned}$$

as $\tau \rightarrow 0$, where $g_0, g_1, \cdots$ are functions independent of $\tau$. Substituting this expansion into (148) and comparing terms of different powers of $\tau$, we can conclude that $g_0=g_0(\lambda , t)$ is independent of x and satisfies

$$\begin{aligned} \begin{aligned}&\partial _t g_0 + {\mathcal {L}}_1 g_1 + f\cdot \nabla _\lambda g_0 - \beta (f\cdot \nabla _\lambda V)g_0 = 0\,, \quad 0 \le t < T\\&g_0(\cdot , T) = 1\,. \end{aligned} \end{aligned}$$

(149)

Taking the expectation with respect to $\mu _{\lambda }$ on both sides of (149) and noticing that ${\mathbf {E}}_{\mu _\lambda }({\mathcal {L}}_1 g_1) = 0$, we obtain

$$\begin{aligned} \begin{aligned}&\partial _t g_0 + f\cdot \nabla _\lambda g_0 - \beta \big (f \cdot {\mathbf {E}}_{\mu _{\lambda }}(\nabla _\lambda V)\big ) g_0 = 0\,, \quad 0 \le t < T\\&g_0(\cdot , T) = 1\,. \end{aligned} \end{aligned}$$

(150)

It is easy to verify that the solution of (150) is given by

$$\begin{aligned} g_0(\lambda , t) = e^{-\beta \int _t^T \big ({\mathbf {E}}_{\mu _{\lambda (s)}} (\nabla _{\lambda } V)\big ) \cdot f(\lambda (s), s)\, ds}\,, \end{aligned}$$

(151)

where $\lambda (s)$ satisfies (146) with initial value $\lambda (t) = \lambda$. Taking the limit $\tau \rightarrow 0$ in (147) then yields

$$\begin{aligned} e^{-\beta \Delta F(T)} = \lim _{\tau \rightarrow 0} {\mathbf {E}}_{\mu (\lambda (0))} \Big (g(\cdot , \lambda (0), 0)\Big ) = g_0(\lambda (0), 0) = e^{-\beta \int _0^T \big ({\mathbf {E}}_{\mu _{\lambda (s)}} (\nabla _{\lambda } V)\big ) \cdot f(\lambda (s), s)\, ds}\,, \end{aligned}$$

(152)

which is equivalent to the thermodynamic integration identity (144).

Adiabatic switching Now we turn to another (equivalent) asymptotic regime where the protocol $\lambda (\cdot )$ is switched infinitely slowly. Specifically, given $\lambda _0, \lambda _1 \in {\mathbb {R}}^m$, the protocol $\lambda (\cdot )$ satisfying $\lambda (0) = \lambda _0$ and $\lambda (T) = \lambda _1$ as $T\rightarrow +\infty$ is called adiabatic switching. For the nonequilibrium process $x(\cdot )$ in (1) under adiabatic switching, it is well known that we have

$$\begin{aligned} F(\lambda _1) - F(\lambda _0)&= \lim _{T\rightarrow +\infty } {\mathbf {E}}_{\lambda _0, 0} \big (W(T)\big ) \nonumber \\&= \lim _{T\rightarrow +\infty } {\mathbf {E}}_{\lambda _0, 0} \left( \int _0^T \nabla _\lambda V(x(s), \lambda (s)) \cdot f(\lambda (s), s)\, ds\right) \,, \end{aligned}$$

(153)

i.e., the free energy difference equals to the average work performed during the switching. In the following we provide a formal mathematical argument to derive the above identity. For this purpose, we define

$$\begin{aligned} u(x,\lambda , t) = {\mathbf {E}}\left( \int _t^T \nabla _\lambda V(x(s), \lambda (s)) \cdot f(\lambda (s), s)\, ds~\Big |~x(t) = x, \lambda (t) = \lambda \right) \,, \end{aligned}$$

(154)

which, by the Feynman–Kac formula, satisfies

$$\begin{aligned} \begin{aligned}&\partial _t u + {\mathcal {L}}_1 u + f\cdot \nabla _\lambda u + f\cdot \nabla _\lambda V = 0\,, \\&u(\cdot , \cdot , T) = 0\,. \end{aligned} \end{aligned}$$

(155)

Notice that, as $T\rightarrow +\infty$, the switching becomes infinitely slow and ${\dot{\lambda }}(t) = f$ goes to zero. Instead, we rescale the time by ${\bar{t}} = \frac{t}{T} \in [0, 1]$ and define ${\bar{\lambda }}({\bar{t}}\,) = \lambda (\frac{{\bar{t}}}{\tau })$, where $\tau = \frac{1}{T} \rightarrow 0$. ${\bar{\lambda }}(\cdot )$ satisfies ${\bar{\lambda }}(0) = \lambda _0, {\bar{\lambda }}(1) = \lambda _1$ and

$$\begin{aligned} \frac{d{\bar{\lambda }}}{d{\bar{t}}} = {\bar{f}}({\bar{\lambda }}({\bar{t}}\,), {\bar{t}}\,)\,, \end{aligned}$$

(156)

where ${\bar{f}}(\cdot , {\bar{t}}\,) = \frac{1}{\tau }f(\cdot , \frac{{\bar{t}}}{\tau })$ is a function of ${\mathcal {O}}(1)$. Under this time scaling, PDE (155) becomes

$$\begin{aligned} \begin{aligned}&\partial _{{\bar{t}}} u + \frac{1}{\tau } {\mathcal {L}}_1 u + {\bar{f}}\cdot \nabla _\lambda u + {\bar{f}}\cdot \nabla _\lambda V = 0\,, \quad 0 \le {\bar{t}} < 1\,,\\&u \equiv 0\,, \quad {\bar{t}} = 1\,. \end{aligned} \end{aligned}$$

(157)

Consider the expansion $u = u_0 + \tau u_1 + \tau ^2 u_2 + \cdots$, then the same argument as above yields that the function $u_0$ is independent of x and satisfies

$$\begin{aligned} \begin{aligned}&\partial _{{\bar{t}}} u_0 + {\bar{f}}\cdot \nabla _\lambda u_0 + {\bar{f}}\cdot {\mathbf {E}}_{\lambda } \big (\nabla _\lambda V\big ) = 0\,, \quad 0 \le {\bar{t}} < 1\,,\\&u_0 \equiv 0\,, \quad {\bar{t}} = 1\,. \end{aligned} \end{aligned}$$

(158)

The solution of (158) can be directly computed:

$$\begin{aligned} u_0(\lambda , {\bar{t}}\,) = \int _{{\bar{t}}}^1 {\mathbf {E}}_{{\bar{\lambda }}(s)} \big (\nabla _\lambda V\big )\cdot {\bar{f}}({\bar{\lambda }}(s), s)\,ds \,, \end{aligned}$$

(159)

where ${\bar{\lambda }}(\cdot )$ satisfies (156) on $[{\bar{t}}, 1]$ with ${\bar{\lambda }}({\bar{t}}\,) = \lambda$. In particular, taking ${\bar{t}} = 0$ and applying the thermodynamic integration identity (144), gives

$$\begin{aligned} u_0(\lambda _0, 0) = \int _{0}^1 {\mathbf {E}}_{{\bar{\lambda }}(s)} \big (\nabla _\lambda V\big )\cdot {\bar{f}}({\bar{\lambda }}(s), s)\,ds = F(\lambda _1) - F(\lambda _0)\,. \end{aligned}$$

(160)

Therefore,

$$\begin{aligned}&\lim _{T\rightarrow +\infty } {\mathbf {E}}_{\lambda _0, 0} \left( \int _0^T \nabla _\lambda V(x, \lambda (s)) \cdot f(\lambda (s), s)\, ds\right) \\&\quad = \lim _{\tau \rightarrow 0} {\mathbf {E}}_{\mu _{\lambda _0}} \big (u(\cdot , \lambda _0, 0)\big )\\&\quad = u_0(\lambda _0, 0) = F(\lambda _1) - F(\lambda _0)\,, \end{aligned}$$

which concludes the proof of (153).

Appendix B: Thermodynamic Integration Identity in the Reaction Coordinate Case

In the reaction coordinate case considered in Sect. 3, connections between the thermodynamic integration identity and the Jarzynski’s equality as well as the adiabatic switching regime can be studied using the same asymptotic argument as in Appendix A. In this section, we omit the derivation and only provide the thermodynamic integration identity. We emphasize that both the identity and its proof can be found in the literature, e.g., [42, 44]. The result is included for readers’ convenience.

Recall the definition of the probability measure $\mu _z$ in (86), where the normalization constant is given by

$$\begin{aligned} Q(z) = \int _{{\mathbb {R}}^n} e^{-\beta V(y)} \delta \big (\xi (y) - z\big )\,dy \,,\quad z \in {\mathbb {R}}^d\,, \end{aligned}$$

(161)

and the free energy is defined in (87). Let $z(s) \in {\mathbb {R}}^d$ satisfy the ODE (101) on [0, T]. Similar to the derivations in (144), and using Lemma 3 below, we can compute

$$\begin{aligned} \begin{aligned}&F(z(T)) - F(z(0)) \\&\quad = -\beta ^{-1} \ln \frac{Q(z(T))}{Q(z(0))}\\&\quad = -\beta ^{-1} \int _0^T \frac{d}{ds}\Big ( \ln \frac{Q(z(s))}{Q(z(0))}\Big )\, ds\\&\quad = -\beta ^{-1} \int _0^T \Big (\frac{1}{Q}\frac{\partial Q}{\partial z_{\gamma }}\Big )\big (z(s)\big )\,{\dot{z}}_{\gamma }(s)\, ds\\&\quad = \int _0^T {\mathbf {E}}_{\mu _{z(s)}}\Big [(a\nabla \xi _{\gamma '})_i (\Psi ^{-1})_{\gamma '\gamma } \frac{\partial V}{\partial y_i} - \frac{1}{\beta } \frac{\partial }{\partial y_i}\Big ( (a\nabla \xi _{\gamma '})_i (\Psi ^{-1})_{\gamma \gamma '}\Big )\Big ]\, {\dot{z}}_\gamma (s)\,ds\,, \end{aligned} \end{aligned}$$

(162)

where Einstein’s summation convention has been used.

Lemma 3

Let the function Q be defined in (161). For $1 \le \gamma \le d$, we have

$$\begin{aligned} \frac{\partial Q}{\partial z_{\gamma }}(z) = -\beta Q(z) \int _{\Sigma _z} \Big [(a\nabla \xi _{\gamma '})_i (\Psi ^{-1})_{\gamma \gamma '} \frac{\partial V}{\partial y_i} - \frac{1}{\beta } \frac{\partial }{\partial y_i}\Big ( (a\nabla \xi _{\gamma '})_i (\Psi ^{-1})_{\gamma \gamma '}\Big )\Big ]\,\mu _z(dy)\,. \end{aligned}$$

Proof

Let $\varphi : {\mathbb {R}}^d \rightarrow {\mathbb {R}}$ be a smooth test function with compact support. For $1 \le \gamma \le d$, integrating by parts and using (161), we have

$$\begin{aligned} \int _{{\mathbb {R}}^d} \varphi (z) \frac{\partial Q}{\partial z_{\gamma }}(z) \, dz = -\int _{{\mathbb {R}}^d} \frac{\partial \varphi }{\partial z_{\gamma }}(z) Q(z)\, dz = -\int _{{\mathbb {R}}^n} \frac{\partial \varphi }{\partial z_{\gamma }}(\xi (y)) e^{-\beta V(y)}\, dy \,. \end{aligned}$$

(163)

On the other hand, from the relation

$$\begin{aligned} \frac{\partial \big (\varphi \circ \xi \big )}{\partial y_i}(y) = \frac{\partial \varphi }{\partial z_{\gamma '}}(\xi (y))\frac{\partial \xi _{\gamma '}}{\partial y_i}(y), \quad 1 \le i \le n\,, \end{aligned}$$

and the definition of the $d \times d$ matrix $\Psi$ in (88), we obtain

$$\begin{aligned} \frac{\partial \varphi }{\partial z_{\gamma }}(\xi (y)) = \Big [\frac{\partial \big (\varphi \circ \xi \big )}{\partial y_i} a_{ij} \frac{\partial \xi _{\gamma '}}{\partial y_j} (\Psi ^{-1})_{\gamma \gamma '}\,\Big ](y)\,. \end{aligned}$$

(164)

Therefore, integrating by parts, (163) simplifies to

$$\begin{aligned}&\int _{{\mathbb {R}}^d} \varphi (z) \frac{\partial Q}{\partial z_{\gamma }}(z) \, dz \\&\quad = \int _{{\mathbb {R}}^n} \varphi (\xi (y)) \frac{\partial }{\partial y_i}\Big (a_{ij} \frac{\partial \xi _{\gamma '}}{\partial y_j} (\Psi ^{-1})_{\gamma \gamma '} e^{-\beta V(y)}\Big )\, dy \, \\&\quad = \int _{{\mathbb {R}}^d} \varphi (z) \Big [ \int _{{\mathbb {R}}^n} \frac{\partial }{\partial y_i}\Big (a_{ij} \frac{\partial \xi _{\gamma '}}{\partial y_j} (\Psi ^{-1})_{\gamma \gamma '} e^{-\beta V(y)}\Big )\, \delta (\xi (y) - z)dy \Big ] dz\, , \end{aligned}$$

from which we can conclude after simplification. $\square$

Appendix C: An Alternative Proof of Theorem 2

In this appendix, we provide an alternative proof of Theorem 2. Different from the proof in Sect. 2.3 where only the Feynman–Kac formula has been used, the proof below relies on the combination of both the Feynman–Kac formula and Girsanov’s Theorem. While the idea is inspired by the derivations in [10], the proof below is shorter.

Alternative proof of Theorem 2

First of all, we recall the definition of u in (43) as well as the equations (40), (44), (45) used in the proof of Theorem 2 in Sect. 2.3. In accordance with (45), we define

$$\begin{aligned} \overline{{\mathcal {L}}} = \Big (J + a\nabla V + \frac{1}{\beta } \nabla \cdot a\Big ) \cdot \nabla + \frac{1}{\beta } a : \nabla ^2 + f \cdot \nabla _\lambda + \epsilon \,\alpha \alpha ^T:\nabla ^2_\lambda \,, \end{aligned}$$

(165)

and consider the function $\omega (x, \lambda ,t) = u\big ( x, \lambda , T-t\,;x',\lambda ',t'\big )$. From (44) and (45), we know that $\omega$ satisfies

$$\begin{aligned} \begin{aligned}&\frac{\partial \omega }{\partial t} + \overline{{\mathcal {L}}}_{(x,\lambda ,t)} \omega + \Big [\text{ div }(J + a\nabla V) + \text{ div }_\lambda \Big (f -\epsilon \nabla _\lambda \cdot (\alpha \alpha ^T)\Big ) + \eta \Big ]\omega = 0\,, \quad \forall t \in [0, T-t')\,,\\&\omega (x,\lambda ,t) = \delta (x'-x)\delta (\lambda '-\lambda )\,,\quad t = T-t'\,, \end{aligned}\nonumber \\ \end{aligned}$$

(166)

where $(x,\lambda ) \in {\mathbb {R}}^n \times {\mathbb {R}}^m$ and $\overline{{\mathcal {L}}}_{(x,\lambda ,t)}$ is the operator (165) evaluated at $(x,\lambda ,t)$. On the other hand, applying the Feynman–Kac formula to (166), we observe that

$$\begin{aligned} \begin{aligned}&\omega (x, \lambda ,t) \\&\quad = \overline{{\mathbf {E}}}_{x,\lambda ,t}\bigg [\exp \bigg (\int _{t}^{T-t'} \Big (\text{ div }\big (J + a\nabla V\big ) + \text{ div }_\lambda \big (f -\epsilon \nabla _\lambda \cdot (\alpha \alpha ^T)\big ) + \eta \Big )\big ({\bar{x}}(s), {\bar{\lambda }}(s),s\big ) ds\bigg )\\&\qquad \times \delta \big ({\bar{x}}(T-t')-x'\big )\delta \big ({\bar{\lambda }}(T-t') -\lambda '\big )\bigg ]\,, \end{aligned} \end{aligned}$$

(167)

where $\overline{{\mathbf {E}}}_{x,\lambda ,t}$ denotes the conditional expectation under the path ensemble of the dynamics

$$\begin{aligned} d {\bar{x}}(s)&= \Big (J+a\nabla V + \frac{1}{\beta }\nabla \cdot a\Big )\big ({\bar{x}}(s), {\bar{\lambda }}(s)\big )\, ds + \sqrt{2\beta ^{-1}} \sigma \big ({\bar{x}}(s), {\bar{\lambda }}(s)\big )\,dw^{(1)}(s) \end{aligned}$$

(168)

and the control protocol

$$\begin{aligned} d{\bar{\lambda }}(s)&= f({\bar{x}}(s), {\bar{\lambda }}(s), s)\, ds + \sqrt{2\epsilon }\, \alpha \big ({\bar{x}}(s), {\bar{\lambda }}(s),s\big ) dw^{(2)}(s)\,, \end{aligned}$$

(169)

starting from ${\bar{x}}(t) = x$ and ${\bar{\lambda }}(t) = \lambda$ at time t. Note that the infinitesimal generator of the dynamics (168) and (169) is given by the operator $\overline{{\mathcal {L}}}$ in (165).

Now we apply Girsanov’s theorem to change the probability measure in (167) from the path ensemble of the dynamics (168), (169) to the path ensemble of the dynamics (15), (3). Specifically, starting from $(x, \lambda )$ at time t, let ${\mathbf {P}}_{x, \lambda }$ and $\overline{{\mathbf {P}}}_{x,\lambda }$ denote the path measures on the time interval $[t,T-t']$ corresponding to (15), (3) and (168), (169), respectively. Applying Girsanov’s theorem, we obtain after some straightforward calculations

$$\begin{aligned} \begin{aligned} \frac{d{\mathbf {P}}_{x,\lambda }}{d \overline{{\mathbf {P}}}_{x,\lambda }} \big (x(\cdot ), \lambda (\cdot )\big )&= \exp \bigg [-\beta \int _{t}^{T-t'} \nabla V\big (x(s), \lambda (s)\big ) \cdot dx(s) \\&\quad + \beta \int _{t}^{T-t'} \Big (\nabla V \cdot \big (J + \frac{1}{\beta }\nabla \cdot a\big )\Big )\big (x(s),\lambda (s)\big )\, ds \bigg ]\,. \end{aligned} \end{aligned}$$

(170)

Therefore, changing the probability measure in (167) from $\overline{{\mathbf {P}}}_{x,\lambda }$ to ${\mathbf {P}}_{x,\lambda }$, using (170), (13), we find

$$\begin{aligned}&u( x, \lambda ,T-t) = \omega ( x, \lambda ,t) \\&\quad = {\mathbf {E}}_{x,\lambda ,t}\bigg [\exp \bigg (\int _{t}^{T-t'} \Big (\text{ div }\big (J + a\nabla V\big ) + \text{ div }_\lambda \big (f -\epsilon \nabla _\lambda \cdot (\alpha \alpha ^T)\big ) + \eta \Big )\big (x(s), \lambda (s),s\big ) ds\bigg ) \\&\qquad \times \delta \big (x(T-t')-x'\big ) \delta \big (\lambda (T-t')-\lambda '\big ) \frac{d\overline{{\mathbf {P}}}_{x,\lambda }}{d{\mathbf {P}}_{x,\lambda }} \big (x(\cdot ), \lambda (\cdot )\big ) \bigg ] \\&\quad = {\mathbf {E}}_{x,\lambda ,t}\bigg [\exp \bigg ( \beta \int _{t}^{T-t'} \nabla V(x(s), \lambda (s)) \cdot dx(s) + \int _{t}^{T-t'} \big (a : \nabla ^2 V\big )\big (x(s), \lambda (s)\big ) ds\\&\qquad + \int _{t}^{T-t'} \Big (\text{ div }_\lambda \big (f -\epsilon \nabla _\lambda \cdot (\alpha \alpha ^T)\big ) + \eta \Big ) \big (x(s), \lambda (s),s\big ) ds\bigg ) \delta \big (x(T-t')-x'\big )\,\delta \big (\lambda (T-t')-\lambda '\big ) \bigg ]\\&\quad = {\mathbf {E}}_{x,\lambda ,t}\bigg [\exp \bigg ( \beta \int _{t}^{T-t'} \nabla V\big (x(s), \lambda (s)\big ) \circ dx(s)\\&\quad \quad + \int _{t}^{T-t'} \Big (\text{ div }_\lambda \big (f -\epsilon \nabla _\lambda \cdot (\alpha \alpha ^T)\big ) + \eta \Big ) \big (x(s), \lambda (s),s\big ) ds\bigg )\\&\qquad \times \delta \big (x(T-t')-x'\big )\,\delta \big (\lambda (T-t')-\lambda '\big ) \bigg ]\,. \end{aligned}$$

Note that in the last equality above, we have converted Ito integration to Stratonovich integration according to (15). Substituting t by $T-t$, integrating by parts, and recalling the expression (43), we obtain

$$\begin{aligned} \begin{aligned}&e^{-\beta V(x',\lambda ')}\, {\mathbf {E}}^R_{x',\lambda ',t'}\bigg [ \exp \bigg (\int _{t'}^t \eta (x^R(s), \lambda ^R(s), T-s) ds\bigg ) \delta \big (x^R(t)-x\big )\delta \big (\lambda ^{R}(t)-\lambda \big )\bigg ]\\&\quad =e^{-\beta V(x,\lambda )}\,{\mathbf {E}}_{x,\lambda ,T-t}\\&\qquad \times \bigg [e^{-\beta {\mathcal {W}}} \exp \bigg (\int _{T-t}^{T-t'} \eta (x(s), \lambda (s), s) ds\bigg ) \delta \big (x(T-t')-x'\big )\delta \big (\lambda (T-t')-\lambda '\big ) \bigg ]\,, \end{aligned} \end{aligned}$$

where ${\mathcal {W}}$ is defined in (42). $\square$

Appendix D: Proof of Theorem 3

In this appendix, we provide the proof of Theorem 3 in Sect. 3.2.

Proof of Theorem 3

We consider the quantities on both sides of the equality (98). For the left hand side of (98), let us fix $(y',t') \in {\mathbb {R}}^n \times [0,T]$ and define the function u by

$$\begin{aligned} u\big (y,t\,;y',t'\big ) = {\mathbf {E}}_{y',t'}^R\bigg [ \exp \bigg (\int _{t'}^t \eta \big (y^R(s), T-s\big ) ds\bigg ) \delta \big (y^R(t)-y\big )\bigg ]\,, \end{aligned}$$

(171)

for $(y,t) \in {\mathbb {R}}^n \times [t',T]$. It is known that u satisfies the PDE

$$\begin{aligned} \begin{aligned}&\frac{\partial u}{\partial t} = \big ({\mathcal {L}}^R\big )^* u + \eta (y,T-t) \,u \,, \quad \forall ~(y, t) \in {\mathbb {R}}^n\times (t',T] \,,\\&u(y, t\,;y',t')=\delta (y-y')\,, \quad \text{ if }~~t=t'\,, \end{aligned} \end{aligned}$$

(172)

where the operator ${\mathcal {L}}^R$ is defined in (97) and $\big ({\mathcal {L}}^R\big )^*$ denotes its formal $L^2$ adjoint. A direct calculation shows that

$$\begin{aligned} \begin{aligned} \big ({\mathcal {L}}^R\big )^*\phi&= \bigg [\frac{\partial }{\partial y_i}\Big ((Pa)_{ij} \frac{\partial V}{\partial y_j}\Big )+ \frac{\partial }{\partial y_i} \Big ((\Psi ^{-1})_{\gamma \gamma '} (a\nabla \xi _\gamma )_i f^{-}_{\gamma '}\Big ) \bigg ]\phi \\&\quad + \bigg [ (Pa)_{ij} \frac{\partial V}{\partial y_j} + \frac{1}{\beta } \frac{\partial (Pa)_{ij}}{\partial y_j} + (\Psi ^{-1})_{\gamma \gamma '} (a\nabla \xi _\gamma )_i f^{-}_{\gamma '} \bigg ] \frac{\partial \phi }{\partial y_i} + \frac{1}{\beta } (Pa)_{ij} \frac{\partial ^2 \phi }{\partial y_i \partial y_j} \,, \end{aligned} \end{aligned}$$

(173)

for a smooth function $\phi$.

For the right hand side of (98), fixing $(y', t') \in {\mathbb {R}}^n \times [0, T]$, we define the function g for $(y,t) \in {\mathbb {R}}^n \times [t',T]$ as

$$\begin{aligned} g(y,t) = {\mathbf {E}}_{y,T-t}\bigg [&e^{-\beta {\mathcal {W}}} \exp \bigg (\int _{T-t}^{T-t'} \eta \big (y(s), s\big ) ds\bigg ) \delta \big (y(T-t')-y'\big ) \bigg ]\,, \end{aligned}$$

where ${\mathcal {W}}$ is defined in (99), and the dynamics $y(\cdot )$ satisfies the SDE (93). Using the same argument as in Lemma 1, we can verify that g satisfies the PDE

$$\begin{aligned} \begin{aligned}&\frac{\partial g}{\partial t} = \overline{{\mathcal {L}}}\, g + \eta (\cdot , T-t) g \,,\qquad \forall \, (y,t) \in {\mathbb {R}}^n \times (t',T] \,,\\&g(y,t) = \delta (y-y') \,, \qquad \text{ if }~~t=t'\,, \end{aligned} \end{aligned}$$

(174)

where the operator $\overline{{\mathcal {L}}}$ is defined as

$$\begin{aligned} \begin{aligned} \overline{{\mathcal {L}}}\,\phi&= \bigg [-\beta (\Psi ^{-1})_{\gamma \gamma '} (a\nabla \xi _\gamma )_i f^-_{\gamma '} \frac{\partial V}{\partial y_i} + \frac{\partial }{\partial y_i} \Big ((\Psi ^{-1})_{\gamma \gamma '} (a\nabla \xi _\gamma )_i f^-_{\gamma '}\Big )\bigg ] \phi \\&\quad + {\mathcal {L}}^{\perp } \phi + (\Psi ^{-1})_{\gamma \gamma '} (a\nabla \xi _\gamma )_if^-_{\gamma '} \frac{\partial \phi }{\partial y_i} \end{aligned} \end{aligned}$$

(175)

for a smooth function $\phi$. Now consider the function $\omega (y,t) = e^{-\beta V(y)} g(y,t)$. A direct calculation shows that

$$\begin{aligned} \begin{aligned} e^{-\beta V} {\mathcal {L}}^{\perp } g&= e^{-\beta V} \bigg [-(Pa)_{ij} \frac{\partial V}{\partial y_j}\frac{\partial \big (e^{\beta V}\omega \big )}{\partial y_i} + \frac{1}{\beta } \frac{\partial (Pa)_{ij}}{\partial y_j} \frac{\partial \big (e^{\beta V}\omega \big )}{\partial y_i} + \frac{1}{\beta } (Pa)_{ij} \frac{\partial ^2 \big (e^{\beta V}\omega \big )}{\partial y_i\partial y_j}\bigg ]\,\\&= \bigg [\frac{\partial }{\partial y_i}\Big ((Pa)_{ij} \frac{\partial V}{\partial y_j}\Big )\bigg ]\omega + \bigg [(Pa)_{ij} \frac{\partial V}{\partial y_j} + \frac{1}{\beta } \frac{\partial (Pa)_{ij}}{\partial y_j}\bigg ] \frac{\partial \omega }{\partial y_i} + \frac{1}{\beta } (Pa)_{ij} \frac{\partial ^2 \omega }{\partial y_i\partial y_j}\,, \\ e^{-\beta V} \frac{\partial g}{\partial y_i}&= e^{-\beta V} \frac{\partial \big (e^{\beta V} \omega \big )}{\partial y_i} = \beta \frac{\partial V}{\partial y_i}\omega + \frac{\partial \omega }{\partial y_i} \,. \end{aligned} \end{aligned}$$

(176)

Combining (97), (174), (175), (176), it follows that the function $\omega$ satisfies the PDE

$$\begin{aligned}&\frac{\partial \omega }{\partial t} = e^{-\beta V} \Big [\overline{{\mathcal {L}}}\,g + \eta (\cdot , T-t) g\Big ]= \big ({\mathcal {L}}^R\big )^* \,\omega + \eta (y,T-t)\,\omega \,,\quad \forall \,(y, t) \in {\mathbb {R}}^n \times (t',T] \,,\\&\omega (y,t) = e^{-\beta V(y')} \delta (y-y')\,,\quad \text{ if }~~ t=t'\,. \end{aligned}$$

Comparing this with the equation of function u in (172), we obtain

$$\begin{aligned} e^{-\beta V(y')} u(y,t\,;y',t') = \omega (y,t), \end{aligned}$$

which is equivalent to (98). $\square$