Stability and Synchronization of Neutral-Type Neural Networks

Abstract

When the states of a system are decided not only by states of the current time and the past time but also by the derivative of the past states, the system can be called a neutral system. The problems of stability and synchronization of neutral neural networks play an important role in the same issues of neural networks. In this chapter, robust stability of neutral neural networks is first discussed.

Appendix

Proof

The proof of (R1) is the same as [32] and omitted here. To prove (R2), we will divide it into five steps. We change $\bar{D}$ into D in subsequence for simplicity.

Step 1    To prove that the solution $x(t, i_0, \xi )$ of the system obeys

(5.292)

In fact, let

$$\begin{aligned} \begin{aligned}&M(t)\\ =&\int _{0}^{t}V_x(s, r(s), x(s)-D(x(s-\tau ), r(s)))dB(s)\\&+ \int _{0}^{t}\int _\mathbb R(V(s, i_0+\bar{h}(r(s-),l), x(s)-D(x(s-\tau ), r(s)))\\&-V(s, r(s), x(s)-D(x(s-\tau ),r(s)))\mu (ds, dl) \end{aligned} \end{aligned}$$

(5.293)

which is a continuous local martingale with $M(0)=0, \textit{a.s.}$ By generalized Itô formula (Lemma 1.10), we have

$$\begin{aligned} \begin{array}{rl} &{} V(t, i, x(t)-D(i, x(t-\tau )))\\ \le &{} V(0, i_0, x(0)-D(i, x(-\tau )))\\ &{} +\int _0^t \mathcal{L}V(s, r(s), x(s), x(s)-D(r(s), x(s-\tau )))ds\\ &{} +M(t)\\ \le &{} V(0, i_0, x(0)-D(i, x(-\tau )))\\ &{} +\int _0^t(\gamma (s)-W_1(x(s))+W_2(x(s-\tau ))\\ &{} -W_3(x(s)-D(r(s), x(s-\tau ))))ds+M(t)\\ \le &{} V(0, i_0, x(0)-D(i, x(-\tau )))\\ &{} +\int _0^t\gamma (s)ds-\int _0^tW_1(x(s))ds+\int _0^tW_2(x(s-\tau ))ds\\ &{} -\int _0^tW_3(x(s)-D(r(s), x(s-\tau )))ds+M(t)\\ = &{} V(0, i_0, x(0)-D(i, \xi (-\tau )))\\ &{} +\int _0^t\gamma (s)ds-\int _0^tW_1(x(s))ds+\int _{-\tau }^{t-\tau }W_2(x(s))ds\\ &{} -\int _0^tW_3(x(s)-D(r(s), x(s-\tau )))ds+M(t)\\ \le &{} V(0, i_0, x(0)-D(i, \xi (-\tau )))+\int _{-\tau }^{0}W_2(x(s))ds\\ &{} +\int _0^t\gamma (s)ds-\int _0^t(W_1(x(s))-W_2(x(s)))ds\\ &{} -\int _0^tW_3(x(s)-D(r(s), x(s-\tau )))ds+M(t) \end{array} \end{aligned}$$

(5.294)

By the convergence theorem of nonnegative semimartingales (Lemma 1.1), we have (5.292).

Step 2    To prove

$$\begin{aligned} \sup \limits _{0\le t<\infty }|x(t)|<\infty ~\textit{a.s.} \end{aligned}$$

(5.295)

Indeed, from (5.292), we have

$$\begin{aligned} \sup \limits _{0\le t<\infty }V(t, r(t), x(t)-D(r(t), x(t-\tau )))<\infty ~\textit{a.s.} \end{aligned}$$

which together with (C1), yields

$$\begin{aligned} \sup \limits _{0\le t<\infty }|x(t)-D(r(t), x(t-\tau ))|<\infty ~\textit{a.s.} \end{aligned}$$

(5.296)

Now, for any $T>0$, by (H2), we have that if $0\le t\le T$, then

$$\begin{aligned} \begin{array}{rl} |x(t)| &{} \le |D(r(t), x(t-\tau ))|+|x(t)-D(r(t), x(t-\tau ))|\\ &{} \le \kappa |x(t-\tau )|+|x(t)-D(r(t), x(t-\tau ))| \end{array} \end{aligned}$$

where $\kappa =\max \limits _{i\in \mathbb S}\kappa _i<1$. This implies

$$\begin{aligned} \begin{array}{rl} &{} \sup \limits _{0\le t\le T} |x(t)|\\ \le &{} \kappa \sup \limits _{0\le t\le T}|x(t-\tau )| +\sup \limits _{0\le t\le T}|x(t)-D(r(t), x(t-\tau ))|\\ = &{} \kappa \beta + \kappa \sup \limits _{0\le t\le T}|x(t)| +\sup \limits _{0\le t\le T}|x(t)-D(r(t), x(t-\tau ))| \end{array} \end{aligned}$$

where $\beta$ is the bound for the initial data $\xi$. Hence,

$$\begin{aligned} \sup \limits _{0\le t\le T} |x(t)|\le \frac{1}{1-\kappa }\left( \kappa \beta +\sup \limits _{0\le t\le T}|x(t)-D(r(t), x(t-\tau ))|\right) \end{aligned}$$

Letting $T\rightarrow \infty$ and using (5.296), we obtain (5.295).

Step 3    To prove

$$\begin{aligned} \lim \limits _{t\rightarrow \infty }W_3(x(t)-D(r(t), x(t-\tau )))=0~\textit{a.s.} \end{aligned}$$

(5.297)

In fact, taking the expectations on both side of (5.294) and letting $t\rightarrow \infty$, we obtain that

$$\begin{aligned} \mathbb {E}\int _0^\infty W(s)ds<\infty \end{aligned}$$

(5.298)

where $W(s)=W_1(x(s))-W_2(x(s))+W_3(z(s)), z(s)=x(s)-D(r(s), x(s-\tau ))$.

This implies

$$\begin{aligned} \int _0^\infty W(s)ds<\infty ~\textit{a.s.} \end{aligned}$$

(5.299)

or equivalently,

$$\begin{aligned} \int _0^\infty (W_1(x(s))-W_2(x(s)))ds<\infty ~\textit{a.s.} \end{aligned}$$

and

$$\begin{aligned} \int _0^\infty W_3(z(s))ds < \infty ~\textit{a.s.} \end{aligned}$$

From (5.299), we have

(5.300)

or equivalently,

and

Now we will prove (5.297): $\lim \limits _{t\rightarrow \infty }W_3(z(t))=0~\textit{a.s.}$ In fact, if (5.297) is false, then

Hence, there is a number $\varepsilon >0$ such that

$$\begin{aligned} \mathbb {P}(\varOmega _1)\ge 3\varepsilon \end{aligned}$$

(5.301)

where

Recall (5.295), as well as the boundedness of the initial data $\xi$, we can find a positive number h, which depends on $\varepsilon$, sufficiently large for

$$\begin{aligned} \mathbb {P}(\varOmega _2)\ge 1-\varepsilon \end{aligned}$$

(5.302)

where $\varOmega _2=\{\sup \limits _{-\tau \le t<\infty }|z(t)|<h\}$

It is easy to see from (5.301) and (5.302) that

$$\begin{aligned} \mathbb {P}(\varOmega _1 \cap \varOmega _2)\ge 2\varepsilon \end{aligned}$$

(5.303)

We now define a sequence of stopping times as follows:

$$\begin{aligned} \begin{array}{l} \tau _h=\inf \{t\ge 0: |x(t)|\wedge |z(t)|\ge h\}\\ \sigma _1=\inf \{t\ge 0: W_3(z(t))\ge 2\varepsilon \}\\ \sigma _{2k}=\{t\ge \sigma _{2k-1}: W_3(z(t))<\varepsilon \}, k=1, 2, \ldots \\ \sigma _{2k+1}=\{t\ge \sigma _{2k}: W_3(z(t))\ge 2\varepsilon \}, k=1, 2, \ldots \end{array} \end{aligned}$$

where throughout this section, we set $\inf \emptyset =\infty$.

From (5.300) and the definition of $\varOmega _1$ and $\varOmega _2$, we observe that if $\omega \in \varOmega _1 \cap \varOmega _2$, then

$$\begin{aligned} \tau _h=\infty ~\mathrm{and} ~\sigma _k<\infty ~\forall k\ge 1 \end{aligned}$$

(5.304)

Let $I_A$ denote the indication function of set A. Noting the fact that $\sigma _{2k}<\infty$, whenever $\sigma _{2k-1} < \infty$, we can derive from (5.298) that

$$\begin{aligned} \begin{array}{rl} \infty &{} > \mathbb {E}\int _0^\infty W_3(z(t))dt\\ &{} \ge \sum \limits _{k=1}^{\infty }\mathbb {E}\left[ I_{\sigma _{2k-1}<\infty , \sigma _{2k}<\infty , \tau _h=\infty }\int _{\sigma _{2k-1}}^{\sigma _{2k}} W_3(z(t))dt\right] \\ &{} \ge \varepsilon \sum \limits _{k=1}^{\infty }\mathbb {E}[I_{\sigma _{2k-1}<\infty , \tau _h=\infty }(\sigma _{2k}-\sigma _{2k-1})]\ \end{array} \end{aligned}$$

(5.305)

On the other hand, ny (H1), there exists a constant $K_h>0$, such that

$$\begin{aligned} |f(t, i, x, y)|^2 \vee |g(t, i, x, y)|^2\le K_h^2 \end{aligned}$$

whenever $|x|\vee |y|<h$ and $(t, i)\in \mathbb R_+\times \mathbb S$.

By the H$\ddot{o}$lder inequality (Lemma 1.15) and the Doob’s martingale inequality (Lemma 1.18), we compute that, for any $T>0$ and $k=1, 2, \ldots$

$$\begin{aligned} \begin{array}{rl} &{} \mathbb {E}[I_{\tau _h\wedge \sigma _{2k-1}<\infty }\sup \limits _{0\le t\le T}|z(\tau _h\wedge (\sigma _{2k-1}+t))-z(\tau _h\wedge \sigma _{2k-1})|^2\\ \le &{} 2K_h^2T(T+4) \end{array} \end{aligned}$$

(5.306)

Since $W_3(\cdot )$ is continuous in $\mathbb {R}^n$, there exists a closed ball $\bar{S}_h=\{x\in \mathbb {R}^n: |x|<h\}$ such that $W_3(\cdot )$ is uniformly continuous in $\bar{S}_h$. We can therefore choose $\delta =\delta (\varepsilon )>0$ so small such that

$$\begin{aligned} |W_3(x)-W_3(y)|<\varepsilon /2~\mathrm{whenever} ~x, y \in \bar{S}_h,~|x-y|<\delta \end{aligned}$$

(5.307)

We furthermore choose $T=T(\varepsilon , \delta , h)>0$ sufficiently small for

$$\begin{aligned} \frac{2K_h^2 T(T+4)}{\delta ^2}<\varepsilon \end{aligned}$$

It then follows from (5.306) and Chebyshev’s inequality (Lemma 1.19) that

Noting that

we hence have

By (5.303) and (5.304), we further compute

(5.308)

By (5.307), we hence obtain that

$$\begin{aligned} \begin{array}{rl} &{} \mathbb {P}\left( \{\sigma _{2k-1}<\infty , \tau _h=\infty \}\right. \\ &{} \cap \left. \left\{ \sup \limits _{0\le t\le T}|W_3(z(\sigma _{2k-1}+t))-W_3(z(\sigma _{2k-1}))|<\varepsilon \right\} \right) \\ > &{} \varepsilon \end{array} \end{aligned}$$

(5.309)

Set

$$\begin{aligned} \bar{\varOmega }_k=\left\{ \sup \limits _{0\le t\le T}|W_3(z(\sigma _{2k-1}+t))-W_3(z(\sigma _{2k-1}))|<\varepsilon \right\} \end{aligned}$$

Noting that

$$\begin{aligned} \sigma _{2k}(\omega )-\sigma _{2k-1}(\omega )\ge T~\mathrm{if}~\omega \in \{\sigma _{2k-1}<\infty , \tau _h=\infty \}\cap \bar{\varOmega }_k \end{aligned}$$

we derive from (5.305) and (5.309) that

$$\begin{aligned} \begin{array}{rl} \infty &{} > \varepsilon \sum \limits _{k=1}^{\infty }\mathbb {E}[I_{(\sigma _{2k-1}<\infty , \tau _h=\infty )}(\sigma _{2k}-\sigma _{2k-1})]\\ &{} \ge \varepsilon \sum \limits _{k=1}^{\infty }\mathbb {E}[I_{(\sigma _{2k-1}<\infty , \tau _h=\infty )\cap \bar{\varOmega }_k}(\sigma _{2k}-\sigma _{2k-1})]\\ &{} \ge \varepsilon T\sum \limits _{k=1}^{\infty }\mathbb {P}(\{\sigma _{2k-1}<\infty , \tau _h=\infty \}\cap \bar{\varOmega }_k)\\ &{} \ge \varepsilon T\sum \limits _{k=1}^{\infty }\varepsilon =\infty \end{array} \end{aligned}$$

which is a contradiction. So (5.297) must hold.

Step 4    To prove that $\mathrm{Ker} (W_3)\not = \emptyset$ and

$$\begin{aligned} \lim \limits _{t\rightarrow \infty }d(x(t; \xi , i_0)-D(x(t-\tau ; \xi , i_0), r(t)), \mathrm{Ker} (W_3))=0~\textit{a.s.} \end{aligned}$$

(5.310)

By (5.297) and (5.296), we see that there is an $\varOmega _0\subset \varOmega$ with $\mathbb {P}(\varOmega _0)=1$ such that

$$\begin{aligned} \lim \limits _{t\rightarrow \infty }W_3(z(t, \omega ))=0~\mathrm{and}~\sup \limits _{0\le t<\infty }|z(t, \omega )|<\infty ~\forall \omega \in \varOmega _0 \end{aligned}$$

(5.311)

Choose any $\omega \in \varOmega _0$. Then $\{z(t, \omega )\}_{t\ge 0}$ is bounded in $\mathbb {R}^n$, so there must be an increasing sequence $\{t_k\}_{k\ge 1}$ such that $t_k\rightarrow \infty$ and $\{z(t_k, \omega )\}_{k\ge 1}$ converges to some $\bar{z}\in \mathbb {R}^n$. Thus

$$\begin{aligned} W_3(\bar{z})=\lim \limits _{k\rightarrow \infty }W_3(z(t_k, \omega ))=0 \end{aligned}$$

which implies that $\bar{z}\in \mathrm{Ker}(W_3)$ whence $\mathrm{Ker} (W_3)\not = \emptyset$. From this, we can show that

$$\begin{aligned} \lim \limits _{t\rightarrow \infty }d(z(t, \omega ), \mathrm{Ker} (W_3))=0~\forall \omega \in \varOmega _0 \end{aligned}$$

(5.312)

If this is false, then there is some $\bar{\omega }\in \varOmega _0$ such that

Hence, there is a subsequence $\{z(t_k, \bar{\omega })\}_{k\ge 0}$ of $\{z(t, \bar{\omega })\}_{t\ge 0}$ such that

$$\begin{aligned} \lim \limits _{k\rightarrow \infty }d(z(t_k, \bar{\omega }), \mathrm{Ker} (W_3))>\bar{\varepsilon } \end{aligned}$$

for some $\bar{\varepsilon }>0$. Since $\{z(t_k, \bar{\omega })\}_{k\ge 0}$ is bounded, we can find its subsequence $\{z(\bar{t}_k, \bar{\omega })\}_{k\ge 0}$ which converges to $\hat{z}\in \mathbb {R}^n$. Clearly, $\hat{z}\not \in \mathrm{Ker} (W_3)$ so $W_3(\hat{z})>0$.But, by (5.311),

$$\begin{aligned} W_3(\hat{z})=\lim \limits _{k\rightarrow \infty }W_3(z(\bar{t}_k, \bar{\omega }))=0 \end{aligned}$$

a contradiction. Hence, (5.312) must hold and (5.310) holds yet.

Step 5    To prove (R2).

Under the assume that

$$\begin{aligned} W_3(x)=0~\mathrm{if~ and ~only ~if}~x=0 \end{aligned}$$

(5.313)

we have $\mathrm{Ker} (W_3)=\{0\}$. It then follows from (5.310) that

$$\begin{aligned} \lim \limits _{t\rightarrow 0}[x(t)-D(x(t-\tau ), r(t))]=\lim \limits _{t\rightarrow 0}z(t)=0~\textit{a.s.} \end{aligned}$$

But by (H2),

$$\begin{aligned} \begin{array}{rl} |x(t)| &{} \le |D(x(t-\tau ), r(t))|+|x(t)-D(x(t-\tau ), r(t))|\\ &{} \le \kappa |x(t-\tau )|+|x(t)-D(x(t-\tau ), r(t))| \end{array} \end{aligned}$$

where $\kappa \in (0, 1)$ has been defined above. Letting $t\rightarrow \infty$ we obtain that

This together with (5.295) yields

$$\begin{aligned} \lim \limits _{t\rightarrow \infty }|x(t)|=0~\textit{a.s.} \end{aligned}$$

which is the (5.264) and the proof is therefore completed.