Communication Reducing Diffusion LMS Robust to Impulsive Noise Using Smart Selection of Communication Nodes

Abstract

The paper proposes a smart method to remove network nodes that are highly prone to impulse noise and replacing them with a linear combination of existing more reliable nodes. This is done to reduce communication cost. To reduce the communication cost in a network, a common way is to remove some nodes randomly and then replace the intermediate estimation of these nodes by the corresponding node estimation. In this direction, the contribution of this paper is twofold. First, we suggest to remove the nodes smartly by omitting the unreliable nodes prone to impulsive noise. This is done by computing the disturbance induced in the adaptation step of the diffusion least mean square. Second, we replace the estimation of removing nodes by a linear combination of existing estimations of reliable nodes instead of just replacing by the estimation of the corresponding node. Also, the coefficients of linear combination are optimized based on the minimum disturbance principle. Furthermore, the minimum achievable disturbance is calculated mathematically and a necessary and sufficient condition for stability of the proposed algorithm is presented. Finally, the simulation results show the efficiency of the proposed method in comparison with some state-of-the-art algorithms.

Appendices

Appendix A: Optimizing the Coefficients of $h_r$

The disturbance incurred by the combination step is as

$$\begin{aligned} D=||\varvec{\omega }_{k,i+1}-\varvec{\omega }_{k,i}||^2_2=\left| \left| \sum _{l\in \mathcal N_\mathrm {k}}c_{lk}\tilde{{{\varvec{\varphi }}}}_{l,i+1}-\varvec{\omega }_{k,i}\right| \right| ^2_2. \end{aligned}$$

(16)

In the following, for the brevity, we omit the index k, i and $i+1$ in somewhere. Hence, we have

$$\begin{aligned} D= & {} \left| \left| \sum _{l=1}^Lc_{lk}\tilde{{{\varvec{\varphi }}}}_{l}+\sum _{l=L+1}^d c_{lk}{{\varvec{\varphi }}}_{l}-\varvec{\omega }_{k}\right| \right| ^2_2 \nonumber \\= & {} \left| \left| \sum _{l=1}^Lc_{lk}(\sum _{r=L+1}^d h_r{{\varvec{\varphi }}}_r)+\sum _{l=L+1}^d c_{lk}{{\varvec{\varphi }}}_{l}-\varvec{\omega }_{k}\right| \right| ^2_2. \end{aligned}$$

(17)

where $h_{rk}$ is replaced by $h_r$ for simplicity. Then, some simple manipulation results to the following formula for the disturbance:

$$\begin{aligned} D=\left| \left| \sum _{r=L+1}^d(h_r\beta _k+c_{lk}){{\varvec{\varphi }}}_r-\varvec{\omega }_{k}\right| \right| ^2_2, \end{aligned}$$

(18)

where $\beta _k\triangleq \sum _{l=1}^L c_{lk}$. By expanding the above formula, we can write

$$\begin{aligned} D=\left( \sum _{r=L+1}^d(h_r\beta _k+c_{rk}){{\varvec{\varphi }}}_r-\varvec{\omega }_{k}\right) ^T\left( \sum _{r^{'}=L+1}^d(h_{r^{'}}\beta _k+c_{r^{'}k}){{\varvec{\varphi }}}_{r^{'}}-\varvec{\omega }_{k}\right) , \end{aligned}$$

(19)

which results to the sum of four terms as follows:

$$\begin{aligned} D= & {} \sum _{r=L+1}^d\sum _{r^{'}=L+1}^d(h_r\beta _k+c_{rk}){{\varvec{\varphi }}}^T_{r}{{\varvec{\varphi }}}_{r^{'}}(h_{r^{'}}\beta _k+c_{r^{'}k}) \nonumber \\&\quad -\, \varvec{\omega }_{k}^{T}\left( \sum _{r^{'}=L+1}^d(h_{r^{'}}\beta _k+c_{r^{'}k}){{\varvec{\varphi }}}_{r^{'}}\right) -\left( \sum _{r=L+1}^d(h_r\beta _k+c_{rk}){{\varvec{\varphi }}}^T_{r}\right) \varvec{\omega }_k+||\varvec{\omega }_k||_2^2.\nonumber \\ \end{aligned}$$

(20)

Since $||\varvec{\omega }_k||_2^2$ is constant with respect to the coefficients $h_r$, minimizing the disturbance is equivalent to minimizing the following expression:

$$\begin{aligned} \tilde{D}= & {} \sum _{r=L+1}^d\sum _{r^{'}=L+1}^d(h_r\beta _k+c_{rk})(h_{r^{'}}\beta _k+c_{r^{'}k})\gamma _{r,r^{'}} \nonumber \\&\quad -\, 2\varvec{\omega }_{k}^{T}\Big (\sum _{r=L+1}^d(h_{r}\beta _k+c_{rk}){{\varvec{\varphi }}}_{r}\Big ), \end{aligned}$$

(21)

where $\gamma _{r,r^{'}}\triangleq {{\varvec{\varphi }}}^T_{r}{{\varvec{\varphi }}}_{r^{'}}$. To minimize $\tilde{D}$ with respect to $h_r$, taking the partial derivative and make this equal to zero. So, with $\frac{\partial \tilde{D}}{\partial h_p}=0$, we reach to the following linear equations:

$$\begin{aligned} \sum _{r=L+1}^d2\beta _k(h_{r}\beta _k+c_{rk})\gamma _{p,r}+2(h_{p}\beta _k+c_{pk})\beta _k\gamma _{r,r}-2\varvec{\omega }_k^T\beta _k{{\varvec{\varphi }}}_p=0. \end{aligned}$$

(22)

The above linear equations with respect to the coefficients, $h_p$ for $L_k+1\le p\le d_k$, can be written in a matrix format as in (7).

Appendix B: Calculating the Necessary and Sufficient Condition of Stability

If we define the disturbance difference as $\Delta \triangleq D_{i+1}-D_i$, then it would be equal to

$$\begin{aligned} \Delta =\tilde{\varvec{\omega }}^T_{k,i+1}\mathbf {B}_{i+1}\tilde{\varvec{\omega }}_{k,i+1}-\tilde{\varvec{\omega }}^T_{k,i}\mathbf {B}_i\tilde{\varvec{\omega }}_{k,i}. \end{aligned}$$

(23)

To guarantee the stability of the proposed algorithm, the disturbance difference should be non-positive $\Delta \le 0$. To ensure this, we expand and simplify the expression of $\Delta$. To do so, we write

$$\begin{aligned}&\tilde{\varvec{\omega }}_{k,i+1}=\varvec{\omega }_{k,i+1}-\sum _{l}c_{lk}{{\varvec{\varphi }}}_{l,i+1}, \end{aligned}$$

(24)

$$\begin{aligned}&\tilde{\varvec{\omega }}_{k,i}=\varvec{\omega }_{k,i}-\sum _{l}c_{lk}{{\varvec{\varphi }}}_{l,i}. \end{aligned}$$

(25)

If we compute the difference of (24) and (25), with some simple simplification, we have

$$\begin{aligned} \tilde{\varvec{\omega }}_{k,i+1}-\tilde{\varvec{\omega }}_{k,i}=-{{\varvec{\varphi }}}_{k,i+1}+\sum _lc_{lk}{{\varvec{\varphi }}}_{l,i}. \end{aligned}$$

(26)

Therefore, we have $\tilde{\varvec{\omega }}_{k,i+1}=\tilde{\varvec{\omega }}_{k,i}-{{\varvec{\varphi }}}_{k,i+1}+\sum _lc_{lk}{{\varvec{\varphi }}}_{l,i}$. Defining $\mathbf {f}_{k,i}\triangleq {{\varvec{\varphi }}}_{k,i+1}-\sum _lc_{lk}{{\varvec{\varphi }}}_{l,i}$, we have $\tilde{\varvec{\omega }}_{k,i+1}=\tilde{\varvec{\omega }}_{k,i}-\mathbf {f}_{k,i}$. Replacing this into (23), we have

$$\begin{aligned} \Delta =(\tilde{\varvec{\omega }}_{k,i+1}-\mathbf {f}_{k,i})^T\mathbf {B}_{i+1}(\tilde{\varvec{\omega }}_{k,i+1}-\mathbf {f}_{k,i})-\tilde{\varvec{\omega }}^T_{k,i}\mathbf {B}_i\tilde{\varvec{\omega }}_{k,i}. \end{aligned}$$

(27)

By some simple manipulations in the above formula, we reach to the following quadratic expression:

$$\begin{aligned} \Delta =\tilde{\varvec{\omega }}_{k,i+1}\mathbf{Q }_i\tilde{\varvec{\omega }}_{k,i+1}-2\mathbf {f}^T_{k,i}\mathbf {B}_{i+1}\tilde{\varvec{\omega }}_{k,i+1}+\mathbf {f}^T_{k,i}\mathbf {B}_{i+1}\mathbf {f}_{k,i}, \end{aligned}$$

(28)

where $\mathbf{Q }_i\triangleq \mathbf {B}_{i+1}-\mathbf {B}_i=\varvec{\Phi }_i(\varvec{\Phi }_i^T\varvec{\Phi }_i)^{-1}\varvec{\Phi }_i^T-\varvec{\Phi }_{i+1}(\varvec{\Phi }_{i+1}^T\varvec{\Phi }_{i+1})^{-1}\varvec{\Phi }_{i+1}^T$. To be ensure that this difference is non-positive, a necessary condition is that the matrix $\mathbf{Q }_i\le 0$ be non-positive. In other words, the necessary condition for stability is that $\varvec{\Phi }_{i+1}(\varvec{\Phi }_{i+1}^T\varvec{\Phi }_{i+1})^{-1}\varvec{\Phi }_{i+1}^T-\varvec{\Phi }_i(\varvec{\Phi }_i^T\varvec{\Phi }_i)^{-1}\varvec{\Phi }_i^T\ge 0$ is nonnegative definite. To find the necessary and sufficient condition for stability, in addition to the necessary condition $\mathbf{Q }_i\le 0$, the maximum of the quadratic expression in (28) should be non-positive. So, taking the derivative of $\Delta$ and enforcing it to zero $\frac{\partial \Delta }{\partial \tilde{\varvec{\omega }}_{k,i}}=0$, we have the maximizing vector equal to $\tilde{\varvec{\omega }}_{k,i}=\mathbf{Q }^{-1}_i\mathbf {B}_{i+1}^T\mathbf {f}_{k,i}$. Replacing this into (28), and with some manipulations, we reach

$$\begin{aligned} \Delta _{\mathrm {max}}=\mathbf {f}^T_{k,i}\mathbf {B}_{i+1}(\mathbf{I }-\mathbf{Q }^{-1}_i\mathbf {B}^T_{i+1})\mathbf {f}_{k,i}\le 0. \end{aligned}$$

(29)

So, we should have $\mathbf {R}_i\triangleq \mathbf {B}_{i+1}(\mathbf{I }-\mathbf{Q }^{-1}_i\mathbf {B}^T_{i+1})\le 0$. Hence, the necessary and sufficient condition for stability of the proposed algorithm is that at every iteration, we have?