A Simple and Exact Algorithm to Solve Linear Problems with $\ell ^1$-Based Regularizers

Abstract

This paper considers $\ell ^1$-based regularized signal estimation that are often used in applications. The estimated signal is obtained as the solution of an optimization problem and the quality of the recovered signal directly depends on the quality of the solver. This paper describes a simple algorithm that computes an exact minimizer of $\Vert D \cdot \Vert _1$ under the constraints $A\mathbf {x}=\mathbf {y}$. A comparative evaluation of the algorithm is presented. An illustrative application to real signals of bacterial flagellar motor is also presented.

Appendix

1.1 A.1 Notations and Definitions

(i) :

(n first integers)

(ii) :

(Vectors) Vectors of, e.g., $\mathbb {R}^n$ are denoted in bold typeface, e.g., $\mathbf {x}$. Other objects like scalars or functions are denoted in non-bold typeface. Entries of $\mathbf {x}$ are denoted, e.g., $x_i$

(iii) :

(Norms) $\Vert \mathbf {x}\Vert _1:=\sum _i |x_i|$, $\Vert \mathbf {x}\Vert _2:= \sqrt{ \sum _i |x_i|^2}$ and $\Vert \mathbf {x}\Vert _\infty := \max _i|x_i|$

(iv) :

(Canonical basis) $\mathbf {e}_i$ denotes the i-th canonical vector of $\mathbb {R}^n,$ and $\mathbf {\tilde{e}}_i$ denotes the i-th canonical vector of $\mathbb {R}^{2n}$

(v) :

(Inner product) $\langle \mathbf {x}, \mathbf {y}\rangle =\sum _i x_i y_i$

(vi) :

(Identity matrix) $I_n$ denotes de identity matrix of size $n \times n$

(vii) :

(Transpose operator) $M^\dagger$ denotes the transpose of a matrix M

(viii) :

(Column of matrix) col(M, i) denotes the i-th column of matrix M

(ix) :

(Convex function) A function $f :\mathbb {R}^n \rightarrow \mathbb {R}\cup \{+\infty \}$ is said to be convex if $\forall (\mathbf {x},\mathbf {y}) \in \mathbb {R}^n \times \mathbb {R}^n$ and $\forall \alpha \in (0,1)$ holds true (in $\mathbb {R}\cup \{+\infty \}$)

(x) :

(Set $\varGamma _0(\mathbb {R}^n)$) The set of lower semi-continuous, convex functions with $\text {dom~}f \ne \emptyset$ is denoted by $\varGamma _0(\mathbb {R}^n)$

(xi) :

(Characteristic function of a set) $\chi _{E}(\mathbf {x})=0$ if $\mathbf {x}\in E$ and $\chi _{E}(\mathbf {x})=+\infty$ otherwise

(xii) :

(Convex conjugate) For any f convex that satisfies $\text {dom~}f \ne \emptyset$, the function $f^*$ defined by $\mathbb {R}^n \ni \mathbf {s}\mapsto f^*(\mathbf {s}):=\sup _{\mathbf {x}\in \text {dom~}f}\left\{ \langle \mathbf {s},\mathbf {x}\rangle -f(\mathbf {x}) \right\}$. (See, e.g., [16, Def. 1.1.1, p. 37])