Brouwer’s Fixed Point Theorem and Sparsity

Abstract

In the generalized linear model and its relatives, the loss depends on the parameter via a transformation (the inverse link function) of the linear function (or linear predictor). In this chapter such a structure is not assumed. Moreover, the chapter hints at cases where the effective parameter space is very large and the localization arguments discussed so far cannot be applied. (The graphical Lasso is an example.) With the help of Brouwer’s fixed point theorem it is shown that when $\dot{R}(\beta )$ is Ω _∗-close to its linear approximation when β is Ω _∗-close to the target β ⁰, then also the Ω-structured sparsity M-estimator $\hat{\beta }$ is Ω _∗-close to the target. Here, the second derivative inverse matrix $\ddot{R}^{-1}(\beta ^{0})$ is assumed to have Ω-small enough rows, where Ω is the dual norm of Ω _∗. Next, weakly decomposable norms Ω are considered. A generalized irrepresentable condition on a set S of indices yields that there is a solution $\tilde{\beta }_{S}$ of the KKT-conditions with zeroes outside the set S. At such a solution $\tilde{\beta }_{S}$ the problem is under certain conditions localized, so that one can apply the linear approximation of $\dot{R}(\beta )$. This scheme is carried out for exponential families and in particular for the graphical Lasso.