Sparse Boosting with Correlation Based Penalty

Abstract

In high dimensional setting, componentwise L ₂boosting method has been used to construct sparse model of high prediction, but it tends to select many ineffective variables. Several sparse boosting methods, such as, SparseL ₂Boosting and Twin Boosting, have been proposed to improve the variable selection of L ₂boosting algorithm. In this paper, we propose a new general sparse boosting method (GSBoosting). The relations are established between GSBoosting and other well known regularized variable selection methods in orthogonal linear model, such as adaptive Lasso, hard thresholds etc. Simulations results show that GSBoosting has good performance in both prediction and variable selection.