Abstract
We introduce a new greedy algorithm to find approximate sparse representations \(\vec s\) of \(\vec x={\vec A}\vec s\) by finding the Basis Pursuit (BP) solution of the linear program \(\min\{{\|s\|}_{\vec 1} \mid {\vec x}={\vec A}{\vec s}\}\). The proposed algorithm is based on the geometry of the polar polytope \(P^* = \{{\vec c} \mid {\tilde{\vec A}}^T{\vec c} \le {\vec 1} \}\) where \({\tilde{\vec A}} = [{\vec A},-{\vec A}]\) and searches for the vertex \({\vec c}^*\in P^*\) which maximizes \({\vec x}^{T}{\vec c}\) using a path following method. The resulting algorithm is in the style of Matching Pursuits (MP), in that it adds new basis vectors one at a time, but it uses a different correlation criterion to determine which basis vector to add and can switch out basis vectors as necessary. The algorithm complexity is of a similar order to Orthogonal Matching Pursuits (OMP). Experimental results show that this algorithm, which we call Polytope Faces Pursuit, produces good results on examples that are known to be hard for MP, and it is faster than the interior point method for BP on the experiments presented.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kreutz-Delgado, K., Murray, J.F., Rao, B.D., Engan, K., Lee, T.W., Sejnowski, T.J.: Dictionary learning algorithms for sparse representation. Neural Computation 15, 349–396 (2003)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing 20, 33–61 (1998)
Mallat, S., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 41, 3397–3415 (1993)
Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: Conference Record of The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, pp. 40–44 (1993)
Gribonval, R., Nielsen, M.: Approximation with highly redundant dictionaries. In: Wavelets: Applications in Signal and Image Processing, Proc. SPIE 2003, San Diego, USA, pp. 216–227 (2003)
Tropp, J.A.: Greed is good: Algorithmic results for sparse approximation. IEEE Transactions on Information Theory 50, 2231–2242 (2004)
Fuchs, J.J.: On sparse representations in arbitrary redundant bases. IEEE Transactions on Information Theory 50, 1341–1344 (2004)
Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proc. Nat. Aca. Sci. 100, 2197–2202 (2003)
Donoho, D.L.: Neighborly polytopes and sparse solutions of underdetermined linear equations. Technical report, Statistics Department, Stanford University (2004)
Plumbley, M.D.: Polar polytopes and recovery of sparse representations (2005) (submitted for publication)
Schrijver, A.: Theory of Linear and Integer Programming. John Wiley & Sons Ltd, Chichester (1998)
Grünbaum, B.: Convex Polytopes, 2nd edn. Graduate Texts in Mathematics, vol. 221. Springer, New York (2003)
Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Computing 25, 227–234 (1995)
Andrle, M., Rebollo-Neira, L.: A swapping-based refinement of orthogonal matching pursuit strategies. Signal Processing (2005) (to appear in Signal Processing)
Kvasnica, M., Grieder, P., Baotić, M.: Multi-Parametric Toolbox, MPT (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Plumbley, M.D. (2006). Recovery of Sparse Representations by Polytope Faces Pursuit. In: Rosca, J., Erdogmus, D., PrÃncipe, J.C., Haykin, S. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2006. Lecture Notes in Computer Science, vol 3889. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11679363_26
Download citation
DOI: https://doi.org/10.1007/11679363_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-32630-4
Online ISBN: 978-3-540-32631-1
eBook Packages: Computer ScienceComputer Science (R0)