Abstract
Compressed sensing was introduced some ten years ago as an effective way of acquiring signals, which possess a sparse or nearly sparse representation in a suitable basis or dictionary. Due to its solid mathematical backgrounds, it quickly attracted the attention of mathematicians from several different areas, so that the most important aspects of the theory are nowadays very well understood. In recent years, its applications started to spread out through applied mathematics, signal processing, and electrical engineering. The aim of this chapter is to provide an introduction into the basic concepts of compressed sensing. In the first part of this chapter, we present the basic mathematical concepts of compressed sensing, including the Null Space Property, Restricted Isometry Property, their connection to basis pursuit and sparse recovery, and construction of matrices with small restricted isometry constants. This presentation is easily accessible, largely self-contained, and includes proofs of the most important theorems. The second part gives an overview of the most important extensions of these ideas, including recovery of vectors with sparse representation in frames and dictionaries, discussion of (in)coherence and its implications for compressed sensing, and presentation of other algorithms of sparse recovery.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Achlioptas, D.: Database-friendly random projections: Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 66, 671–687 (2003)
Arora, S., Barak, B.: Computational Complexity: A Modern Approach. Cambridge University Press, Cambridge (2009)
Baraniuk, R., Cevher, V., Duarte, M.F., Hegde, C.: Model-based compressive sensing, IEEE Trans. Inf. Theory 56, 1982–2001 (2010)
Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28, 253–263 (2008)
Baraniuk, R., Steeghs, P.: Compressive radar imaging. In: Proc. IEEE Radar Conf., Boston, pp. 128–133 (2007)
Baron, D., Duarte, M.F., Sarvotham, S., Wakin, M.B., Baraniuk, R.: Distributed compressed sensing of jointly sparse signals. In: Proc. Asilomar Conf. Signals, Systems, and Computers, Pacic Grove (2005)
Becker, S., Candés, E.J., Grant, M.: Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3, 165–218 (2010)
Blumensath, T., Davies, M.: Iterative hard thresholding for compressive sensing. Appl. Comput. Harmon. Anal. 27, 265–274 (2009)
Cai, T., Wang, L., Xu, G.: New bounds for restricted isometry constants. IEEE Trans. Inf. Theory 56, 4388–4394 (2010)
Cai, T., Wang, L., Xu, G.: Shifting inequality and recovery of sparse vectors. IEEE Trans. Signal Process. 58, 1300–1308 (2010)
Candés, E.J.: The restricted isometry property and its implications for compressed sensing. C. R. Acad. Sci., Paris, Ser. I 346, 589–592 (2008)
Candés, E.J., Plan, Y.: A probabilistic and RIPless theory of compressed sensing. IEEE Trans. Inf. Theory 57, 7235–7254 (2011)
Candés, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52, 489–509 (2006)
Candés, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59, 1207–1223 (2006)
Candés, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51, 4203–4215 (2005)
Candés, E.J., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52, 5406–5425 (2006)
Candés, E.J., Tao, T.: The Dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35, 2313–2351 (2007)
Cevher, V., Tyagi, H.: Active learning of multi-index function models. In: Proc. NIPS (The Neural Information Processing Systems), Lake Tahoe, Reno (2012)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)
Cohen, A., Dahmen, W., DeVore, R.: Compressed sensing and best k-term approximation. J. Am. Math. Soc. 22, 211–231 (2009)
Cohen, A., Daubechies, I., DeVore, R., Kerkyacharian, G., Picard, D.: Capturing ridge functions in high dimensions from point queries. Constr. Approx. 35, 225–243 (2012)
Dasgupta, S., Gupta, A.: An elementary proof of a theorem of Johnson and Lindenstrauss. Random Struct. Algorithm. 22, 60–65 (2003)
Davenport, M.A., Duarte, M.F., Eldar, Y.C., Kutyniok, G.: Introduction to compressed sensing. Compressed sensing, pp. 1–64. Cambridge University Press, Cambridge (2012)
DeVore, R., Petrova, G., Wojtaszczyk, P.: Approximation of functions of few variables in high dimensions. Constr. Approx. 33, 125–143 (2011)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52, 1289–1306 (2006)
Donoho, D.L., Tsaig, Y., Drori, I., Starck, J.-L.: Sparse solution of underdetermined systems of linear equations by stagewise Orthogonal Matching Pursuit. IEEE Trans. Inf. Theory 58, 1094–1121 (2012)
Dorfman, R.: The detection of defective members of large populations. Ann. Math. Stat. 14, 436–440 (1943)
Du, D., Hwang, F.: Combinatorial group testing and its applications. World Scientic, Singapore (2000)
Duarte, M., Davenport, M., Takhar, D., Laska, J., Ting, S., Kelly, K., Baraniuk R.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25, 83–91 (2008)
Duarte, M., Wakin, M., Baraniuk, R.: Fast reconstruction of piecewise smooth signals from random projections. In: Proc. Work. Struc. Parc. Rep. Adap. Signaux (SPARS), Rennes (2005)
Duarte, M., Wakin, M., Baraniuk, R.: Wavelet-domain compressive signal reconstruction using a hidden Markov tree model. In: Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP), Las Vegas (2008)
Eldar, Y., Mishali, M.: Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 55, 5302–5316 (2009)
Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, New York (2010)
Figueiredo, M., Nowak, R., Wright, S.: Gradient projections for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Select. Top. Signal Process. 1, 586–597 (2007)
Fornasier, M., Rauhut, H.: Compressive sensing. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, pp. 187–228. Springer, Heidelberg (2011)
Fornasier, M., Rauhut, H.: Recovery algorithms for vector valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46, 577–613 (2008)
Fornasier, M., Schnass, K., Vybíral, J.: Learning functions of few arbitrary linear parameters in high dimensions. Found. Comput. Math. 12, 229–262 (2012)
Foucart, S.: A note on guaranteed sparse recovery via 1-minimization. Appl. Comput. Harmon. Anal. 29, 97–103 (2010)
Foucart, S., Lai M.: Sparsest solutions of underdetermined linear systems via l q -minimization for 0 < q ≤ 1. Appl. Comput. Harmon. Anal. 26, 395–407 (2009)
Foucart, S., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Birkhäuser/Springer, New York (2013)
Friedman, J., Hastie, T., Tibshirani, R.: Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw. 33, 1–22 (2010)
Gärtner, B., Matoušek, J.: Understanding and Using Linear Programming. Springer, Berlin (2006)
Gilbert, A., Li, Y., Porat, E., and Strauss, M.: Approximate sparse recovery: optimizaing time and measurements. In: Proc. ACM Symp. Theory of Comput., Cambridge (2010)
Gilbert, A., Strauss, M., Tropp, J., Vershynin, R.: One sketch for all: fast algorithms for compressed sensing. In: Proc. ACM Symp. Theory of Comput., San Diego (2007)
Jasper, J., Mixon, D.G., Fickus M.: Kirkman equiangular tight frames and codes. IEEE Trans. Inf. Theory 60, 170–181 (2014)
Johnson, W.B., Lindenstrauss, J.: Extensions of Lipschitz mappings into a Hilbert space. In: Conf. in Modern Analysis and Probability, pp. 189–206 (1984)
Krahmer, F., Ward, R.: New and improved Johnson-Lindenstrauss embeddings via the restricted isometry property. SIAM J. Math. Anal. 43, 1269–1281 (2011)
La, C., Do, M.N.: Tree-based orthogonal matching pursuit algorithm for signal reconstruction. In: IEEE Int. Conf. Image Processing (ICIP), Atlanta (2006)
Ledoux, M.: The Concentration of Measure Phenomenon. American Mathematical Society, Providence (2001)
Ledoux, M., Talagrand, M.: Probability in Banach Spaces. Isoperimetry and Processes. Springer, Berlin (1991)
Loris, I.: On the performance of algorithms for the minimization of ℓ 1-penalized functions. Inverse Prob. 25, 035008 (2009)
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007)
Mallat, S., Zhang, Z.: Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41, 3397–3415 (1993)
Matoušek, J.: On variants of the Johnson-Lindenstrauss lemma. Rand. Struct. Algorithm. 33, 142–156 (2008)
Milman, V.D., Schechtman, G.: Asymptotic theory of finite-dimensional normed spaces. Springer, Berlin (1986)
Mishali, M., Eldar, Y.: From theory to practice: Sub-nyquist sampling of sparse wideband analog signals. IEEE J. Sel. Top. Signal Process. 4, 375–391 (2010)
Needell, D., Tropp, J.: CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26, 301–321 (2009)
Pietsch, A.: Operator Ideals. North-Holland, Amsterdam (1980)
Rauhut, H.: Random sampling of sparse trigonometric polynomials. Appl. Comput. Harmon. Anal. 22, 16–42 (2007)
Rauhut, H., Schnass, K., Vandergheynst, P.: Compressed sensing and redundant dictionaries. IEEE Trans. Inf. Theor. 54, 2210–2219 (2008)
Rauhut, H., Ward, R.: Sparse Legendre expansions via ℓ 1-minimization. J. Approx. Theory 164, 517–533 (2012)
Schnass, K., Vybíral, J.: Compressed learning of high-dimensional sparse functions. In: IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 3924–3927 (2011)
Stojnic, M., Parvaresh, F., Hassibi, B.: On the reconstruction of block-sparse signals with an optimal number of measurements. IEEE Trans. Signal Process. 57, 3075–3085 (2009)
Tibshirani, R.: Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. B 58, 267–288 (1996)
Tropp, J.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theor. 50, 2231–2242 (2004)
Tropp, J.: Algorithms for simultaneous sparse approximation. Part II: Convex relaxation. Signal Process. 86, 589–602 (2006)
Tropp, J., Gilbert, A.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53, 4655–4666 (2007)
Tropp, J., Gilbert, A., Strauss, M.: Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit. Signal Process. 86, 572–588 (2006)
Tropp, J., Laska, J., Duarte, M., Romberg, J., Baraniuk, R.: Beyond Nyquist: efficient sampling of sparse bandlimited signals. IEEE Trans. Inf. Theor. 56, 520–544 (2010)
Wakin, M.B., Sarvotham, S., Duarte, M.F., Baron, D., Baraniuk, R.: Recovery of jointly sparse signals from few random projections. In: Proc. Workshop on Neural Info. Proc. Sys. (NIPS), Vancouver (2005)
Welch, L.: Lower bounds on the maximum cross correlation of signals. IEEE Trans. Inf. Theory 20, 397–399 (1974)
Wojtaszczyk, P.: Complexity of approximation of functions of few variables in high dimensions. J. Complex. 27, 141–150 (2011)
Yin, W., Osher, S., Goldfarb, D., Darbon, J.: Bregman iterative algorithms for ℓ 1-minimization with applications to compressed sensing. SIAM J. Imag. Sci. 1, 143–168 (2008)
Acknowledgements
We would like to thank Sandra Keiper and Holger Rauhut for careful reading of a previous version of this Introduction and for useful comments. The last author was supported by the ERC CZ grant LL1203 of the Czech Ministry of Education.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Boche, H., Calderbank, R., Kutyniok, G., Vybíral, J. (2015). A Survey of Compressed Sensing. In: Boche, H., Calderbank, R., Kutyniok, G., Vybíral, J. (eds) Compressed Sensing and its Applications. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-16042-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-16042-9_1
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-319-16041-2
Online ISBN: 978-3-319-16042-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)