Skip to main content
SpringerLink
Log in

Versatile and Scalable Cosparse Methods for Physics-Driven Inverse Problems

  • Chapter
  • First Online:
Compressed Sensing and its Applications

Part of the book series: Applied and Numerical Harmonic Analysis ((ANHA))

Abstract

Solving an underdetermined inverse problem implies the use of a regularization hypothesis. Among possible regularizers, the so-called sparsity hypothesis, described as a synthesis (generative) model of the signal of interest from a low number of elementary signals taken in a dictionary, is now widely used. In many inverse problems of this kind, it happens that an alternative model, the cosparsity hypothesis (stating that the result of some linear analysis of the signal is sparse), offers advantageous properties over the classical synthesis model. A particular advantage is its ability to intrinsically integrate physical knowledge about the observed phenomenon, which arises naturally in the remote sensing contexts through some underlying partial differential equation. In this chapter, we illustrate on two worked examples (acoustic source localization and brain source imaging) the power of a generic cosparse approach to a wide range of problems governed by physical laws, how it can be adapted to each of these problems in a very versatile fashion, and how it can scale up to large volumes of data typically arising in applications.

Srđan Kitić contributed to the results reported in this chapter when he was with Univ Rennes, Inria, CNRS, IRSIA. The chapter was written while he was a postdoc with Technicolor, Rennes.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The operators for which \({\langle \mathfrak {L} \boldsymbol {\MakeLowercase {\mathit {p}}}_1, \, \boldsymbol {\MakeLowercase {\mathit {p}}}_2 \rangle = \langle \boldsymbol {\MakeLowercase {\mathit {p}}}_1, \, \mathfrak {L} \boldsymbol {\MakeLowercase {\mathit {p}}}_2 \rangle }\) holds. Otherwise, setting the adjoint boundary conditions would be required.

  2. 2.

    The change of notation, in particular from x/c to z for the unknown, is meant to cover both cases in a generic framework.

  3. 3.

    j denotes an iteration index.

  4. 4.

    g ∗(λ) :=sup z g(z) −z T λ.

  5. 5.

    The spatial dimensions remain fixed to ensure solvability of the inverse problem.

  6. 6.

    Note that, in this simplistic setting, a fast computation of Ω −1 c using the Thomas algorithm [77] could be exploited. The reader is reminded that this is not a generally available commodity, which is the main incentive for considering the analysis counterpart.

  7. 7.

    The value of ∥AΨ∥is actually somewhat lower than ∥Ψ∥; it depends on the number of microphones m and their random placement.

  8. 8.

    This metric is more reliable than the corresponding one with respect to the source signal, since small defects in support estimation should not disproportionately affect performance.

  9. 9.

    Vertical axis, k m, the ratio between the number of sources and sensors; horizontal axis, m s, the proportion of the discretized space occupied by sensors

References

  1. L. Albera, A. Ferreol, D. Cosandier-Rimele, I. Merlet, F. Wendling, Brain source localization using a fourth-order deflation scheme. IEEE Trans. Biomed. Eng. 55(2), 490–501 (2008)

    Google Scholar 

  2. L. Albera, S. Kitic, N. Bertin, G. Puy, R. Gribonval, Brain source localization using a physics-driven structured cosparse representation of EEG signals, in 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), September 21–24 (2014)

    Google Scholar 

  3. N. Antonello, T. van Waterschoot, M. Moonen, P.A. Naylor, Identification of surface acoustic impedances in a reverberant room using the FDTD method, in 2014 14th International Workshop on Acoustic Signal Enhancement (IWAENC), pp. 114–118 (IEEE, New York, 2014)

    Google Scholar 

  4. J. Barker, R. Marxer, E. Vincent, S. Watanabe, The third chime speech separation and recognition challenge: dataset, task and baselines, in 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Dec (2015), pp. 504–511

    Google Scholar 

  5. A. Beck, M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)

    Google Scholar 

  6. N. Bertin, S. Kitić, R. Gribonval, Joint estimation of sound source location and boundary impedance with physics-driven cosparse regularization, in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York, 2016), pp. 6340–6344

    Google Scholar 

  7. C. Bilen, S. Kitić, N. Bertin, R. Gribonval, Sparse acoustic source localization with blind calibration for unknown medium characteristics, in iTwist-2nd international-Traveling Workshop on Interactions Between Sparse Models and Technology (2014)

    Google Scholar 

  8. C. Blandin, A. Ozerov, E. Vincent, Multi-source TDOA estimation in reverberant audio using angular spectra and clustering. Signal Process. 92(8), 1950–1960 (2012)

    Google Scholar 

  9. T. Blumensath, M.E. Davies, Sampling theorems for signals from the union of finite-dimensional linear subspaces. IEEE Trans. Inf. Theory 55(4), 1872–1882 (2009)

    Google Scholar 

  10. P. Bochev, R.B. Lehoucq, On the finite element solution of the pure Neumann problem. SIAM Rev. 47(1), 50–66 (2005)

    Google Scholar 

  11. S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Google Scholar 

  12. J. Campbell, Introduction to Remote Sensing (Guilford Press, New York, 1996)

    Google Scholar 

  13. E.J. Candès, J.K. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)

    Google Scholar 

  14. A. Chambolle, T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)

    Google Scholar 

  15. A. Chambolle, T. Pock, An introduction to continuous optimization for imaging. Acta Numer. 25, 161–319 (2016)

    Google Scholar 

  16. V. Chandrasekaran, M.I. Jordan, Computational and statistical tradeoffs via convex relaxation. Proc. Natl. Acad. Sci. 110(13), E1181–E1190 (2013)

    Google Scholar 

  17. G. Chardon, T. Nowakowski, J. De Rosny, L. Daudet, A blind dereverberation method for narrowband source localization. J. Select. Topics Signal Process. 9, 815–824 (2015)

    Google Scholar 

  18. P.L. Combettes, J.-C. Pesquet, Proximal splitting methods in signal processing, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering (Springer, New York, 2011), pp. 185–212

    Google Scholar 

  19. L. Condat, A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)

    Google Scholar 

  20. G. Dassios, A. Fokas, The definite non-uniqueness results for deterministic eeg and meg data. Inv. Prob. 29(6), 065012 (2013)

    Google Scholar 

  21. A. Deleforge, F. Forbes, R. Horaud, Acoustic space learning for sound-source separation and localization on binaural manifolds. Int. J. Neural Syst. 25(01), 1440003 (2015)

    Google Scholar 

  22. W. Deng, W. Yin, On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. 66(3), 889–916 (2016)

    Google Scholar 

  23. A. Devaney, G. Sherman, Nonuniqueness in inverse source and scattering problems. IEEE Trans. Antennas Propag. 30(5), 1034–1037 (1982)

    Google Scholar 

  24. I. Dokmanić, Listening to Distances and Hearing Shapes. PhD thesis, EPFL (2015)

    Google Scholar 

  25. I. Dokmanić, M. Vetterli, Room helps: acoustic localization with finite elements, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012 (IEEE, New York, 2012), pp. 2617–2620

    Google Scholar 

  26. P.L. Dragotti, M. Vetterli, T. Blu, Sampling moments and reconstructing signals of finite rate of innovation: Shannon meets Strang-fix. IEEE Trans. Signal Process. 55(5), 1741–1757 (2007)

    Google Scholar 

  27. J. Eckstein, D.P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55(1), 293–318 (1992)

    Google Scholar 

  28. M. Elad, P. Milanfar, R. Rubinstein, Analysis versus synthesis in signal priors. Inv. Prob. 23(3), 947 (2007)

    Google Scholar 

  29. Q. Fang, D.A. Boas, Tetrahedral mesh generation from volumetric binary and grayscale images. in 2009 IEEE International Symposium on Biomedical Imaging from Nano to Macro (IEEE, New York, 2009), pp. 1142–1145

    Google Scholar 

  30. D.C.-L. Fong, M. Saunders, LSMR: an iterative algorithm for sparse least-squares problems. SIAM J. Sci. Comput. 33(5), 2950–2971 (2011)

    Google Scholar 

  31. I. Fried, Bounds on the spectral and maximum norms of the finite element stiffness, flexibility and mass matrices. Int. J. Solids Struct. 9(9), 1013–1034 (1973)

    Google Scholar 

  32. A. Friedman, Partial Differential Equations of Parabolic Type (Courier Dover Publications, New York, 2008)

    Google Scholar 

  33. D. Gabay, B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl. 2(1), 17–40 (1976)

    Google Scholar 

  34. S. Gannot, D. Burshtein, E. Weinstein, Signal enhancement using beamforming and nonstationarity with applications to speech. IEEE Trans. Signal Process. 49(8), 1614–1626 (2001)

    Google Scholar 

  35. M.S. Gockenbach, Understanding and Implementing the Finite Element Method (SIAM, Philadelphia, 2006)

    Google Scholar 

  36. T. Goldstein, B. O’Donoghue, S. Setzer, R. Baraniuk, Fast alternating direction optimization methods. SIAM J. Imag. Sci. 7(3), 1588–1623 (2014)

    Google Scholar 

  37. R. Grech, T. Cassar, J. Muscat, K.P. Camilleri, S.G. Fabri, M. Zervakis, P. Xanthopoulos, V. Sakkalis, B. Vanrumste, Review on solving the inverse problem in EEG source analysis. J. NeuroEng. Rehabil. 7, 5–25 (2008)

    Google Scholar 

  38. H. Hallez, B. Vanrumste, R. Grech, J. Muscat, W. De Clercq, A. Vergult, Y. D ’asseler, K.P. Camilleri, S.G. Fabri, S.V. Huffel, I. Lemahieu, Review on solving the forward problem in EEG source analysis. J. Neuroeng. Rehabil. 4(4) (2007). https://doi.org/10.1186/1743-0003-4-46

  39. M. Herman, T. Strohmer et al., General deviants: an analysis of perturbations in compressed sensing. IEEE J. Sel. Top. Sign. Proces. 4(2), 342–349 (2010)

    Google Scholar 

  40. V. Isakov, Inverse Problems for Partial Differential Equations, vol. 127 (Springer Science & Business Media, Berlin, 2006)

    Google Scholar 

  41. B.H. Jansen, V.G. Rit, Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns? Biol. Cybern. 73(4), 357–366 (1995)

    Google Scholar 

  42. R. Jenatton, J.-Y. Audibert, F. Bach, Structured variable selection with sparsity-inducing norms. J. Mach. Learn. Res. 12, 2777–2824 (2011)

    Google Scholar 

  43. C.R. Johnson, Computational and numerical methods for bioelectric field problems. Crit. Rev. Biomed. Eng. 25(1), 1–81 (1997)

    Google Scholar 

  44. U.S. Kamilov, H. Mansour, Learning optimal nonlinearities for iterative thresholding algorithms. IEEE Signal Process. Lett. 23(5), 747–751 (2016)

    Google Scholar 

  45. Y. Kim, P. Nelson, Optimal regularisation for acoustic source reconstruction by inverse methods. J. Sound Vib. 275(3–5), 463–487 (2004)

    Google Scholar 

  46. S. Kitić, N. Bertin, R. Gribonval, Hearing behind walls: localizing sources in the room next door with cosparsity, in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York, 2014), pp. 3087–3091

    Google Scholar 

  47. S. Kitić, L. Albera, N. Bertin, R. Gribonval, Physics-driven inverse problems made tractable with cosparse regularization. IEEE Trans. Signal Process. 64(2), 335–348 (2016)

    Google Scholar 

  48. K. Kowalczyk, M. van Walstijn, Modeling frequency-dependent boundaries as digital impedance filters in FDTD and K-DWM room acoustics simulations. J. Audio Eng. Soc. 56(7/8), 569–583 (2008)

    Google Scholar 

  49. H. Kuttruff, Room Acoustics (CRC Press, Boca Raton, 2016)

    Google Scholar 

  50. J. Le Roux, P.T. Boufounos, K. Kang, J.R. Hershey, Source localization in reverberant environments using sparse optimization, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2013 (IEEE, New York, 2013), pp. 4310–4314

    Google Scholar 

  51. R.J. LeVeque, Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time-Dependent Problems (SIAM, Philadelphia, 2007)

    Google Scholar 

  52. T.T. Lin, F.J. Herrmann, Compressed wavefield extrapolation. Geophysics 72(5), SM77–SM93 (2007)

    Google Scholar 

  53. D. Malioutov, A sparse signal reconstruction perspective for source localization with sensor arrays. Master’s thesis (2003)

    Google Scholar 

  54. D. Malioutov, M. Çetin, A.S. Willsky, A sparse signal reconstruction perspective for source localization with sensor arrays. IEEE Trans. Signal Process. 53(8), 3010–3022 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  55. M. Mohr, B. Vanrumste, Comparing iterative solvers for linear systems associated with the finite difference discretisation of the forward problem in electro-encephalographic source analysis. Med. Biol. Eng. Comput. 41, 75–84 (2003)

    Article  Google Scholar 

  56. J.-J. Moreau, Proximité et dualité dans un espace hilbertien. Bull. de la Société mathématique de France, 93, 273–299 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  57. J.C. Mosher, R.M. Leahy, Source localization using recursively applied and projected (RAP) music. IEEE Trans. Signal Process. 47(2), 332–340 (1999)

    Article  Google Scholar 

  58. W. Munk, P. Worcester, C. Wunsch, Ocean Acoustic Tomography (Cambridge University Press, Cambridge, 2009)

    Google Scholar 

  59. G. Mur, Absorbing boundary conditions for the finite-difference approximation of the time-domain electromagnetic-field equations. IEEE Trans. Electromagn. Comput. EMC-23(4), 377–382 (1981)

    Google Scholar 

  60. J. Murray-Bruce, P.L. Dragotti, Solving physics-driven inverse problems via structured least squares, in 2016 24th European Signal Processing Conference (EUSIPCO) (IEEE, New York, 2016), pp. 331–335

    Google Scholar 

  61. S. Nam, M.E. Davies, M. Elad, R. Gribonval, The cosparse analysis model and algorithms. Appl. Comput. Harmon. Anal. 34(1), 30–56 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  62. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, vol. 87 (Springer Science & Business Media, New York, 2013)

    MATH  Google Scholar 

  63. R. Oostenvelda, P. Praamstra, The five percent electrode system for high-resolution EEG and ERP measurements. Clin. Neurophysiol. Elsevier, 112(4), 713–719 (2001)

    Article  Google Scholar 

  64. A. Pezeshki, Y. Chi, L.L. Scharf, E.K. Chong, Compressed sensing, sparse inversion, and model mismatch, in Compressed Sensing and Its Applications (Springer, New York, 2015), pp. 75–95

    Book  MATH  Google Scholar 

  65. T. Piotrowski, D. Gutierrez, I. Yamada, J. Zygierewicz, Reduced-rank neural activity index for EEG/MEG multi-source localization, in ICASSP’14, IEEE International Conference on Acoustics Speech and Signal Processing, Florence, May 4–9 (2014)

    Google Scholar 

  66. L.C. Potter, E. Ertin, J.T. Parker, M. Cetin, Sparsity and compressed sensing in radar imaging. Proc. IEEE 98(6), 1006–1020 (2010)

    Article  Google Scholar 

  67. J. Provost, F. Lesage, The application of compressed sensing for photo-acoustic tomography. IEEE Trans. Med. Imag. 28(4), 585–594 (2009)

    Article  Google Scholar 

  68. A. Rosenthal, D. Razansky, V. Ntziachristos, Fast semi-analytical model-based acoustic inversion for quantitative optoacoustic tomography. IEEE Trans. Med. Imag. 29(6), 1275–1285 (2010)

    Article  Google Scholar 

  69. Y. Saad, Iterative Methods for Sparse Linear Systems (SIAM, Philadelphia, 2003)

    Book  MATH  Google Scholar 

  70. O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, F. Lenzen, Variational Methods in Imaging, vol. 320 (Springer, Berlin, 2009)

    MATH  Google Scholar 

  71. P. Schimpf, C. Ramon, J. Haueisen, Dipole models for the EEG and MEG. IEEE Trans. Biomed. Eng. 49(5), 409–418 (2002)

    Article  Google Scholar 

  72. R. Schmidt, Multiple emitter location and signal parameter estimation. IEEE Trans. Antennas Propag. 34(3), 276–280 (1986). Reprint of the original 1979 paper from the RADC Spectrum Estimation Workshop

    Google Scholar 

  73. S. Shalev-Shwartz, N. Srebro, SVM optimization: inverse dependence on training set size, in Proceedings of the 25th international conference on Machine learning (ACM, New York, 2008), pp. 928–935

    Google Scholar 

  74. R. Shefi, M. Teboulle, Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization. SIAM J. Optim. 24(1), 269–297 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  75. J.-L. Starck, J.M. Fadili, An overview of inverse problem regularization using sparsity, in IEEE ICIP, Cairo, Nov (2009), pp. 1453–1456

    Google Scholar 

  76. G. Strang, Computational Science and Engineering, vol. 791 (Wellesley-Cambridge Press, Wellesley, 2007)

    MATH  Google Scholar 

  77. L. Thomas, Using a computer to solve problems in physics. Appl. Digital Comput. 458, 44–45 (1963)

    Google Scholar 

  78. J.A. Tropp, S.J. Wright, Computational methods for sparse solution of linear inverse problems. Proc. IEEE 98(6), 948–958 (2010)

    Article  Google Scholar 

  79. K. Uutela, M. Hamalainen, E. Somersalo, Visualization of magnetoencephalographic data using minimum current estimates. Elsevier NeuroImage 10(2), 173–180 (1999)

    Article  Google Scholar 

  80. J.-M. Valin, F. Michaud, J. Rouat, D. Létourneau, Robust sound source localization using a microphone array on a mobile robot, in Proceedings. 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003. (IROS 2003), vol. 2, pp. 1228–1233 (IEEE, New York, 2003)

    Google Scholar 

  81. F. Vatta, F. Meneghini, F. Esposito, S. Mininel, F. Di Salle, Realistic and spherical head modeling for EEG forward problem solution: a comparative cortex-based analysis. Comput. Intell. Neurosci. 2010, 11pp. (2010)

    Google Scholar 

  82. M. Vetterli, P. Marziliano, T. Blu, Sampling signals with finite rate of innovation. IEEE Trans. Signal Process. 50(6), 1417–1428 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  83. C. Wolters, H. Köstler, C. Möller, J. Härdtlein, A. Anwander, Numerical approaches for dipole modeling in finite element method based source analysis. Int. Congr. Ser. 1300, 189–192 (2007)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the European Research Council, PLEASE project (ERC-StG-2011-277906).

Author information

Authors and Affiliations

  1. Technicolor, Cesson-Sévigné, France

    Srđan Kitić

  2. Université de Rennes 1 (LTSI), Rennes, France

    Siouar Bensaid & Laurent Albera

  3. Univ Rennes, Inria, CNRS, IRISA, F-35000, Rennes, France

    Nancy Bertin & Rémi Gribonval

Authors
  1. Srđan Kitić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Siouar Bensaid
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laurent Albera
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nancy Bertin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rémi Gribonval
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rémi Gribonval .

Editor information

Editors and Affiliations

  1. Fakultät für Elektrotechnik und Informationstechnik, Technische Universität München, Munich, Bavaria, Germany

    Holger Boche

  2. Institut für Telekommunikationssysteme, Technische Universität Berlin, Berlin, Germany

    Giuseppe Caire

  3. Department of Electrical & Computer Engineering, Duke University, Durham, North Carolina, USA

    Robert Calderbank

  4. Institut für Mathematik, Technische Universität Berlin, Berlin, Germany

    Maximilian März

  5. Institut für Mathematik, Technische Universität Berlin, Berlin, Germany

    Gitta Kutyniok

  6. Lehrstuhl und Institute für Statistik, RWTH Aachen, Aachen, Germany

    Rudolf Mathar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kitić, S., Bensaid, S., Albera, L., Bertin, N., Gribonval, R. (2017). Versatile and Scalable Cosparse Methods for Physics-Driven Inverse Problems. In: Boche, H., Caire, G., Calderbank, R., März, M., Kutyniok, G., Mathar, R. (eds) Compressed Sensing and its Applications. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-69802-1_10

Download citation

Publish with us

Policies and ethics

Search

Navigation