Skip to main content
SpringerLink
Log in

First Order Algorithms in Variational Image Processing

  • Chapter
  • First Online:
Splitting Methods in Communication, Imaging, Science, and Engineering

Part of the book series: Scientific Computation ((SCIENTCOMP))

Abstract

The success of non-smooth variational models in image processing is heavily based on efficient algorithms. Taking into account the specific structure of the models as sum of different convex terms, splitting algorithms are an appropriate choice. Their strength consists in the splitting of the original problem into a sequence of smaller proximal problems which are easy and fast to compute.

Operator splitting methods were first applied to linear, single-valued operators for solving partial differential equations in the 60th of the last century. More than 20 years later these methods were generalized in the convex analysis community to the solution of inclusion problems, where the linear operators have to be replaced by nonlinear, set-valued, monotone operators. Again after more than 20 years splitting methods became popular in image processing. In particular, operator splittings in combination with (augmented) Lagrangian methods and primal-dual methods have been applied very successfully.

In this chapter we give an overview of first order algorithms recently used to solve convex non-smooth variational problems in image processing. We present computational studies providing a comparison of different methods and also illustrating their success in applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    There exist different notations for the convergence rate of algorithms in the literature. Sometimes the notation of this chapter is also called “superlinear convergence” while \(\|\hat{x} - x^{(r)}\| \leq C\delta ^{r}\), δ < 1 is used for the linear convergence. But if C = C(r) → 0 as r → +∞ in the last formula, this could be also meant by “superlinear convergence”.

References

  1. J. F. P.-J. Abascal, J. Chamorro-Servent, J. Aguirre, S. Arridge, T. Correia, J. Ripoll, J. J. Vaquero, and M. Desco. Fluorescence diffuse optical tomography using the split Bregman method. Med. Phys., 38:6275, 2011.

    Article  Google Scholar 

  2. R. E. Alvarez and A. Macovski. Energy-selective reconstructions in X-ray computerized tomography. Phys. Med. Biol., 21(5):733–744, 1976.

    Article  Google Scholar 

  3. S. Anthoine, J.-F. Aujol, Y. Boursier, and C. Mélot. On the efficiency of proximal methods in CBCT and PET. In Proc. IEEE Int. Conf. Image Proc. (ICIP), 2011.

    Google Scholar 

  4. S. Anthoine, J.-F. Aujol, Y. Boursier, and C. Mélot. Some proximal methods for CBCT and PET. In Proc. SPIE (Wavelets and Sparsity XIV), volume 8138, 2011.

    Google Scholar 

  5. K. J. Arrow, L. Hurwitz, and H. Uzawa. Studies in Linear and Nonlinear Programming. Stanford University Press, 1958.

    MATH  Google Scholar 

  6. H. Attouch and J. Bolte. On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program., 116(1–2):5–16, 2009.

    Article  MATH  Google Scholar 

  7. H. Attouch, J. Bolte, and B. F. Svaiter. Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. Series A, 137(1–2):91–129, 2013.

    Article  MATH  Google Scholar 

  8. J.-P. Aubin. Optima and Equilibria: An Introduction to Nonlinear Analysis. Springer, Berlin, Heidelberg, New York, 2nd edition, 2003.

    Google Scholar 

  9. M. Bachmayr and M. Burger. Iterative total variation schemes for nonlinear inverse problems. Inverse Problems, 25(10):105004, 2009.

    Google Scholar 

  10. E. Bae, J. Yuan, and X.-C. Tai. Global minimization for continuous multiphase partitioning problems using a dual approach. International Journal of Computer Vision, 92(1):112–129, 2011.

    Article  MATH  Google Scholar 

  11. J. Barzilai and J. M. Borwein. Two-point step size gradient methods. IMA J. Numer. Anal., 8(1):141–148, 1988.

    Article  MATH  Google Scholar 

  12. H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, New York, 2011.

    Book  MATH  Google Scholar 

  13. A. Beck and M. Teboulle. Fast gradient-based algorithms for constrained total variation image denoising and deblurring. SIAM J. Imag. Sci., 2:183–202, 2009.

    Article  Google Scholar 

  14. S. Becker, J. Bobin, and E. J. Candès. NESTA: a fast and accurate first-order method for sparse recovery. SIAM J. Imag. Sci., 4(1):1–39, 2011.

    Article  MATH  Google Scholar 

  15. M. Benning, L. Gladden, D. Holland, C.-B. Schönlieb, and T. Valkonen. Phase reconstruction from velocity-encoded MRI measurement - A survey of sparsity-promoting variational approaches. J. Magn. Reson., 238:26–43, 2014.

    Article  Google Scholar 

  16. M. Benning, P. Heins, and M. Burger. A solver for dynamic PET reconstructions based on forward-backward-splitting. In AIP Conf. Proc., volume 1281, pages 1967–1970, 2010.

    Google Scholar 

  17. D. P. Bertsekas. Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York, 1982.

    MATH  Google Scholar 

  18. D. P. Bertsekas. Incremental proximal methods for large scale convex optimization. Math. Program., Ser. B, 129(2):163–195, 2011.

    Article  MATH  Google Scholar 

  19. J. M. Bioucas-Dias and M. A. T. Figueiredo. Multiplicative noise removal using variable splitting and constrained optimization. IEEE Trans. Image Process., 19(7):1720–1730, 2010.

    Article  Google Scholar 

  20. A. Björck. Least Squares Problems. SIAM, Philadelphia, 1996.

    Book  MATH  Google Scholar 

  21. D. Boley. Local linear convergence of the alternating direction method of multipliers on quadratic or linear programs. SIAM J. Optim., 2014.

    Google Scholar 

  22. J. Bolte, S. Sabach, and M. Teboulle. Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program., Series A, 2013.

    Google Scholar 

  23. S. Bonettini and V. Ruggiero. On the convergence of primal-dual hybrid gradient algorithms for total variation image restoration. J. Math. Imaging Vis., 44:236–253, 2012.

    Article  MATH  Google Scholar 

  24. J. F. Bonnans, J. C. Gilbert, C. Lemaréchal, and C. A. Sagastizábal. A family of variable metric proximal methods. Mathematical Programming, 68:15–47, 1995.

    MATH  Google Scholar 

  25. R. I. Boţ and C. Hendrich. A Douglas-Rachford type primal-dual method for solving inclusions with mixtures of composite and parallel-sum type monotone operators. SIAM Journal on Optimization, 23(4):2541–2565, 2013.

    Article  MATH  Google Scholar 

  26. R. I. Boţ and C. Hendrich. Convergence analysis for a primal-dual monotone + skew splitting algorithm with applications to total variation minimization. Journal of Mathematical Imaging and Vision, 49(3):551–568, 2014.

    Article  MATH  Google Scholar 

  27. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1):101–122, 2011.

    MATH  Google Scholar 

  28. K. Bredies, K. Kunisch, and T. Pock. Total generalized variation. SIAM Journal on Imaging Sciences, 3(3):492–526, 2010.

    Article  MATH  Google Scholar 

  29. F. E. Browder and W. V. Petryshyn. The solution by iteration of nonlinear functional equations in Banach spaces. Bulletin of the American Mathematical Society, 72:571–575, 1966.

    Article  MATH  Google Scholar 

  30. C. Brune, A. Sawatzky, and M. Burger. Primal and dual Bregman methods with application to optical nanoscopy. International Journal of Computer Vision, 92(2):211–229, 2010.

    Article  MATH  Google Scholar 

  31. M. Burger and S. Osher. A guide to the TV zoo. In Level Set and PDE Based Reconstruction Methods in Imaging, pages 1–70. Springer, 2013.

    Google Scholar 

  32. M. Burger, E. Resmerita, and L. He. Error estimation for Bregman iterations and inverse scale space methods in image restoration. Computing, 81(2–3):109–135, 2007.

    Article  MATH  Google Scholar 

  33. J. V. Burke and M. Qian. A variable metric proximal point algorithm for monotone operators. SIAM Journal on Control and Optimization, 37:353–375, 1999.

    Article  MATH  Google Scholar 

  34. F. Büther, M. Dawood, L. Stegger, F. Wübbeling, M. Schäfers, O. Schober, and K. P. Schäfers. List mode-driven cardiac and respiratory gating in PET. J. Nucl. Med., 50(5):674–681, 2009.

    Article  Google Scholar 

  35. D. Butnariu and A. N. Iusem. Totally convex functions for fixed points computation and infinite dimensional optimization, volume 40 of Applied Optimization. Kluwer, Dordrecht, 2000.

    Google Scholar 

  36. C. Byrne. A unified treatment of some iterative algorithms in signal processing and image reconstruction. Inverse Problems, 20:103–120, 2004.

    Article  MATH  Google Scholar 

  37. J.-F. Cai, E. J. Candès, and Z. Shen. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4):1956–1982, 2010.

    Article  MATH  Google Scholar 

  38. J.-F. Cai, S. Osher, and Z. Shen. Convergence of the linearized Bregman iteration for ℓ 1-norm minimization. Mathematics of Computation, 78(268):2127–2136, 2009.

    Article  MATH  Google Scholar 

  39. J. Cammin, J. S. Iwanczyk, and K. Taguchi. Emerging Imaging Technologies in Medicine, chapter Spectral/Photo-Counting Computed Tomography, pages 23–39. CRC Press, 2012.

    Google Scholar 

  40. R. Carmi, G. Naveh, and A. Altman. Material separation with dual-layer CT. In Proc. IEEE Nucl. Sci. Symp. Conf. Rec., pages 1876–1878, 2005.

    Google Scholar 

  41. A. Chambolle, V. Caselles, D. Cremers, M. Novaga, and T. Pock. An introduction to total variation for image analysis. In Theoretical Foundations and Numerical Methods for Sparse Recovery, volume 9 of Radon Series Compl. Appl. Math., pages 263–340. Walter de Gruyter, 2010.

    Google Scholar 

  42. A. Chambolle and T. Pock. Diagonal preconditioning for first order primal-dual algorithms in convex optimization. ICCV, pages 1762–1769, 2011.

    Google Scholar 

  43. A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision, 40(1):120–145, 2011.

    Article  MATH  Google Scholar 

  44. T. F. Chan and R. Glowinski. Finite element approximation and iterative solution of a class of mildly non-linear elliptic equations. Technical report, STAN-CS-78-674, Stanford University, 1978.

    Google Scholar 

  45. R. Chartrand, E. Y. Sidky, and X. Pan. Nonconvex compressive sensing for X-ray CT: an algorithm comparison. In Asilomar Conference on Signals, Systems, and Computers, 2013.

    Google Scholar 

  46. C. Chaux, M. EI-Gheche, J. Farah, J. Pesquet, and B. Popescu. A parallel proximal splitting method for disparity estimation from multicomponent images under illumination variation. Journal of Mathematical Imaging and Vision, 47(3):1–12, 2012.

    Google Scholar 

  47. C. H. Chen, B. S. He, Y. Y. Ye, and X. M. Yuan. The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Mathematical Programming, to appear.

    Google Scholar 

  48. G. Chen and M. Teboulle. A proximal-based decomposition method for convex minimization problems. Mathematical Programming, 64:81–101, 1994.

    Article  MATH  Google Scholar 

  49. G. H.-G. Chen and R. T. Rockafellar. Convergence rates in forward-backward splitting. SIAM Journal on Optimization, 7:421–444, 1997.

    Article  MATH  Google Scholar 

  50. G. Chierchia, N. Pustelnik, J.-C. Pesquet, and B. Pesquet-Popescu. Epigraphical projection and proximal tools for solving constrained convex optimization problems: Part I. arXiv preprint arXiv:1210.5844 (2014).

    Google Scholar 

  51. K. Choi, J. Wang, L. Zhu, T.-S. Suh, S. Boyd, and L. Xing. Compressed sensing based cone-beam computed tomography reconstruction with a first-order method. Med. Phys., 37(9):5113–5125, 2010.

    Article  Google Scholar 

  52. E. Chouzenoux, J.-C. Pesquet, and A. Repetti. Variable metric forward-backward algorithm for minimizing the sum of a differentiable function and a convex function. J. Optim. Theory Appl., 2013.

    Google Scholar 

  53. P. Combettes and J.-C. Pesquet. Proximal splitting methods in signal processing. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering, pages 185–212. Springer, 2011.

    Google Scholar 

  54. P. Combettes and J.-C. Pesquet. Primal-dual splitting algorithm for solving inclusions with mixture of composite, Lipschitzian, and parallel-sum type monotone operators. Set-Valued and Variational Analysis, 20(2):307–330, 2012.

    Article  MATH  Google Scholar 

  55. P. L. Combettes. Solving monotone inclusions via compositions of nonexpansive averaged operators. Optimization, 53(5–6):475–504, 2004.

    Article  MATH  Google Scholar 

  56. P. L. Combettes and J.-C. Pesquet. Proximal thresholding algorithm for minimization over orthonormal bases. SIAM Journal on Optimization, 18(4):1351–1376, 2007.

    Article  MATH  Google Scholar 

  57. P. L. Combettes and B. C. Vu. Variable metric forward-backward splitting with applications to monotone inclusions in duality. Optimization, pages 1–30, 2012.

    Google Scholar 

  58. L. Condat. A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl., 158(2):460–479, 2013.

    Article  MATH  Google Scholar 

  59. W. Cong, J. Yang, and G. Wang. Differential phase-contrast interior tomography. Phys. Med. Biol., 57:2905–2914, 2012.

    Article  Google Scholar 

  60. J. Dahl, P. J. Hansen, S. H. Jensen, and T. L. Jensen. Algorithms and software for total variation image reconstruction via first order methods. Numerical Algorithms, 53:67, 2010.

    Article  MATH  Google Scholar 

  61. I. Daubechies, M. Defrise, and C. De Mol. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics, 51:1413–1541, 2004.

    Article  MATH  Google Scholar 

  62. I. Daubechies, M. Fornasier, and I. Loris. Accelerated projected gradient methods for linear inverse problems with sparsity constraints. The Journal of Fourier Analysis and Applications, 14(5–6):764–792, 2008.

    Article  MATH  Google Scholar 

  63. C. Davis. All convex invariant functions of Hermitian matrices. Archive in Mathematics, 8:276–278, 1957.

    Article  MATH  Google Scholar 

  64. D. Davis and W. Yin. Convergence rate analysis of several splitting schemes. In: R. Glowinski, S. Osher, W. Yin (eds.) Splitting Methods in Communication and Imaging, Science and Engineering. Springer, 2016.

    Google Scholar 

  65. D. Davis and W. Yin. Faster convergence rates of relaxed Peaceman-Rachford and ADMM under regularity assumptions. ArXiv Preprint 1407.5210, 2014.

    Google Scholar 

  66. N. Dey, L. Blanc-Feraud, C. Zimmer, P. Roux, Z. Kam, J.-C. Olivo-Marin, and J. Zerubia. Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution. Microsc. Res. Tech., 69:260–266, 2006.

    Article  Google Scholar 

  67. J. Duchi, S. Shalev-Shwartz, Y. Singer, and T. Chandra. Efficient projections onto the l1-ball for learning in high dimensions. In ICML ’08 Proceedings of the 25th International Conference on Machine Learning, ACM New York, 2008.

    Google Scholar 

  68. J. Eckstein and D. P. Bertsekas. An alternating direction method for linear programming. Tech. Report MIT Lab. for Info. and Dec. Sys., 1990.

    Google Scholar 

  69. J. Eckstein and D. P. Bertsekas. On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming, 55:293–318, 1992.

    Article  MATH  Google Scholar 

  70. E. Esser. Applications of Lagrangian-based alternating direction methods and connections to split Bregman. Technical report, UCLA Computational and Applied Mathematics, March 2009.

    Google Scholar 

  71. E. Esser, X. Zhang, and T. F. Chan. A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J Imag Sci, 3(4):1015–1046, 2010.

    Article  MATH  Google Scholar 

  72. F. Facchinei and J.-S. Pang. Finite-Dimensional Variational Inequalities and Complementarity Problems, volume II. Springer, New York, 2003.

    MATH  Google Scholar 

  73. J. A. Fessler. Conjugate-gradient preconditioning methods: Numerical results. Technical Report 303, Commun. Signal Process. Lab., Dept. Elect. Eng. Comput. Sci., Univ. Michigan, Ann Arbor, MI, Jan. 1997. available from http://web.eecs.umich.edu/~fessler/.

  74. J. A. Fessler and S. D. Booth. Conjugate-gradient preconditioning methods for shift-variant PET image reconstruction. IEEE Trans. Image Process., 8(5):688–699, 1999.

    Article  MATH  Google Scholar 

  75. S. Feuerlein, E. Roessl, R. Proksa, G. Martens, O. Klass, M. Jeltsch, V. Rasche, H.-J. Brambs, M. H. K. Hoffmann, and J.-P. Schlomka. Multienergy photon-counting K-edge imaging: Potential for improved luminal depiction in vascular imaging. Radiology, 249(3):1010–1016, 2008.

    Article  Google Scholar 

  76. M. Figueiredo and J. Bioucas-Dias. Deconvolution of Poissonian images using variable splitting and augmented Lagrangian optimization. In IEEE Workshop on Statistical Signal Processing, Cardiff, 2009.

    Google Scholar 

  77. T. G. Flohr, C. H. McCollough, H. Bruder, M. Petersilka, K. Gruber, C. Süß, M. Grasruck, K. Stierstorfer, B. Krauss, R. Raupach, A. N. Primak, A. Küttner, S. Achenbach, C. Becker, A. Kopp, and B. M. Ohnesorge. First performance evaluation of a dual-source CT (DSCT) system. Eur. Radiol., 16:256–268, 2006.

    Article  Google Scholar 

  78. M. Fornasier. Theoretical Foundations and Numerical Methods for Sparse Recovery, volume 9. Walter de Gruyter, 2010.

    Book  MATH  Google Scholar 

  79. G. Frassoldati, L. Zanni, and G. Zanghirati. New adaptive stepsize selections in gradient methods. Journal of Industrial and Management Optimization, 4(2):299–312, 2008.

    Article  MATH  Google Scholar 

  80. M. Freiberger, C. Clason, and H. Scharfetter. Total variation regularization for nonlinear fluorescence tomography with an augmented Lagrangian splitting approach. Appl. Opt., 49(19):3741–3747, 2010.

    Article  Google Scholar 

  81. K. Frick, P. Marnitz, and A. Munk. Statistical multiresolution estimation for variational imaging: With an application in Poisson-biophotonics. J. Math. Imaging Vis., 46:370–387, 2013.

    Article  MATH  Google Scholar 

  82. D. Gabay. Applications of the method of multipliers to variational inequalities. In M. Fortin and R. Glowinski, editors, Augmented Lagrangian Methods: Applications to the Solution of Boundary Value Problems, chapter IX, pages 299–340. North-Holland, Amsterdam, 1983.

    Chapter  Google Scholar 

  83. D. Gabay and B. Mercier. A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Computer and Mathematics with Applications, 2:17–40, 1976.

    Article  MATH  Google Scholar 

  84. H. Gao, S. Osher, and H. Zhao. Mathematical Modeling in Biomedical Imaging II: Optical, Ultrasound, and Opto-Acoustic Tomographies, chapter Quantitative Photoacoustic Tomography, pages 131–158. Springer, 2012.

    Google Scholar 

  85. H. Gao, H. Yu, S. Osher, and G. Wang. Multi-energy CT based on a prior rank, intensity and sparsity model (PRISM). Inverse Problems, 27(11):115012, 2011.

    Google Scholar 

  86. R. Glowinski. On alternating direction methods of multipliers: a historical perspective. In Modeling, Simulation and Optimization for Science and Technology, pages 59–82. Springer, 2014.

    Google Scholar 

  87. R. Glowinski and P. Le Tallec. Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, volume 9 of SIAM Studies in Applied and Numerical Mathematics. SIAM, Philadelphia, 1989.

    Google Scholar 

  88. R. Glowinski and A. Marroco. Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue française d’automatique, informatique, recherche opérationnelle. Analyse numérique, 9(2):41–76, 1975.

    MATH  Google Scholar 

  89. D. Goldfarb and K. Scheinberg. Fast first-order methods for composite convex optimization with line search. SIAM Journal on Imaging Sciences, 2011.

    Google Scholar 

  90. T. Goldstein, X. Bresson, and S. Osher. Geometric applications of the split Bregman method: Segmentation and surface reconstruction. J. Sci. Comput., 45:272–293, 2010.

    Article  MATH  Google Scholar 

  91. T. Goldstein and S. Osher. The split Bregman method for L1-regularized problems. SIAM Journal on Imaging Sciences, 2(2):323–343, 2009.

    Article  MATH  Google Scholar 

  92. B. Goris, W. Van den Broek, K. J. Batenburg, H. H. Mezerji, and S. Bals. Electron tomography based on a total variation minimization reconstruction technique. Ultramicroscopy, 113:120–130, 2012.

    Article  Google Scholar 

  93. O. Güler. New proximal point algorithms for convex minimization. SIAM J. Optim., 2(4):649–664, 1992.

    Article  MATH  Google Scholar 

  94. S. Harizanov, J.-C. Pesquet, and G. Steidl. Epigraphical projection for solving least squares Anscombe transformed constrained optimization problems. In A. K. et al., editor, Scale-Space and Variational Methods in Computer Vision. Lecture Notes in Computer Science, SSVM 2013, LNCS 7893, pages 125–136, Berlin, 2013. Springer.

    Google Scholar 

  95. B. He, L.-Z. Liao, D. Han, and H. Yang. A new inexact alternating directions method for monotone variational inequalities. Math. Program., Ser. A, 92(1):103–118, 2002.

    Article  MATH  Google Scholar 

  96. B. He and H. Yang. Some convergence properties of the method of multiplieres for linearly constrained monotone variational operators. Operation Research Letters, 23:151–161, 1998.

    Article  Google Scholar 

  97. B. He and X. Yuan. On the \(\mathcal{O}(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM Journal on Numerical Analysis, 2:700–709, 2012.

    Article  MATH  Google Scholar 

  98. B. S. He, H. Yang, and S. L. Wang. Alternating direction method with self-adaptive penalty parameters for monotone variational inequalties. J. Optimiz. Theory App., 106(2):337–356, 2000.

    Article  MATH  Google Scholar 

  99. S. W. Hell. Toward fluorescence nanoscopy. Nat. Biotechnol., 21(11):1347–1355, 2003.

    Article  Google Scholar 

  100. S. W. Hell. Far-field optical nanoscopy. Science, 316(5828):1153–1158, 2007.

    Article  Google Scholar 

  101. M. R. Hestenes. Multiplier and gradient methods. Journal of Optimization Theory and Applications, 4:303–320, 1969.

    Article  MATH  Google Scholar 

  102. M. Hong and Z. Q. Luo. On linear convergence of the alternating direction method of multipliers. Arxiv preprint 1208.3922, 2012.

    Google Scholar 

  103. J. Huang, S. Zhang, and D. Metaxas. Efficient MR image reconstruction for compressed MR imaging. Med. Image Anal., 15:670–679, 2011.

    Article  Google Scholar 

  104. X. Jia, Y. Lou, R. Li, W. Y. Song, and S. B. Jiang. GPU-based fast cone beam CT reconstruction from undersampled and noisy projection data via total variation. Med. Phys., 37(4):1757–1760, 2010.

    Article  Google Scholar 

  105. B. Kaltenbacher, A. Neubauer, and O. Scherzer. Iterative regularization methods for nonlinear ill-posed problems, volume 6. Walter de Gruyter, 2008.

    Book  MATH  Google Scholar 

  106. S. H. Kang, B. Shafei, and G. Steidl. Supervised and transductive multi-class segmentation using p-Laplacians and RKHS methods. J. Visual Communication and Image Representation, 25(5):1136–1148, 2014.

    Article  Google Scholar 

  107. K. C. Kiwiel. Free-steering relaxation methods for problems with strictly convex costs and linear constraints. Mathematics of Operations Research, 22(2):326–349, 1997.

    Article  MATH  Google Scholar 

  108. K. C. Kiwiel. Proximal minimization methods with generalized Bregman functions. SIAM Journal on Control and Optimization, 35(4):1142–1168, 1997.

    Article  MATH  Google Scholar 

  109. G. F. Knoll. Radiation Detection and Measurement. Wiley, 3rd edition, 2000.

    Google Scholar 

  110. N. Komodakis and J.-C. Pesquet. Playing with duality: an overview of recent primal-dual approaches for solving large-scale optimization problems. ArXiv Preprint arXiv:1406.5429, 2014.

    Google Scholar 

  111. S. Kontogiorgis and R. R. Meyer. A variable-penalty alternating directions method for convex optimization. Math. Program., 83(1–3):29–53, 1998.

    MATH  Google Scholar 

  112. M. A. Krasnoselskii. Two observations about the method of successive approximations. Uspekhi Matematicheskikh Nauk, 10:123–127, 1955. In Russian.

    Google Scholar 

  113. R. A. Kruger, S. J. Riederer, and C. A. Mistretta. Relative properties of tomography, K-edge imaging, and K-edge tomography. Med. Phys., 4(3):244–249, 1977.

    Article  Google Scholar 

  114. M.-J. Lai and W. Yin. Augmented ℓ 1 and nuclear-norm models with a globally linearly convergent algorithm. SIAM Journal on Imaging Sciences, 6(2):1059–1091, 2013.

    Article  MATH  Google Scholar 

  115. J. Lellmann, J. Kappes, J. Yuan, F. Becker, and C. Schnörr. Convex multi-class image labeling with simplex-constrained total variation. In X.-C. Tai, K. Morken, M. Lysaker, and K.-A. Lie, editors, Scale Space and Variational Methods, volume 5567 of LNCS, volume 5567 of Lecture Notes in Computer Science, pages 150–162. Springer, 2009.

    Google Scholar 

  116. P. L. Lions and B. Mercier. Splitting algorithms for the sum of two linear operators. SIAM Journal on Numerical Analysis, 16:964–976, 1979.

    Article  MATH  Google Scholar 

  117. M. Lustig, D. Donoho, and J. M. Pauly. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med., 58:1182–1195, 2007.

    Article  Google Scholar 

  118. S. Ma, W. Y. Y. Zhang, and A. Chakraborty. An efficient algorithm for compressed MR imaging using total variation and wavelets. In Proc. IEEE Comput. Vision Pattern Recognit., 2008.

    Google Scholar 

  119. P. Machart, S. Anthoine, and L. Baldassarre. Optimal computational trade-off of inexact proximal methods. arXiv preprint arXiv:1210.5034, 2012.

    Google Scholar 

  120. W. R. Mann. Mean value methods in iteration. Proceedings of the American Mathematical Society, 16(4):506–510, 1953.

    Article  MATH  Google Scholar 

  121. B. Martinet. Régularisation d’inéquations variationnelles par approximations successives. Revue Française d’lnformatique et de Recherche Operationelle, 4(3):154–158, 1970.

    MATH  Google Scholar 

  122. A. Mehranian, A. Rahmim, M. R. Ay, F. Kotasidis, and H. Zaidi. An ordered-subsets proximal preconditioned gradient algorithm for edge-preserving PET image reconstruction. Med. Phys., 40(5):052503, 2013.

    Google Scholar 

  123. J. Müller, C. Brune, A. Sawatzky, T. Kösters, F. Wübbeling, K. Schäfers, and M. Burger. Reconstruction of short time PET scans using Bregman iterations. In Proc. IEEE Nucl. Sci. Symp. Conf. Rec., 2011.

    Google Scholar 

  124. F. Natterer and F. Wübbeling. Mathematical Methods in Image Reconstruction. SIAM, 2001.

    Book  MATH  Google Scholar 

  125. A. S. Nemirovsky and D. B. Yudin. Problem Complexity and Method Efficiency in Optimization. J. Wiley & Sons, Ltd., 1983.

    Google Scholar 

  126. Y. Nesterov. Introductory Lectures on Convex Optimization - A Basic Course, volume 87 of Applied Optimization. Springer US, 2004.

    Book  MATH  Google Scholar 

  127. Y. Nesterov. Gradient methods for minimizing composite functions. Math. Program., Series B, 140(1):125–161, 2013.

    Article  MATH  Google Scholar 

  128. Y. E. Nesterov. A method of solving a convex programming problem with convergence rate O(1∕k 2). Soviet Mathematics Doklady, 27(2):372–376, 1983.

    MATH  Google Scholar 

  129. Y. E. Nesterov. Smooth minimization of non-smooth functions. Mathematical Programming, 103:127–152, 2005.

    Article  MATH  Google Scholar 

  130. H. Nien and J. A. Fessler. Fast X-ray CT image reconstruction using the linearized augmented Lagrangian method with ordered subsets. arXiv preprint arXiv:1402.4381, 2014.

    Google Scholar 

  131. M. Nilchian, C. Vonesch, P. Modregger, M. Stampanoni, and M. Unser. Fast iterative reconstruction of differential phase contrast X-ray tomograms. Optics Express, 21(5):5511–5528, 2013.

    Article  Google Scholar 

  132. P. Ochs, Y. Chen, T. Brox, and T. Pock. iPiano: Inertial proximal algorithm for nonconvex optimization. SIAM J Imaging Sci, 7(2):1388–1419, 2014.

    Article  MATH  Google Scholar 

  133. Z. Opial. Weak convergence of a sequence of successive approximations for nonexpansive mappings. Bulletin of the American Mathematical Society, 73:591–597, 1967.

    Article  MATH  Google Scholar 

  134. J. M. Ortega and W. C. Rheinboldt. Iterative Solution of Nonlinear Equations in Several Variables. SIAM, New York, 1970.

    MATH  Google Scholar 

  135. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin. An iterative regularization method for the total variation based image restoration. Multiscale Modeling and Simulation, 4:460–489, 2005.

    Article  MATH  Google Scholar 

  136. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin. An iterative regularization method for total variation-based image restoration. Multiscale Modeling & Simulation, 4(2):460–489, 2005.

    Article  MATH  Google Scholar 

  137. D. Pan, C. O. Schirra, A. Senpan, A. H. Schmieder, A. J. Stacy, E. Roessl, A. Thran, S. A. Wickline, R. Proksa, and G. M. Lanza. An early investigation of ytterbium nanocolloids for selective and quantitative “multicolor” spectral CT imaging. ACS Nano, 6(4):3364–3370, 2012.

    Article  Google Scholar 

  138. L. A. Parente, P. A. Lotito, and M. V. Solodov. A class of inexact variable metric proximal point algorithms. SIAM J. Optim., 19(1):240–260, 2008.

    Article  MATH  Google Scholar 

  139. N. Parikh and S. Boyd. Proximal algorithms. Foundations and Trends in Optimization, 1(3):123–231, 2013.

    Google Scholar 

  140. T. Pock, A. Chambolle, D. Cremers, and H. Bischof. A convex relaxation approach for computing minimal partitions. IEEE Conference on Computer Vision and Pattern Recognition, pages 810–817, 2009.

    Google Scholar 

  141. M. J. D. Powell. A method for nonlinear constraints in minimization problems. Optimization, 1972.

    Google Scholar 

  142. K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger. SENSE: Sensitivity encoding for fast MRI. Magn. Reson. Med., 42:952–962, 1999.

    Article  Google Scholar 

  143. N. Pustelnik, C. Chaux, J.-C. Pesquet, and C. Comtat. Parallel algorithm and hybrid regularization for dynamic PET reconstruction. In Proc. IEEE Nucl. Sci. Symp. Conf. Rec., 2010.

    Google Scholar 

  144. S. Ramani and J. A. Fessler. Parallel MR image reconstruction using augmented Lagrangian methods. IEEE Trans. Med. Imag., 30(3):694–706, 2011.

    Article  Google Scholar 

  145. S. Ramani and J. A. Fessler. A splitting-based iterative algorithm for accelerated statistical X-ray CT reconstruction. IEEE Trans. Med. Imag., 31(3):677–688, 2012.

    Article  Google Scholar 

  146. R. T. Rockafellar. Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. Oper. Res., 1(2):97–116, 1976.

    Article  MATH  Google Scholar 

  147. R. T. Rockafellar. Monotone operators and the proximal point algorithm. SIAM Journal on Control and Optimization, 14:877–898, 1976.

    Article  MATH  Google Scholar 

  148. R. T. Rockafellar. Convex Analysis. Princeton University Press, Princeton, 10 edition, 1997.

    MATH  Google Scholar 

  149. E. Roessl and C. Herrmann. Cramér-Rao lower bound of basis image noise in multiple-energy x-ray imaging. Phys. Med. Biol., 54(5):1307–1318, 2009.

    Article  Google Scholar 

  150. E. Roessl and R. Proksa. K-edge imaging in x-ray computed tomography using multi-bin photon counting detectors. Phys. Med. Biol., 52(15):4679–4696, 2007.

    Article  Google Scholar 

  151. L. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60:259–268, 1992.

    Article  MATH  Google Scholar 

  152. A. Sawatzky. Performance of first-order algorithms for TV penalized weighted least-squares denoising problem. In Image and Signal Processing, volume 8509 of Lecture Notes in Computer Science, pages 340–349. Springer International Publishing, 2014.

    Google Scholar 

  153. A. Sawatzky, C. Brune, T. Kösters, F. Wübbeling, and M. Burger. EM-TV methods for inverse problems with Poisson noise. In Level Set and PDE Based Reconstruction Methods in Imaging, pages 71–142. Springer, 2013.

    Google Scholar 

  154. A. Sawatzky, D. Tenbrinck, X. Jiang, and M. Burger. A variational framework for region-based segmentation incorporating physical noise models. J. Math. Imaging Vis., 47(3):179–209, 2013.

    Article  MATH  Google Scholar 

  155. A. Sawatzky, Q. Xu, C. O. Schirra, and M. A. Anastasio. Proximal ADMM for multi-channel image reconstruction in spectral X-ray CT. IEEE Trans. Med. Imag., 33(8):1657–1668, 2014.

    Article  Google Scholar 

  156. H. Schäfer. Über die Methode sukzessiver Approximationen. Jahresbericht der Deutschen Mathematiker-Vereinigung, 59:131–140, 1957.

    MATH  Google Scholar 

  157. K. P. Schäfers, T. J. Spinks, P. G. Camici, P. M. Bloomfield, C. G. Rhodes, M. P. Law, C. S. R. Baker, and O. Rimoldi. Absolute quantification of myocardial blood flow with H2 15O and 3-dimensional PET: An experimental validation. J. Nucl. Med., 43:1031–1040, 2002.

    Google Scholar 

  158. C. O. Schirra, B. Brendel, M. A. Anastasio, and E. Roessl. Spectral CT: a technology primer for contrast agent development. Contrast Media Mol. Imaging, 9(1):62–70, 2014.

    Article  Google Scholar 

  159. C. O. Schirra, E. Roessl, T. Koehler, B. Brendel, A. Thran, D. Pan, M. A. Anastasio, and R. Proksa. Statistical reconstruction of material decomposed data in spectral CT. IEEE Trans. Med. Imag., 32(7):1249–1257, 2013.

    Article  Google Scholar 

  160. J. P. Schlomka, E. Roessl, R. Dorscheid, S. Dill, G. Martens, T. Istel, C. Bäumer, C. Herrmann, R. Steadmann, G. Zeitler, A. Livne, and R. Proksa. Experimental feasibility of multi-energy photon-counting K-edge imaging in pre-clinical computed tomography. Phys. Med. Biol., 53(15):4031–4047, 2008.

    Article  Google Scholar 

  161. M. Schmidt, N. Le Roux, and F. Bach. Convergence rates of inexact proximal-gradient methods for convex optimization. Technical report, arXiv e-print, 2011. http://arxiv.org/abs/1109.2415.

  162. M. Schrader, S. W. Hell, and H. T. M. van der Voort. Three-dimensional super-resolution with a 4Pi-confocal microscope using image restoration. J. Appl. Phys., 84(8):4033–4042, 1998.

    Article  Google Scholar 

  163. T. Schuster, B. Kaltenbacher, B. Hofmann, and K. S. Kazimierski. Regularization methods in Banach spaces, volume 10. Walter de Gruyter, 2012.

    Book  MATH  Google Scholar 

  164. S. Setzer. Operator splittings, Bregman methods and frame shrinkage in image processing. International Journal of Computer Vision, 92(3):265–280, 2011.

    Article  MATH  Google Scholar 

  165. S. Setzer, G. Steidl, and J. Morgenthaler. A cyclic projected gradient method. Computational Optimization and Applications, 54(2):417–440, 2013.

    Article  MATH  Google Scholar 

  166. S. Setzer, G. Steidl, and T. Teuber. Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image R., 21:193–199, 2010.

    Article  Google Scholar 

  167. E. Y. Sidky, J. H. Jörgensen, and X. Pan. Convex optimization problem prototyping for image reconstruction in computed tomography with the Chambolle-Pock algorithm. Phys. Med. Biol., 57(10):3065–3091, 2012.

    Article  Google Scholar 

  168. G. Steidl and T. Teuber. Removing multiplicative noise by Douglas-Rachford splitting methods. J. Math. Imaging Vis., 36:168–184, 2010.

    Article  MATH  Google Scholar 

  169. T. Teuber. Anisotropic Smoothing and Image Restoration Facing Non-Gaussian Noise. PhD thesis, Technische Universität Kaiserslautern, Apr. 2012. available from https://kluedo.ub.uni-kl.de/frontdoor/index/index/docId/3219.

  170. P. Tseng. Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM Journal on Control and Optimization, 29:119–138, 1991.

    Article  MATH  Google Scholar 

  171. P. Tseng. On accelerated proximal gradient methods for convex-concave optimization. Technical report, Dept. of Mathematics, University of Washington, Seattle, 2008.

    Google Scholar 

  172. T. Valkonen. A primal-dual hybrid gradient method for nonlinear operators with applications to MRI. Inverse Problems, 30(5):055012, 2014.

    Google Scholar 

  173. B. Vandeghinste, B. Goossens, J. D. Beenhouwer, A. Pizurica, W. Philips, S. Vandenberghe, and S. Staelens. Split-Bregman-based sparse-view CT reconstruction. Proc. Int. Meeting Fully 3D Image Recon. Rad. Nucl. Med., pages 431–434, 2011.

    Google Scholar 

  174. Y. Vardi, L. A. Shepp, and L. Kaufman. A statistical model for positron emission tomography. J. Am. Stat. Assoc., 80(389):8–20, 1985.

    Article  MATH  Google Scholar 

  175. S. Villa, S. Salzo, L. Baldassarre, and A. Verri. Accelerated and inexact forward-backward algorithms. SIAM J. Optim., 23(3):1607–1633, 2013.

    Article  MATH  Google Scholar 

  176. J. von Neumann. Some matrix inequalities and metrization of matrix-space. In Collected Works, Pergamon, Oxford, 1962, Volume IV, 205–218, pages 286–300. Tomsk University Review, 1937.

    Google Scholar 

  177. B. C. Vũ. A splitting algorithm for dual monotone inclusions involving cocoercive operators. Advances in Computational Mathematics, 38(3):667–681, 2013.

    Article  MATH  Google Scholar 

  178. G. Wang, H. Yu, and B. D. Man. An outlook on x-ray CT research and development. Med. Phys., 35(3):1051–1064, 2008.

    Article  Google Scholar 

  179. J. Wang, T. Li, H. Lu, and Z. Liang. Penalized weighted least-squares approach to sinogram noise reduction and image reconstruction for low-dose X-ray computed tomography. IEEE Trans. Med. Imag., 25(10):1272–1283, 2006.

    Article  Google Scholar 

  180. K. Wang, R. Su, A. A. Oraevsky, and M. A. Anastasio. Investigation of iterative image reconstruction in three-dimensional optoacoustic tomography. Phys. Med. Biol., 57:5399–5423, 2012.

    Article  Google Scholar 

  181. S. L. Wang and L. Z. Liao. Decomposition method with a variable parameter for a class of monotone variational inequality problems. J. Optimiz. Theory App., 109(2):415–429, 2001.

    Article  MATH  Google Scholar 

  182. G. A. Watson. Characterization of the subdifferential of some matrix norms. Linear Algebra and its Applications, 170:33–45, 1992.

    Article  MATH  Google Scholar 

  183. M. N. Wernick and J. N. Aarsvold, editors. Emission Tomography: The Fundamentals of PET and SPECT. Elsevier Academic Press, 2004.

    Google Scholar 

  184. Q. Xu, A. Sawatzky, and M. A. Anastasio. A multi-channel image reconstruction method for grating-based X-ray phase-contrast computed tomography. In Proc. SPIE 9033, Medical Imaging 2014: Physics of Medical Imaging, 2014.

    Google Scholar 

  185. Q. Xu, A. Sawatzky, M. A. Anastasio, and C. O. Schirra. Sparsity-regularized image reconstruction of decomposed K-edge data in spectral CT. Phys. Med. Biol., 59(10):N65, 2014.

    Google Scholar 

  186. M. Yan and W. Yin. Self equivalence of the alternating direction method of multipliers. In: R. Glowinski, S. Osher, W. Yin (eds.) Splitting Methods in Communication and Imaging, Science and Engineering. Springer, 2016.

    Google Scholar 

  187. W. Yin. Analysis and generalizations of the linearized Bregman method. SIAM Journal on Imaging Sciences, 3(4):856–877, 2010.

    Article  MATH  Google Scholar 

  188. J. Yuan, C. Schnörr, and G. Steidl. Simultaneous higher order optical flow estimation and decomposition. SIAM Journal on Scientific Computing, 29(6):2283–2304, 2007.

    Article  MATH  Google Scholar 

  189. R. Zhang, J.-B. Thibault, C. A. Bouman, and K. D. S. J. Hsieh. A model-based iterative algorithm for dual-energy X-ray CT reconstruction. In Proc. Int. Conf. Image Form. in X-ray CT, pages 439–443, 2012.

    Google Scholar 

  190. X. Zhang, M. Burger, and S. Osher. A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput., 46(1):20–46, 2011.

    Article  MATH  Google Scholar 

  191. M. Zhu and T. F. Chan. An efficient primal-dual hybrid gradient algorithm for total variation image restoration. UCLA CAM Report 08-34, 2008.

    Google Scholar 

  192. H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of Royal Statistical Society: Series B (Statistical Methodology), 67:301–320, 2005.

    Article  MATH  Google Scholar 

  193. Y. Zou and M. D. Silver. Analysis of fast kV-switching in dual energy CT using a pre-reconstruction decomposition technique. In Proc. SPIE (Medical Imaging 2008), volume 6913, page 691313, 2008.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Computer Science, University of Münster, 48149, Münster, Germany

    M. Burger & A. Sawatzky

  2. Department of Mathematics, University of Kaiserslautern, Paul Ehrlich Str. 31, 67653, Kaiserslautern, Germany

    G. Steidl

Authors
  1. M. Burger
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Sawatzky
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. Steidl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Burger .

Editor information

Editors and Affiliations

  1. Department of Mathematics, University of Houston, Houston, Texas, USA

    Roland Glowinski

  2. Department of Mathematics, UCLA, Los Angeles, California, USA

    Stanley J. Osher

  3. Department of Mathematics, UCLA, Los Angeles, California, USA

    Wotao Yin

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Burger, M., Sawatzky, A., Steidl, G. (2016). First Order Algorithms in Variational Image Processing. In: Glowinski, R., Osher, S., Yin, W. (eds) Splitting Methods in Communication, Imaging, Science, and Engineering. Scientific Computation. Springer, Cham. https://doi.org/10.1007/978-3-319-41589-5_10

Download citation

Publish with us

Policies and ethics

Search

Navigation