Abstract
We study the connection between higher order total variation (TV) regularization and support vector regression (SVR) with spline kernels in a one-dimensional discrete setting. We prove that the contact problem arising in the tube formulation of the TV minimization problem is equivalent to the SVR problem. Since the SVR problem can be solved by standard quadratic programming methods this provides us with an algorithm for the solution of the contact problem even for higher order derivatives. Our numerical experiments illustrate the approach for various orders of derivatives and show its close relation to corresponding nonlinear diffusion and diffusion–reaction equations.
This joint research was supported by the Deutsche Forschungsgemeinschaft within the projects WE 2602/2-2 and SCHN 457/5-2. This is gratefully acknowledged.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill–posed problems. Inverse Problems 10, 1217–1229 (1994)
Chambolle, A.: An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20, 89–97 (2004)
Chan, T.F., Marquina, A., Mulet, P.: High-order total variation-based image restoration. SIAM Journal on Scientific Computing 22(2), 503–516 (2000)
Davies, P.L., Kovac, A.: Local extremes, runs, strings and multiresolution. Annals of Statistics 29, 1–65 (2001)
Didas, S.: Higher order variational methods for noise removal in signals and images. Diploma Thesis, Dept. of Math., Saarland University, Saarbrücken, Germany (2004)
Girosi, F.: An equivalence between sparse approximation and support vector machines. Neural Computation 10(6), 1455–1480 (1998)
Hinterberger, W., Hintermüller, M., Kunisch, K., von Oehsen, M., Scherzer, O.: Tube methods for BV regularization. Journal of Mathematical Imaging and Vision 19, 223–238 (2003)
Loog, M.: Support blob machines: The sparsification of linear scale-space. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 14–24. Springer, Heidelberg (2004)
Lysaker, M., Lundervold, A., Tai, X.-C.: Noise removal using fourth-order partial differential equations with applications to medical magnetic resonance images in space and time. Technical Report CAM-02-44, Department of Mathematics, University of California (2002)
Mammen, E., van de Geer, S.: Locally adaptive regression splines. Annals of Statistics 25(1), 387–413 (1997)
Mangasarian, O.L., Schumaker, L.L.: Best summation formulae and discrete splines via mathematical programming. SIAM Journal on Numerical Analysis 10(3), 448–459 (1973)
Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. University Lecture Series, vol. 22. AMS, Providence (2001)
Nielsen, M., Florack, L., Deriche, R.: Regularization, scale-space and edge detection filters. Journal of Mathematical Imaging and Vision 7, 291–307 (1997)
Obereder, A., Osher, S., Scherzer, O.: On the use of dual norms in bounded variation type regularization. Technical report, Department of Computer Science, University of Innsbruck (2004)
Pollak, I., Willsky, A.S., Huang, Y.: Nonlinear evolution equations as fast and exact solvers of estimation problems. IEEE Trans. on Signal Processing (in print)
Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)
Scherzer, O.: Denoising with higher order derivatives of bounded variation and an application to parameter estimation. Computing 60, 1–27 (1998)
Scherzer, O.: Taut string algorithm and regularization programs with G-norm. Journal of Mathematical Imaging and Vision (in print)
Scherzer, O., Weickert, J.: Relations between regularization and diffusion filtering. Journal of Mathematical Imaging and Vision 12, 43–63 (2000)
Steidl, G.: A note on the dual treatment of higher order regularization functionals. Technical Report 278, Institute of Mathematics, University of Mannheim (2004)
Steidl, G., Weickert, J., Brox, T., Mrázek, P., Welk, M.: On the equivalence of soft wavelet shrinkage, total variation diffusion, total variation regularization, and SIDEs. SIAM Journal on Numerical Analysis 42(2), 686–713 (2004)
Strong, D.M.: Adaptive Total Variation Minimizing Image Restoration. PhD thesis, Department of Mathematics, University of California (1997)
Vapnik, V.N.: Statistical Learning Theory. John Wiley and Sons, Inc., Chichester (1998)
Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)
Wahba, G.: Spline Models for Observational Data. SIAM, Philadelphia (1990)
You, Y.-L., Kaveh, M.: Fourth-order partial differential equations for noise removal. IEEE Transactions on Image Processing 9(10), 1723–1730 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Steidl, G., Didas, S., Neumann, J. (2005). Relations Between Higher Order TV Regularization and Support Vector Regression. In: Kimmel, R., Sochen, N.A., Weickert, J. (eds) Scale Space and PDE Methods in Computer Vision. Scale-Space 2005. Lecture Notes in Computer Science, vol 3459. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11408031_44
Download citation
DOI: https://doi.org/10.1007/11408031_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25547-5
Online ISBN: 978-3-540-32012-8
eBook Packages: Computer ScienceComputer Science (R0)