Abstract
We give a new proof of the convergence of the SMO algorithm for SVM training over linearly separable problems that partly builds on the one by Mitchell et al. for the convergence of the MDM algorithm to find the point of a convex set closest to the origin. Our proof relies in a simple derivation of SMO that we also present here and, while less general, it is considerably simpler than previous ones and yields algorithmic insights into the working of SMO.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods - Support Vector Machines, pp. 185–208 (1999)
Joachims, T.: Making large-scale support vector machine learning practical. In: Advances in Kernel Methods - Support Vector Machines, pp. 169–184 (1999)
Fan, R.-E., Chen, P.-H., Lin, C.-J.: Working set selection using second order information for training support vector machines. Journal of Machine Learning Research 6, 1889–1918 (2005)
Keerthi, S.S., Shevade, S.K., Bhattacharyya, C., Murthy, K.R.K.: Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Computation 13(3), 637–649 (2001)
Lin, C.-J.: On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks 12(6), 1288–1298 (2001)
Lin, C.-J.: Asymptotic convergence of an SMO algorithm without any assumptions. IEEE Transactions on Neural Networks 13(1), 248–250 (2002)
Chen, P.-H., Fan, R.-E., Lin, C.-J.: A study on SMO-type decomposition methods for support vector machines. IEEE Transactions on Neural Networks 17, 893–908 (2006)
Mitchell, B.F., Dem’yanov, V.F., Malozemov, V.N.: Finding the point of a polyhedron closest to the origin. SIAM J. Contr. 12, 19–26 (1974)
Bennett, K.P., Bredensteiner, E.J.: Duality and geometry in SVM classifiers. In: Proceedings of the 17th International Conference on Machine Learning, pp. 57–64 (2000)
López, J., Barbero, Á., Dorronsoro, J.: On the equivalence of the SMO and MDM algorithms for SVM training. In: Machine Learning and Knowledge Discovery in Databases - ECML 2008 Proceedings, Part II. LNCS (LNAI). Springer, Heidelberg (2008)
Burges, C.J.C., Crisp, D.J.: Uniqueness theorems for kernel methods. Neurocomputing 55(1-2), 187–220 (2003)
Shawe-Taylor, J., Cristianini, N.: On the generalization of soft margin algorithms. IEEE Transactions on Information Theory 48(10), 2721–2735 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
López, J., Dorronsoro, J.R. (2009). A Simple Proof of the Convergence of the SMO Algorithm for Linearly Separable Problems. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds) Artificial Neural Networks – ICANN 2009. ICANN 2009. Lecture Notes in Computer Science, vol 5768. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04274-4_93
Download citation
DOI: https://doi.org/10.1007/978-3-642-04274-4_93
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04273-7
Online ISBN: 978-3-642-04274-4
eBook Packages: Computer ScienceComputer Science (R0)