Abstract
Discrete support vector machines are models for classification recently introduced in the context of statistical learning theory. Their distinctive feature is the formulation of mixed integer programming problems aimed at deriving optimal separating hyperplanes with minimum empirical error and maximum generalization capability. A new family of discrete SVM is proposed in this paper, for which the hyperplane establishes a variable softening of the margin to improve the separation among distinct classes. Theoretical bounds are derived to finely tune the parameters of the optimization problem. Computational tests on benchmark datasets in the biolife science application domain indicate the effectiveness of the proposed approach, that appears dominating against traditional SVM in terms of accuracy and percentage of support vectors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Allwein, E., Schapire, R., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)
Andreeva, A., Howorth, D., Brenner, S.E., Hubbard, T.J., Chothia, C., Murzin, A.G.: SCOP database in 2004: refinements integrate structure and sequence family data. Nucleic Acids Res. 32, D226–D229 (2004)
Burges, C.J.C.: A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery 2, 121–167 (1998)
Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines (2001)
Cristianini, N., Shawe-Taylor, J.: An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge (2000)
Ding, C.H., Dubchak, I.: Multi-class protein fold recognition using support vector machines and neural networks. Bioinformatics 17, 349–358 (2001)
Dubchak, I., Muchnik, I., Mayor, C., Dralyuk, I., Kim, S.H.: Recognition of a protein fold in the context of the Structural Classification of Proteins (SCOP) classification. Proteins 35, 401–407 (1999)
Hettich, S., Blake, C., Merz, C.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Koehler, G.J., Erenguc, S.: Minimizing misclassifications in linear discriminant analysis. Decision Sciences 21, 63–85 (1990)
Kohavi, R.: A study of cross-validation and bootstrapping for accuracy estimation and model selection. In: Proc. of the 14th International Joint Conference on Artificial Intelligence, pp. 338–345. Morgan Kaufmann, San Francisco (1995)
Mangasarian, O.L.: Linear and nonlinear separation of patterns by linear programming. Operations Research 13, 444–452 (1965)
Murzin, A.G., Brenner, S.E., Hubbard, T., Chothia, C.: SCOP: a structural classification of protein database for the investigation of sequence and structures. J. Mol. Biol. 247, 536–540 (1995)
Orsenigo, C., Vercellis, C.: Multivariate classification trees based on minimum features discrete support vector machines. IMA Journal of Management Mathematics 14, 221–234 (2003)
Orsenigo, C., Vercellis, C.: Discrete support vector decision trees via tabu-search. Journal of Computational Statistics and Data Analysis 47, 311–322 (2004)
Orsenigo, C., Vercellis, C.: Rule induction through discrete support vector decision trees. In: Triantaphyllou, E., Felici, G. (eds.) Data Mining and Knowledge Discovery Approaches Based on Rule Induction Techniques, pp. 305–325. Springer, Heidelberg (2006)
Orsenigo, C., Vercellis, C.: Multicategory classification via discrete support vector machines. Computational Management Science (in press, 2007a)
Orsenigo, C., Vercellis, C.: Accurately learning from few examples with a polyhedral classifier. Computational Optimization and Applications (in press, 2007b)
Orsenigo, C., Vercellis, C.: Protein folding classification through multicategory discrete SVM. In: Felici, G., Vercellis, C. (eds.) Mathematical Methods for Knowledge Discovery and Data Mining, Idea Group, USA (in press, 2007c)
Schölkopf, B., Smola, A.J.: Learning with kernels. Support vector machines, regularization, optimization and beyond. MIT Press, Cambridge (2002)
Vapnik, V.: The nature of statistical learning theory. Springer, Heidelberg (1995)
Vapnik, V.: Statistical Learning Theory. Wiley, Chichester (1998)
Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M.: Use of the Zero-Norm with Linear Models and Kernel Methods. Journal of Machine Learning Research 3, 1439–1461 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Orsenigo, C., Vercellis, C. (2007). Softening the Margin in Discrete SVM. In: Perner, P. (eds) Advances in Data Mining. Theoretical Aspects and Applications. ICDM 2007. Lecture Notes in Computer Science(), vol 4597. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73435-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-73435-2_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73434-5
Online ISBN: 978-3-540-73435-2
eBook Packages: Computer ScienceComputer Science (R0)