Abstract
Improved twin support vector machine (ITSVM) is a binary classification model with strong theoretical interpretation. Compared with Twin bound support vector machine (TWBSVM), it avoids the matrix inverse operation. However, the disadvantage of slow speed is exposed during the training process. Thus authors are motivated to employ safe screening method to reduce the scale of dual ITSVM and accelerate its computational speed. More specifically, the proposed method is to identify redundant points in advance and eliminate them before actually solving the problem. If safe screening method is directly used to accelerate ITSVM, it will be inevitable to bring matrix inverse operation during the screening process. In this case, a screening method which is distinct from existing safe screening method is devised to avoid calculating inverse matrix. Meanwhile an improved dual coordinate descent method (DCDM) is employed to accelerate ITSVM. Experiments on eleven real data sets are conducted to demonstrate the effectiveness of the proposed acceleration method.
Similar content being viewed by others
References
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
Steinwart I, Christmann A (2008) Support vector machines. Springer, New York
Schölkop B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge
Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
Deng N, Tian Y, Zhang C (2013) Support vector machines: optimization based theory, algorithms, and extensions. Chapman and Hall/CRC, New York
Jayadeva RK, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910
Jayadeva RK, Chandra S (2017) Twin support vector machines. Springer, Cham
Brugger D (2006) Parallel support vector machines. Universität Tübingen, Tübingen
Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079
Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl Based Syst 43:74–81
Shao Y, Chen W, Wang Z, Li C, Deng N (2015) Weighted linear loss twin support vector machine for large-scale classification. Knowl Based Syst 73(1):276–288
Peng X, Xu D (2013) A twin-hypersphere support vector machine classifier and the fast learning algorithm. Inf Sci 221:12–27
Tomar D, Agarwal S (2015) Twin support vector machine: a review from 2007 to 2014. Egypt Inf J 16(1):55–69
Qi Z, Tian Y, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316
Cevikalp H (2017) Best fitting hyperplanes for classification. IEEE Trans Pattern Anal Mach Intell 39(6):1076–1088
Yan H, Ye Q, Zhang T, Yu D, Yuan X, Xu Y, Fu L (2018) Least squares twin bounded support vector machines based on L1-norm distance metric for classification. Pattern Recognit 74:434–447
Ye Q, Zhao H, Li Z, Yang X, Gao S, Yin T, Ye N (2018) L1-norm distance minimization-based fast robust twin support vector k-plane clustering. IEEE Trans Neural Netw Learn Syst 29:4494–4503
Xu Y, Yang Z, Pan X (2017) A novel twin support-vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370
Arun Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543
Yao X, Wang Z, Zhang H (2018) Identification method for a class of periodic discrete-time dynamic nonlinear systems based on Sinusoidal ESN. Neurocomputing 275:1511–1521
El Ghaoui L, Viallon V, Rabbani T (2010) Safe feature elimination in sparse supervised learning. Pac J Optim 8(4):667–698
Ogawa K, Suzuki Y, Takeuchi I (2013) Safe screening of mon-support vectors in pathwise SVM computation. In: International conference on machine learning, pp 1382–1390
Wang J, Zhou J, Wonka P, Ye J (2013) Lasso screening rules via dual polytope projection. In: Neural information processing systems, pp 1070–1078
Wang J, Wonka P, Ye J (2014) Scaling svm and least absolute deviations via exact data reduction. In: International conference on machine learning, pp 523–531
Wang J, Zhou J, Liu J, Wonka P, Ye J (2014) A safe screening rule for sparse logistic regression. In: Neural information processing systems, vol 2, pp 1053–1061
Yang T, Wang J, Sun Q, Hibar DP, Jahanshad N, Liu L, Wang Y, Zhan L, Thompson PM, Ye J (2015) Detecting genetic risk factors for Alzheimer's disease in whole genome sequence data via lasso screening. In: IEEE international symposium on biomedical imaging, pp 985–989
Pan X, Yang Z, Xu Y, Wang L (2018) Safe screening rules for accelerating twin support vector machine classification. IEEE Trans Neural Netw Learn Syst 29(5):1876–1887
Yang Z, Xu Y (2018) A safe screening rule for Laplacian support vector machine. Eng Appl Artif Intell 67:309–316
Pan X, Pang X, Wang H, Xu Y (2018) A safe screening based framework for support vector regression. Neurocomputing 287:163–172
Pang X, Xu C, Xu Y (2018) Scaling KNN multi-class twin support vector machine via safe instance reduction. Knowl Based Syst 148:17–30
Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw Learn Syst 22(6):962–968
Tian Y, Ju X, Qi Z, Shi Y (2014) Improved twin support vector machine. Sci China Math 57(2):417–432
Güler O (2010) Foundations of optimization. Springer, New York
Hsieh C, Chang K, Lin C, Keerthi SS, Sundararajan S (2008) A dual coordinate descent method for large-scale linear svm. In: International conference on machine learning, pp 408–415
Luo Z, Tseng P (1992) On the convergence of the coordinate descent method for convex differentiable minimization. J Optim Theory Appl 72(1):7–35
Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml/index.php
Acknowledgements
The authors would like to thank the reviewers for the helpful comments and suggestions, which have improved the presentation. This work was supported in part by National Natural Science Foundation of China (No. 11671010) and Beijing Natural Science Foundation (No.4172035).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wu, W., Xu, Y. Accelerating improved twin support vector machine with safe screening rule. Int. J. Mach. Learn. & Cyber. 10, 3587–3600 (2019). https://doi.org/10.1007/s13042-019-00946-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-019-00946-1