Abstract
The proximal support vector machine via generalized eigenvalues (GEPSVM) is an excellent classifier for binary classification problem. However, in conventional GEPSVM the distance is measured by L2-norm, which makes it prone to being affected by the presence of outliers by the square operation. To alleviate this, we propose a robust and effective GEPSVM classification algorithm based on L1-norm distance metric, termed as L1-GEPSVM. The optimization goal is to minimize the intra-class distance dispersion, and maximize the inter-class distance dispersion simultaneously. It is known that the application of L1-norm distance is often used as a simple and powerful way to reduce the impact of outliers, which improves the generalization ability and flexibility of the model. In addition, we develop an effective iterative algorithm to solve the L1-norm optimal problems, which is easy to implement and its convergence to a local optimum is theoretically ensured. Thus, the classification performance of L1-GEPSVM is more robust than GEPSVM. Finally, the feasibility and effectiveness of L1-GEPSVM are further verified by extensive experimental results on artificial datasets, UCI datasets and NDC datasets.
Similar content being viewed by others
References
Vapnik VN (2008) Statistical learning theory. Encycl Sci Learn 41(4):3185–3185
Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167
Deng N, Tian Y, Zhang C (2012) Support vector machines: optimization based theory, algorithms, and extensions. Chapman and Hall/CRC, Boca Raton
Boser BE, Guyon IM, Vapnik VN (1996) A training algorithm for optimal margin classifiers. In: Proceedings of annual ACM workshop on computational learning theory, vol 5, pp 144–152
Song Q, Hu W, Xie W (2002) Robust support vector machine with bullet hole image classification. IEEE Trans Syst Man Cybern Part C 32(4):440–448
Yin H, Jiao X, Chai Y, Fang B (2015) Scene classification based on single-layer SAE and SVM. Expert Syst Appl 42(7):3368–3380
Muralidharan V, Sugumaran V, Indira V (2014) Fault diagnosis of monoblock centrifugal pump using SVM. Eng Sci Technol Int J 17(3):152–157
Subasi A (2013) Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders. Comput Biol Med 43(5):576
Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69
Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: ACM SIGKDD international conference on knowledge discovery and data mining, pp 77–86
Khemchandani R, Jayadeva S (2007) Fuzzy twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910
Ye Q, Ye N (2009) Improved proximal support vector machine via generalized eigenvalues. In: International joint conference on computational sciences and optimization, pp 705–709
Guarracino MR, Cifarelli C, O.S. Pardalos PM (2007) A classification method based on generalized eigenvalue problems. Optim Methods Softw 22(1):73–81
Shao YH, Deng NY, Chen WJ, Wang Z (2013) Improved generalized eigenvalue proximal support vector machine. IEEE Signal Process Lett 20(3):213–216
Li CN, Shao YH, Deng NY (2015) Robust L1-norm non-parallel proximal support vector machine. Optimization 65(1):169–183
Marghny MH, Elaziz RMA, Taloba AI (2015) Differential search algorithm-based parametric optimization of fuzzy generalized eigenvalue proximal support vector machine. Int J Comput Appl 108(19):38–46
Ding S, Zhang N, Zhang X, Wu F (2016) Twin support vector machine: theory, algorithm and applications. Neural Comput Appl. doi:10.1007/s00521-016-2245-4
Ye Q, Zhao C, Chen X (2011) A feature selection method for TWSVM via a regularization technique. J Comput Res Dev 48(6):1029–1037
Zhang X, Ding S, Sun T (2016) Multi-class LSTMSVM based on optimal directed acyclic graph and shuffled frog leaping algorithm. Int J Mach Learn Cybern 7(2):241–251
Kwak N (2014) Principal component analysis by Lp-norm maximization. IEEE Trans Cybern 44(5):594–609
Kwak N (2008) Principal component analysis based on l1-norm maximization. IEEE Trans Pattern Anal Mach Intell 30(9):1672
Li CN, Shao YH, Deng NY (2015) Robust L1-norm two-dimensional linear discriminant analysis. Neural Netw 65:92–104
Wang H, Lu X, Hu Z, Zheng W (2013) Fisher discriminant analysis with L1-norm. IEEE Trans Cybern 44(6):828–842
Zhong F, Zhang J (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027
Ye Q, Yang J, Liu F, Zhao C, Ye N, Yin T (2016) L1-norm distance linear discriminant analysis based on an effective iterative algorithm. IEEE Trans Circuits Syst Video Technol PP(99):1–14
Jenatton R, Obozinski G, Bach F (2009) Structured sparse principal component analysis. J Mach Learn Res 9(2):131–160
Chen X, Yang J, Ye Q, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recognit 44(10):2643–2655
Bache K, Lichman M (2013) http://archive.ics.uci.edu/ml/
Musicant DR (1998) http://research.cs.wisc.edu/dmi/svm/ndc/
Yang X, Chen S, Chen B, Pan Z (2009) Proximal support vector machine using local information. Neurocomputing 73(1–3):357–365
Xue H, Chen S (2011) Glocalization pursuit support vector machine. Neural Comput Appl 20(7):1043–1053
Ye Q, Zhao C, Gao S, Zheng H (2012) Weighted twin support vector machines with local information and its application. Neural Netw 35(11):31–39
Ding S, Hua X, Yu J (2014) An overview on nonparallel hyperplane support vector machine algorithms. Neural Comput Appl 25(5):975–982
Lin G, Tang N, Wang H (2014) Locally principal component analysis based on L1-norm maximisation. Image Process Iet 9(2):91–96
Fawcett T (2005) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874
Acknowledgements
The work is supported in part by the National Science Foundation of China under Grants 61401214, 61773210, 61603184, 61603190 and 61772273, the Natural Science Foundation of Jiangsu Province under Grants BK20171453, BK20140794, the Jiangsu Key Laboratory for Internet of Things and Mobile Internet Technology, and the Jiangsu Key Laboratory of Image and Video Understanding for Social Safety.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yan, H., Ye, Q., Zhang, T. et al. L1-Norm GEPSVM Classifier Based on an Effective Iterative Algorithm for Classification. Neural Process Lett 48, 273–298 (2018). https://doi.org/10.1007/s11063-017-9714-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-017-9714-3