Skip to main content

Advertisement

SpringerLink
Log in

Energy-based structural least squares MBSVM for classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multiple birth support vector machine (MBSVM) is an extension of twin support vector machine on multi-class classification problem. In MBSVM, the size of each QP problem is restricted by the number of patterns in one of the K classes, so the computational complexity of MBSVM is much lower and the training speed of it is faster than the existing multi-class SVM. However, MBSVM neglects the structural information of data which may contain some significant prior knowledge for training classifiers. In this paper, we first present an improved version of structural least square twin support vector machine (S-LSTWSVM), called energy-based structural least square twin support vector machine (ES-LSTWSVM), which converts the constraints of the S-LSTWSVM into an energy-based model by introducing an energy for each hyperplane. Then we use the strategy of “rest-versus-one” in MBSVM to extend ES-LSTWSVM into the multi-class classification, called energy-based structural least squares MBSVM (ESLS-MBSVM). In order to prove the validity of ESLS-MBSVM, the experiment has been performed on UCI datasets. The experimental results show that our ESLS-MBSVM is effective and has good classification performance. In order to better illustrate the experimental results, we use Friedman test and ROC analysis for statistical comparisons.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Cortes C, Vapnik VN (1995) Support vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  2. Vapnik VN (1995) The nature of statistical learning theory. Springer, New York

    Book  Google Scholar 

  3. Jayadeva R, Khemchandai SC (2007) Twin support vector machine classification for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  4. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  5. Kumar M, Arun MG (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  6. Chen XB, Yang J, Ye QL, Liang J (2011) Recursive projection twin support vector machine via within-class variance minimization. Pattern Recogn 44(10):2643–2655

    Article  Google Scholar 

  7. Shao YH, Chen WJ, Deng NY (2014) Nonparallel hyperplane support vector machine for binary classification problems. Inf Sci 263:22–35

    Article  MathSciNet  Google Scholar 

  8. Tian YJ, Qi ZQ, Ju XC, Shi Y, Liu XH (2014) Nonparallel support vector machines for pattern classification. IEEE Transactions on Cybernetics 44(7):1067–1079

    Article  Google Scholar 

  9. Qi ZQ, Tian YJ, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recogn 46(1):305–316

    Article  Google Scholar 

  10. Shao YH, Wang Z, Chen WJ, Deng NY (2013) A regularization for the projection twin support vector machine. Knowl-Based Syst 37(none):203–210

    Article  Google Scholar 

  11. Ding SF, Hua XP (2014) Recursive least squares projection twin support vector machines for nonlinear classification. Neurocomputing 130(3):3–9

    Google Scholar 

  12. Xie XJ, Sun SL, Chen HH, Qian JB (2018) Domain Adaptation with Twin Support Vector Machines. Neural Process Lett 48(2):1213–1226

    Article  Google Scholar 

  13. Xie XJ (2018) Regularized multi-view least squares twin support vector machines. Appl Intell

  14. Nasiri JA, Charkari NM, Mozafari K (2014) Energy-based model of least squares twin support vector machines for human action recognition. Signal Process 104(6):248–257

    Article  Google Scholar 

  15. Rigollet P (2006) Generalization error bounds in semi-supervised classification under the cluster assumption. J Mach Learn Res 8(3):1369–1392

    MATH  Google Scholar 

  16. Yeung DS, Wang D, Ng W, Tsang E, Wang X (2007) Structured large margin machines: sensitive to data distributions. Mach Learn 68(2):171–200

    Article  Google Scholar 

  17. Lanckriet GRG, Ghaoui LE, Bhattacharyya C, Jordan MI (2002) A robust minimax approach to classification. J Mach Learn Res 3:555–582

    MathSciNet  MATH  Google Scholar 

  18. Huang K, Yang H, King I, Lyu MR (2008) Maxi-min margin machine: learning large margin classifiers locally and globally. IEEE Trans Neural Netw 19(2):260–272

    Article  Google Scholar 

  19. Xue H, Chen S, Yang Q (2011) Structural regularized support vector machine: a framework for structural large margin classifier. IEEE Trans Neural Netw 22(4):573–587

    Article  Google Scholar 

  20. Qi Z, Tian Y, Shi Y (2013) Structural twin support vector machine for classification. Knowl-Based Syst 43:74–81

    Article  Google Scholar 

  21. Angulo C, Parra X, Catala A (2003) K-svcr. a support vector machine for multi-class classification. Neurocomputing 55(1):57–77

    Article  Google Scholar 

  22. Xu Y, Guo R, Wang L (2013) A twin multi-class classification support vector machine. Cogn Comput 5(4):580–588

    Article  Google Scholar 

  23. Nasiri JA, Charkari NM, Jalili S (2015) Least squares twin multi-class classification support vector machine. Pattern Recogn 48(3):984–992

    Article  Google Scholar 

  24. López J, Maldonado S, Carrasco M (2017) A robust formulation for twin multiclass support vector machine. Appl Intell 47(4):1031–1043

    Article  Google Scholar 

  25. de Lima MD, Luiza CN, Rommel B (2018) Improvements on least squares twin multi-class classification support vector machine. Neurocomputing. S0925231218307768

  26. Yang ZX, Shao YH, Zhang XS (2013) Multiple birth support vector machine for multi-class classification. Neural Comput & Applic 22(1 Supplement):153–161

    Article  Google Scholar 

  27. Zhang XK, Ding SF, Xue Y (2016) An improved multiple birth support vector machine for pattern classification. Neurocomputing:225

  28. Ding SF, Zhang XK, An YX, Xue Y (2017) Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification. Pattern Recogn 67:32–46

    Article  Google Scholar 

  29. Chen SG, Wu XJ (2016) Multiple birth least squares support vector machine for multi-class classification. Int J Mach Learn Cybern

  30. Xu Y, Pan X, Zhou Z, Yang Z, Zhang Y (2015) Structural least square twin support vector machine for classification. Appl Intell 42(3):527–536

    Article  Google Scholar 

  31. Wardjr J (1963) Hierarchical grouping to optimize an objective function. Publ Am Stat Assoc 58(301):9

    MathSciNet  Google Scholar 

  32. Xu X, Ding SF, Du MJ, Xue Y (2018) DPCG: an efficient density peaks clustering algorithm based on grid. Int J Mach Learn Cybern 9(5):743–754

    Article  Google Scholar 

  33. Du MJ, Ding SF, Xue Y (2018) A robust density peaks clustering algorithm using fuzzy neighborhood. Int J Mach Learn Cybern 9(7):1131–1140

    Article  Google Scholar 

  34. Xu X, Ding SF, Shi ZZ (2018) An improved density peaks clustering algorithm with fast finding cluster centers. Knowl-Based Syst 158:65–74

    Article  Google Scholar 

  35. Ding SF, Du MJ, Sun TF, Xue Y (2017) An entropy based density peaks clustering algorithm for mixed type data employing fuzzy neighborhood. Knowl-Based Syst 133:294–313

    Article  Google Scholar 

  36. Salvador S, Chan P (2004) Determining the Number of Clusters/Segments in Hierarchical Clustering/Segmentation Algorithms. IEEE International Conference on Tools with Artificial Intelligence

  37. Williams CKI (2013) Learning with kernels: support vector machines, regularization, optimization, and beyond. Publ Am Stat Assoc 98(462):1

    Google Scholar 

  38. Lichman M (2013) UCI machine learning repository, University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml

Download references

Acknowledgements

This work is supported by the Fundamental Research Funds for the Central Universities (No.2017XKZD03).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Songhui Shi, Shifei Ding & Zichen Zhang

  2. Mine Digitization Engineering Research Center, Ministry of Education of the People’s Republic of China, Xuzhou, 221116, China

    Songhui Shi, Shifei Ding & Zichen Zhang

  3. School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, China

    Weikuan Jia

Authors
  1. Songhui Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zichen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weikuan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Ethics declarations

Conflict of Interest

We declare that we have no signicant competing nancial, professional or personal interests that might have in uenced the performance or presentation of the work described in this manuscript.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, S., Ding, S., Zhang, Z. et al. Energy-based structural least squares MBSVM for classification. Appl Intell 50, 681–697 (2020). https://doi.org/10.1007/s10489-019-01536-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01536-y

Keywords

Search

Navigation