Skip to main content
SpringerLink
Log in

L1-norm Laplacian support vector machine for data reduction in semi-supervised learning

  • S.I. : New Trends of Neural Computing for Advanced Applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

As a semi-supervised learning method, Laplacian support vector machine (LapSVM) is popular. Unfortunately, the model generated by LapSVM has a poor sparsity. A sparse decision model has always been fascinating because it could implement data reduction and improve performance. To generate a sparse model of LapSVM, we propose an \(\ell _1\)-norm Laplacian support vector machine (\(\ell _1\)-norm LapSVM), which replaces the \(\ell _2\)-norm with the \(\ell _1\)-norm in LapSVM. The \(\ell _1\)-norm LapSVM has two techniques that can induce sparsity: the \(\ell _1\)-norm regularization and the hinge loss function. We discuss two situations for the \(\ell _1\)-norm LapSVM, linear and nonlinear ones. In the linear \(\ell _1\)-norm LapSVM, the sparse decision model implies that features with nonzero coefficients are contributive. In other words, the linear \(\ell _1\)-norm LapSVM can perform feature selection to achieve the goal of data reduction. Moreover, the nonlinear (kernel) \(\ell _1\)-norm LapSVM can also implement data reduction in terms of sample selection. In addition, the optimization problem of the \(\ell _1\)-norm LapSVM is a convex quadratic programming one. That is, the \(\ell _1\)-norm LapSVM has a unique and global solution. Experimental results on semi-supervised classification tasks have shown a comparable performance of our \(\ell _1\)-norm LapSVM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Adankon MM, Cheriet M (2009) Model selection for LS-SVM: application to handwriting recognition. Pattern Recogn 42(12):3264–3270

    Article  MATH  Google Scholar 

  2. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(1):2399–2434

    MathSciNet  MATH  Google Scholar 

  3. Bennett KP, Demiriz A (1999) Semi-supervised support vector machines. In: Proceedings of international conference on neural information processing systems, pp 368–374

  4. Bruzzone L, Chi M, Marconcini M (2006) A novel transductive SVM for semisupervised classification of remote-sensing images. IEEE Trans Geosci Remote Sens 44(11):3363–3373

    Article  Google Scholar 

  5. Bühler T, Hein M (2009) Spectral clustering based on the graph p-laplacian. In: Proceedings of international conference on machine learning, pp 81–88

  6. Chen L, Yang M (2017) Semi-supervised dictionary learning with label propagation for image classification. Comput Vis Media 3(1):83–94

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen Y, Wang G, Dong S (2003) Learning with progressive transductive support vector machine. Pattern Recogn Lett 24(12):1845–1855

    Article  Google Scholar 

  8. Cheng S, Huang Q, Liu J, Tang X (2013) A novel inductive semi-supervised SVM with graph-based self-training. In: Proceedings of international conference on intelligent science and intelligent data engineering. Lecture notes in computer science, vol 7751, pp 82–89. Springer, Berlin

  9. Dheeru D, Karra Taniskidou E (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  10. Dong A, Fl Chung, Deng Z, Wang S (2016) Semi-supervised SVM with extented hidden features. IEEE Trans Cybern 46(12):2924–2937

    Article  Google Scholar 

  11. Fan J, Tian Z, Zhao M, Chow TW (2018) Accelerated low-rank representation for subspace clustering and semi-supervised classification on large-scale data. Nerual Networks 100:39–48

    Article  MATH  Google Scholar 

  12. Fan J, Zhang Y, Udell M (2020) Polynomial matrix completion for missing data imputation and transductive learning. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), pp. 3842–3849

  13. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of international conference on knowledge discovery and data mining, pp 77–86

  14. Gammerman A, Vovk V, Vapnik V (2013) Learning by transduction. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Francisco, CA, pp 148–155

  15. Gao Y, Ma J, Yuille AL (2017) Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. IEEE Trans Image Process 26(5):2545–2560

    Article  MathSciNet  MATH  Google Scholar 

  16. Gasso G, Zapien K, Canu S (2007) Sparsity regularization path for semi-supervised SVM. In: Proceedings of international conference on machine learning and applications, pp 25–30

  17. Gu Z, Zhang Z, Sun J, Li B (2017) Robust image recognition by l1-norm twin-projection support vector machine. Neurocomputing 223:1–11

    Article  Google Scholar 

  18. Han M, Yin J (2008) The hidden neurons selection of the wavelet networks using support vector machines and ridge regression. Neurocomputing 72(1):471–479

    Article  Google Scholar 

  19. Huimin P, Qiang L, Liran Y, Ping Z (2020) A novel semi-supervised support vector machine with asymmetric squared loss. In: Advances in data analysis and classification, vol 9

  20. Jiang J, Ma J, Chen C, Jiang X, Wang Z (2017) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern 47(11)

  21. Le HM, Thi HAL, Nguyen MC (2015) Sparse semi-supervised support vector machines by DC programming and DCA. Neurocomputing 153:62–76

    Article  Google Scholar 

  22. Li Z, Zhang Z, Qin J, Zhang Z, Shao L (2020) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst 3(3):789–800

    MathSciNet  Google Scholar 

  23. Li Y, Kwok JT, Zhou Z (2009) Semi-supervised learning using label mean. In: Proceedings of international conference on machine learning, pp 633–640

  24. Liu RJ, Wang YH, Baba T, Uehara Y, Masumoto D, Nagata S (2008) SVM-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recogn 41(8):2645–2655

    Article  MATH  Google Scholar 

  25. Liu Z, Liu H, Zhao Z (2018) Weighted least squares support vector machine for semi-supervised classification. Wireless Pers Commun 103(1):797–808

    Article  Google Scholar 

  26. Ma J, Tian J, Bai X, Tu Z (2013) Regularized vector field learning with sparse approximation for mismatch removal. Pattern Recogn 46:3519–3532

    Article  MATH  Google Scholar 

  27. Peng X, Wang Y (2010) A bi-fuzzy progressive transductive support vector machine (bfptsvm) algorithm. Expert Syst Appl 37(1):527–533

    Article  Google Scholar 

  28. Poggio T, Girosi F (1998) A sparse representation for function approximation. Neural Comput 10(6):1445–1454

    Article  Google Scholar 

  29. Refaeilzadeh P, Tang L, Liu H (2016) Encyclopedia of database systems. In: Cross-validation. Springer, Berlin, pp 532–538

  30. Schölkopf B (2008) Sparseness of support vector machines. Mach Learn 4(6):1071–1105

    MathSciNet  Google Scholar 

  31. Sun Y, Zhang Z, Jiang W, Zhang Z, Zhang L, Wang M (2020) Discriminative local sparse representation by robust adaptive dictionary pair learning. IIEEE Trans Neural Netw Learn Syst 31:1–15

    MathSciNet  Google Scholar 

  32. Tan J, Zhen L, Deng N, Zhang Z (2014) Laplacian p-norm proximal support vector machine for semi-supervised classification. Neurocomputing 144(1):151–158

    Article  Google Scholar 

  33. Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Article  Google Scholar 

  34. Wang XY, Wang T, Bu J (2011) Color image segmentation using pixel wise support vector machine classification. Pattern Recogn 44(4):777–787

    Article  MATH  Google Scholar 

  35. Yang N, Sang Y, He R, Wang X (2010) Label propagation algorithm based on non-negative sparse representation. In: International conference on life system modeling and intelligent computing, pp 348–357

  36. Zhang Z, Chow TW (2012) Maximum margin multisurface support tensor machines with application to image classification and segmentation. Expert Syst Appl 49:849–860

    Article  Google Scholar 

  37. Zhang L, Zhou W (2010) On the sparseness of 1-norm support vector machines. Neural Netw 23(3):373–385

    Article  MATH  Google Scholar 

  38. Zhang L, Zhou W, Jiao L (2004) Hidden space support vector machines. IEEE Trans Neural Netw 15(6):1424–1434

    Article  Google Scholar 

  39. Zhang L, Zhou W, Chang P, Liu J, Yan Z, Wang T, Li F (2012) Kernel sparse representation-based classifier. IEEE Trans Signal Process 60:1684–1695

    Article  MathSciNet  MATH  Google Scholar 

  40. Zhang L, Zhou W, Li F (2015) Kernel sparse representation-based classifier ensemble for face recognition. Multimed Tools Appl 74(1):123–137

    Article  Google Scholar 

  41. Zhang Z, Jiang W, Qin J, Zhang L, Li F, Zhang M, Yan S (2018) Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans Neural Netw Learn Syst 29(8):3798–3814

    Article  MathSciNet  Google Scholar 

  42. Zhang Z, Jiang W, Zhang Z, Li S, Liu G, Qin J (2019) Scalable block-diagonal locality-constrained projective dictionary learning. In: Twenty-eighth international joint conference on artificial intelligence IJCAI-19

  43. Zhao M, Liu J, Zhang Z, Fan J (2020) A scalable sub-graph regularization for efficient content based image retrieval with long-term relevance feedback enhancement. Knowledge-based systems, p 106505. https://doi.org/10.1016/j.knosys.2020.106505

  44. Zhou W, Zhang L, Jiao L (2006) Hidden space principal component analysis. In: Proceedings of Pacific-Asia conference on advances in knowledge discovery and data mining, pp 801–805

  45. Zhu X, Goldberg A (2009) Introduction to semi-supervised learning. Morgan and Claypool, Vermont

    Book  MATH  Google Scholar 

  46. Zhu J, Rosset S, Hastie T, Tibshirani R (2003) 1-norm support vector machines. In: Proceedings of the 16th international conference on neural information processing ,Systems vol 16(1), pp 49–56

Download references

Acknowledgements

We would like to thank three anonymous reviewers and Editor Zhao for their valuable comments and suggestions, which have significantly improved this paper.

Funding

This work was supported in part by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant No. 19KJA550002, by the Six Talent Peak Project of Jiangsu Province of China under Grant No. XYDXX-054, by the Priority Academic Program Development of Jiangsu Higher Education Institutions, and by the Collaborative Innovation Center of Novel Software Technology and Industrialization.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology and Joint International Research Laboratory of Machine Learning and Neuromorphic Computing, Soochow University, Suzhou, 215006, Jiangsu, China

    Xiaohan Zheng, Li Zhang & Zhiqiang Xu

  2. Provincial Key Laboratory for Computer Information Processing Technology, Soochow University, Suzhou, 215006, Jiangsu, China

    Xiaohan Zheng, Li Zhang & Zhiqiang Xu

Authors
  1. Xiaohan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiqiang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights statement

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, X., Zhang, L. & Xu, Z. L1-norm Laplacian support vector machine for data reduction in semi-supervised learning. Neural Comput & Applic 35, 12343–12360 (2023). https://doi.org/10.1007/s00521-020-05609-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05609-9

Keywords

Search

Navigation