Skip to main content
SpringerLink
Log in

A Study on GMLVQ Convex and Non-convex Regularization

  • Conference paper
  • First Online:
Advances in Self-Organizing Maps and Learning Vector Quantization

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 428))

Abstract

In this work we investigate the effect of convex and non-convex regularization on the Generalized Matrix Learning Vector Quantization (GMLVQ) classifier, in order to obtain sparse models that guarantee a better generalization ability. Three experiments are used for evaluating six different sparse models in terms of classification accuracy and qualitative sparseness. The results show that non-convex models outperform traditional convex sparse models and non-regularized GMLVQ.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Antoniadis, A., Fan, J.: Regularization of wavelet approximations. J. Am. Stat. Assoc. 939–967 (2011)

    Google Scholar 

  2. Bertsekas, D.P.: Nonlinear programming. Athena Sci. (1999)

    Google Scholar 

  3. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)

    Google Scholar 

  4. Candes, E., Tao, T.: The dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 2313–2351 (2007)

    Google Scholar 

  5. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  6. Figueiredo, M.A.: Adaptive sparseness for supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1150–1159 (2003)

    Article  MathSciNet  Google Scholar 

  7. Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Netw. 15(8), 1059–1068 (2002)

    Article  Google Scholar 

  8. Kaden, M., Lange, M., Nebel, D., Riedel, M., Geweniger, T., Villmann, T.: Aspects in classification learning-review of recent developments in learning vector quantization. Found. Comput. Decis. Sci. 39(2), 79–105 (2014)

    MathSciNet  Google Scholar 

  9. Kohonen, T.: Self-organizing maps, vol. 30. Springer (2001)

    Google Scholar 

  10. Leahy, R.M., Jeffs, B.D.: On the design of maximally sparse beamforming arrays. IEEE Trans. Antennas Propag. 39(8), 1178–1187 (1991)

    Article  Google Scholar 

  11. Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems, pp. 801–808 (2006)

    Google Scholar 

  12. Ng, A.Y.: Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 78. ACM (2004)

    Google Scholar 

  13. Nikolova, M.: Local strong homogeneity of a regularized estimator. SIAM J. Appl. Math. 61(2), 633–658 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  14. Nova, D., Estévez, P.A.: A review of learning vector quantization classifiers. Neural Comput. Appl. 25(3–4), 511–524 (2014)

    Article  Google Scholar 

  15. Osborne, M.R., Presnell, B., Turlach, B.A.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal.-Inst. Math. Appl. 20(3), 389–404 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  16. Sato, A., Yamada, K.: Generalized learning vector quantization. Adv. Neural Inf. Process. Syst. 423–429 (1996)

    Google Scholar 

  17. Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. IEEE Trans. Neural Netw. 21(5), 831–840 (2010)

    Article  Google Scholar 

  18. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Royal Stat. Soc. Ser. B (Methodological) 267–288 (1996)

    Google Scholar 

  19. Tropp, J., et al.: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theor. 52(3), 1030–1051 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  20. Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M.: Use of the zero norm with linear models and kernel methods. J. Mach. Learn. Res. 3, 1439–1461 (2003)

    MATH  MathSciNet  Google Scholar 

  21. Zhang, T.: Some sharp performance bounds for least squares regression with l1 regularization. Ann. Stat. 37(5A), 2109–2144 (2009)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This contribution was funded by CONICYT-CHILE under grants Fondecyt 1140816 and DPI20140090.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Chile, Casilla 412-3, Santiago, Chile

    David Nova & Pablo A. Estévez

  2. Advanced Mining Technology Center, University of Chile, Santiago, Chile

    Pablo A. Estévez

  3. Millennium Institute of Astrophysics, Santiago, Chile

    David Nova & Pablo A. Estévez

Authors
  1. David Nova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pablo A. Estévez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pablo A. Estévez .

Editor information

Editors and Affiliations

  1. Department of Statistics, Rice University, Houston, Texas, USA

    Erzsébet Merényi

  2. Department of Electrical and Computer En, Air Force Institute of Technology, Wright-Patterson AFB, Ohio, USA

    Michael J. Mendenhall

  3. Applied Physics, Rice University, Houston, Texas, USA

    Patrick O'Driscoll

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Nova, D., Estévez, P.A. (2016). A Study on GMLVQ Convex and Non-convex Regularization. In: Merényi, E., Mendenhall, M., O'Driscoll, P. (eds) Advances in Self-Organizing Maps and Learning Vector Quantization. Advances in Intelligent Systems and Computing, vol 428. Springer, Cham. https://doi.org/10.1007/978-3-319-28518-4_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28518-4_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28517-7

  • Online ISBN: 978-3-319-28518-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation