Skip to main content
SpringerLink
Log in

Boosting

  • Living reference work entry
  • First Online:
Computer Vision

  • 276 Accesses

Synonyms

Boosting algorithm

Related Concepts

Definition

Boosting is an ensemble meta-learning algorithm for supervised learning such as classification and regression problems. In the boosting algorithm, weak hypotheses are sequentially learned at each stage and aggregated into a single highly accurate hypothesis.

Background

Boosting is an important branch of ensemble learning in machine learning. In the paradigm of ensemble learning, many hypotheses learned from observed samples are aggregated into a single accurate hypothesis. The ensemble learning includes popular learning methods such as bagging and random forest as well as boosting. Boosting is, however, thought of as one of the most promising ensemble methods for classification and regression problems.

The study of boosting has started from the following question: Is it possible to boost “weak learner” into “strong learner”? which was given by Kearns and Valiant in 1988. The weak learner intuitively denotes a learning...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227

    Google Scholar 

  2. Freund Y, Schapire RE (1995) A decision-theoretic generalization of on-line learning and an application to boosting. In: Proceedings of the second European conference on computational learning theory, EuroCOLT ’95, pp 23–37

    Google Scholar 

  3. Friedman J, Hastie T, Tibshirani R (1998) Additive logistic regression: a statistical view of boosting. Ann Stat 28:337–407

    Article  MathSciNet  MATH  Google Scholar 

  4. Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232

    Article  MathSciNet  MATH  Google Scholar 

  5. Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth and Brooks, Monterey

    MATH  Google Scholar 

  6. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD, pp 785–794

    Google Scholar 

  7. Ke G et al (2017) LightGBM: a highly efficient gradient boosting decision tree. In: Advances in neural information processing systems, vol 30, pp 3146–3154

    Google Scholar 

  8. Morishita S (2002) Computing optimal hypotheses efficiently for boosting, pp 471–481. Springer, Berlin/Heidelberg

    MATH  Google Scholar 

  9. Kudo T, Maeda E, Matsumoto Y (2005) An application of boosting to graph classification. In: Advances in neural information processing systems, vol 17, pp 729–736

    Google Scholar 

  10. Saigo H, Nowozin S, Kadowaki T, Kudo T, Tsuda K (2009) gBoost: a mathematical programming approach to graph classification and regression. Mach Learn 75(1):69–89

    Article  Google Scholar 

  11. Kashihara K, Hatano K, Bannai H, Takeda M (2010) Sparse substring pattern set discovery using linear programming boosting. In: Proceedings of the 13th international conference on discovery science (DS 2010), vol 6332, pp 132–143

    Google Scholar 

  12. Mangasarian OL (1999) Arbitrary-norm separating plane. Oper Res Lett 24:15–23

    Article  MathSciNet  MATH  Google Scholar 

  13. Schapire RE, Freund Y, Bartlett P, Lee WS (1998) Boosting the margin: a new explanation for the effectiveness of voting methods. Ann Stat 26(5):1651–1686

    Article  MathSciNet  MATH  Google Scholar 

  14. Koltchinskii V, Panchenko D (2002) Empirical margin distributions and bounding the generalization error of combined classifiers. Ann Stat 30(1):1–50

    MathSciNet  MATH  Google Scholar 

  15. Mohri M, Rostamizadeh A, Talwalker A (2018) Foundation of machine learning, 2nd edn. The MIT Press, Cambridge

    MATH  Google Scholar 

  16. Demiriz A, Bennett KP, Shawe-Taylor J (2002) Linear programming boosting via column generation. Mach Learn 46(1–3):225–254

    Article  MATH  Google Scholar 

  17. Warmuth M, Glocer K, Rätsch G (2007) Boosting algorithms for maximizing the soft margin. In: Advances in neural information processing systems 20 (NIPS 2007), pp 1585–1592

    Google Scholar 

  18. Warmuth M, Glocer K, Vishwanathan SVN (2008) Entropy regularized LPBoost. In: Proceedings of the 19th international conference on algorithmic learning theory, pp 256–271

    Google Scholar 

  19. Shalev-Shwartz S (2014) Selfieboost: a boosting algorithm for deep learning. arXiv:1411.3436

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tokyo Institute of Technology, Tokyo, Japan

    Takafumi Kanamori & Osamu Watanabe

  2. Kyushu University, Fukuoka, Japan

    Kohei Hatano

Authors
  1. Takafumi Kanamori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kohei Hatano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Osamu Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Takafumi Kanamori .

Section Editor information

  1. Waseda University, Tokyo, Japan

    Hiroshi Ishikawa

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Kanamori, T., Hatano, K., Watanabe, O. (2020). Boosting. In: Computer Vision. Springer, Cham. https://doi.org/10.1007/978-3-030-03243-2_836-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03243-2_836-1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03243-2

  • Online ISBN: 978-3-030-03243-2

  • eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation