Skip to main content
SpringerLink
Log in

ROC Analysis

  • Reference work entry
Encyclopedia of Machine Learning

Synonyms

Receiver operating characteristic analysis

Definition

ROC analysis investigates and employs the relationship between sensitivity and specificity of a binary classifier. Sensitivity or true positive rate measures the proportion of positives correctly classified; specificity or true negative rate measures the proportion of negatives correctly classified. Conventionally, the true positive rate (tpr) is plotted against the false positive rate (fpr), which is one minus true negative rate. If a classifier outputs a score proportional to its belief that an instance belongs to the positive class, decreasing the decision threshold – above which an instance is deemed to belong to the positive class – will increase both true and false positive rates. Varying the decision threshold from its maximal to its minimal value results in a piecewise linear curve from (0, 0) to (1, 1), such that each segment has a nonnegative slope (Fig. 1). This ROC curve is the main tool used in ROC analysis....

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Recommended Reading

  • Bourke, C., Deng, K., Scott, S., Schapire, R., & Vinodchandran, N. V. (2008). On reoptimizing multi-class classifiers. Machine Learning, 71(2–3), 219–242.

    Article  Google Scholar 

  • Brier, G. (1950). Verification of forecasts expressed in terms of probabilities. Monthly Weather Review, 78, 1–3.

    Article  Google Scholar 

  • Drummond, C., & Holte, R. (2006). Cost curves: An improved method for visualizing classifier performance. Machine Learning, 65(1), 95–130.

    Article  Google Scholar 

  • Egan, J. (1975). Signal detection theory and ROC analysis. Series in cognitition and perception. New York: Academic Press.

    Google Scholar 

  • Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.

    Article  MathSciNet  Google Scholar 

  • Fawcett, T., & Niculescu-Mizil, A. (2007). PAV and the ROC convex hull. Machine Learning, 68(1), 97–106.

    Article  Google Scholar 

  • Ferri, C., Hernández-Orallo, J., & Salido, M. (2003). Volume under the ROC surface for multi-class problems. In Proceedings of the fourteenth (ECML 2003) (pp. 108–120). Lecture Notes in Computer Science 2837. Berlin: Springer.

    Google Scholar 

  • Flach, P. (2003). The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In Proceedings of the twentieth international conference on machine learning (ICML 2003) (pp. 194–201). Washington, DC: AAAI Press.

    Google Scholar 

  • Flach, P. (2004). The many faces of ROC analysis in machine learning. ICML-04 Tutorial. http://www.cs.bris.ac.uk/flach/ICML04tutorial/. Accessed on 16 December 2009.

  • Fuernkranz, J., & Flach, P. (2005). ROC ‘n’ Rule learning – towards a better understanding of covering algorithms. Machine Learning, 58(1), 39–77.

    Article  MATH  Google Scholar 

  • Hand, D., & Till, R. (2001). A simple generalization of the area under the ROC curve to multiple class classification problems. Machine Learning, 45(2), 171–186.

    Article  MATH  Google Scholar 

  • Lachiche, N., & Flach, P. (2003). Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves. In Proceedings of the twentieth international conference on machine learning (ICML’03) (pp. 416–423). Washington, DC: AAAI Press.

    Google Scholar 

  • Mossman, D. (1999). Three-way ROCs. Medical Decision Making, 19, 78–89.

    Article  Google Scholar 

  • Provost, F., & Fawcett, T. (2001). Robust classification for imprecise environments. Machine Learning, 42(3), 203–231.

    Article  MATH  Google Scholar 

  • Srinivasan, A. (1999). Note on the location of optimal classifiers in n-dimensional ROC space. Technical report PRG-TR-2-99. Oxford University Computing Laboratory, Oxford.

    Google Scholar 

  • Zadrozny, B., & Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 694–699). New York: ACM.

    Google Scholar 

Download references

Author information

Authors and Affiliations

    Authors
    1. Peter A. Flach
      View author publications

      You can also search for this author in PubMed Google Scholar

    Editor information

    Editors and Affiliations

    1. School of Computer Science and Engineering, University of New South Wales, Sydney, Australia, 2052

      Claude Sammut

    2. Faculty of Information Technology, Clayton School of Information Technology, Monash University, P.O. Box 63, Victoria, Australia, 3800

      Geoffrey I. Webb

    Rights and permissions

    Reprints and permissions

    Copyright information

    © 2011 Springer Science+Business Media, LLC

    About this entry

    Cite this entry

    Flach, P.A. (2011). ROC Analysis. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-30164-8_733

    Download citation

    Publish with us

    Policies and ethics

    Search

    Navigation