Skip to main content
SpringerLink
Log in

The Generalized Regression Neural Network Oracle

  • Chapter
  • First Online:
The Art and Science of Machine Intelligence

Abstract

In this chapter, we describe what are best characterized as complex adaptive systems and give several mixture of expert systems as examples of these complex systems. This background discussion is followed by three theoretical sections covering the topics of kernel-based probability estimation systems, a generalized neural network example, and a derivation of an ensemble combination and finally, a two-view ensemble combination. A summary of the equations describing the oracle follows these sections for those readers who do not want to work through all that mathematics. The next section introduces Receiver Operator Characteristic (ROC) analysis, a popular method for quantitatively assessing the performance of learning classifier systems. Next is the definition of “trouble-makers”, and how they were discovered, followed by a discussion of the development of two hybrids: an Evolutionary Programming-Adaptive boosting (EP-AB) and a Generalized Regression Neural Network (GRNN) oracle for the purpose of demonstrating the existence of the trouble-makers by using an ROC measure of performance analysis. That discussion is followed by a detailed discussion of how to perform and evaluate an ROC analysis as well as a detailed practice example for those readers not familiar with this measure of performance technology. This chapter concludes with a research study on how to use the oracle to establish if the data sample size is adequate to accurately meet a 95% confidence interval imposed on the variance (or standard deviation) for the oracle. This is an important research study as very little effort is generally put into establishing the correct data set size for accurate, predictable, and repeatable performance results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The term nonparametric is used here to mean that the parameters of the distributions are not specified in the hypothesis. Instead, the parameters are chosen to best represent the observed data.

Abbreviations

AB:

Adaptive boosting

ANN:

Artificial neural network

AUC:

Area under the curve

CAS:

Complex Adaptive System

EP:

Evolutionary programming

FN:

False negative

FP:

False positive

GRNN:

Generalized Regression Neural Network

LDA:

Linear discriminant analysis

LR:

Logistic regression

MLFN:

Multi-layered feed forward neural network

MLP:

Multi-layer perceptron

MOE:

Margin of error

PNN:

Probabilistic Neural Network

ROC:

Receiver operator characteristic

SLT:

Statistical learning theory

SVM:

Support vector machine

References

  • Bagnasco S, Bottigli U, Cerello P, Cheran S, Delogu P, Fantacci ME, Fauci F, Forni G, Lauria A, Torres EL, Magro R, Masala GL, Oliva P, Palmiero R, Ramello L, Raso G, Retico A, Sitta M, Stumbo S, Tangaro S, Zanon E (2005) GPCALMA: a GRID based tool for mammographic screening. Methods Inf Med 44(2):244–248

    Article  Google Scholar 

  • Baker JA, Kornguth PJ, Lo JY, Williford ME, Floyd CE Jr (1995) Breast cancer: prediction with artificial neural network based on BI-RADS standardized lexicon. Radiology 196:817–822

    Article  Google Scholar 

  • BI-RADS (1993) Breast Imaging—Reporting and Data System (BI-RADS). American College of Radiology, Virginia

    Google Scholar 

  • Brown G, Wyatt J, Harris R, Yao X (2005) Diversity creation methods: a survey and categorisation. Inf Fusion 6:5–20

    Article  Google Scholar 

  • Cacoullos T (1966) Estimation of a multivariate density. Ann Inst Stat Math 18:179–189

    Article  MathSciNet  Google Scholar 

  • Hand DJ, Till RJ (2001) A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach Learn 45(2):171–186

    Article  Google Scholar 

  • Land WH, Masters T, Lo JY (2000a) Performance evaluation using the GRNN Oracle and a new evolutionary programming/adaptive boosting hybrid for breast cancer benign/malignant diagnostic aids, ANNIE

    Google Scholar 

  • Land WH, Masters T, Lo JY (2000b) Application of a new evolutionary programming/adaptive boosting hybrid to breast cancer diagnosis. In: IEEE congress on evolutionary computation proceedings (CEC2000)

    Google Scholar 

  • Land WH, Masters TD, Lo JY (2000c) Application of a GRNN oracle to the intelligent combination of several breast cancer benign/malignant predictive paradigms. In: Hanson KM (ed) Medical imaging 2000; image processing, proceedings of SPIE, San Diego, CA, pp 77–85

    Google Scholar 

  • Land WH, Masters T, Lo JY, McKee D, (2000d) Using evolutionary computation to develop neural network breast cancer benign/malignant classification models. In: 4th world conference on systemics, cybernetics and informatics (SCI2000), vol 10, pp 343–347

    Google Scholar 

  • Land WH, Margolis D, Kallergi M, Heine JJ (2010) A kernel approach for ensemble decision combinations with two-view mammography applications. Int J Funct Inf Pers Med 3(2):157–182

    Google Scholar 

  • Lo JY, Baker JA, Kornguth PJ, Floyd CE Jr (1995) Computer-aided diagnosis of breast cancer: artificial neural network approach for optimized merging of mammographic features. Acad Radiol 2:841–850

    Article  Google Scholar 

  • Lo JY, Baker JA, Kornguth PJ, Iglehart JD, Floyd CE Jr (1999) Effect of patient history data on the prediction of breast cancer from mammographic findings with artificial neural networks. Acad Radiol 6:10–15

    Article  Google Scholar 

  • Masters T (1995) Advanced algorithms for neural networks, A C++ source book. Wiley, New York, ISBN 0-471-10588-09 (paper/disk)

    Google Scholar 

  • Masters T, Land WH (1997) A new training method for the general regression neural network. In: IEEE international SMC conference proceedings, pp 1990–1995

    Google Scholar 

  • Masters T, Land WH, Maniccam S (1998) An oracle based on the general regression neural network. In: IEEE international conference on systems, man, and cybernetics—SMC, vol 2, pp 1615–1618

    Google Scholar 

  • Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philos Trans R Soc Lond A 209:415–446

    Article  Google Scholar 

  • Nadaraya EA (1964) On estimating regression. Theory Probab Appl 9:141–142

    Article  Google Scholar 

  • Parzen E (1962) On estimation of a probability density function and mode. Ann Math Stat 33(3):1065

    Article  MathSciNet  Google Scholar 

  • Pierson M (2002) Complex adaptive systems, a definition. http://radio-weblogs.com/0107584/stories/2002/05/13/complexAdaptiveSystemsADefinition.html [last visited 2015-05-19]

  • Polikar R (2006) Ensemble based systems in decision making. IEEE Circuits Syst Mag 6(3):21–45

    Article  Google Scholar 

  • Powers DMW (2011) Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. J Mach Learn Technol 2(1):37–63

    MathSciNet  Google Scholar 

  • Raponi M, Zhang Y, Yu J, Chen G, Lee G, Taylor JMG, Macdonald J, Thomas D, Moskaluk C, Wang Y, Beer DG (2006) Gene expression signatures for predicting prognosis of squamous cell and adenocarcinomas of the lung. Cancer Res 66(15):7466–7472. [http://cancerres.aacrjournals.org/cgi/content/abstract/66/15/7466]

    Article  Google Scholar 

  • Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Shedden K, Taylor JMG, Enkemann SA, Tsao M-S, Yeatman TJ, Gerald WL, Eschrich S, Jurisica I, Giordano TJ, Misek DE, Chang AC, Zhu CQ, Strumpf D, Hanash S, Shepherd FA, Ding K, Seymour L, Naoki K, Pennell N, Weir B, Verhaak R, Ladd-Acosta C, Golub T, Gruidl M, Sharma A, Szoke J, Zakowski M, Rusch V, Kris M, Viale A, Motoi N, Travis W, Conley B, Seshan VE, Meyerson M, Kuick R, Dobbin KK, Lively T, Jacobson JW (2008) Beer, gene expression–based survival prediction in lung adenocarcinoma: a multi-site, blinded validation study. Nat Med 14(8):822–827

    Article  Google Scholar 

  • Specht DF (1990) Probabilistic neural networks. Neural Netw 3:109–118

    Article  Google Scholar 

  • Specht DF (1991) A general regression neural network. IEEE Trans Neural Netw 2:568–576

    Article  Google Scholar 

  • Stern H (2003) Improving on the mixture of experts algorithm. Computational Neuroscience Project, Dalhousie University, Halifax

    Google Scholar 

  • Tang K, Wang R, Chen T (2011), Towards maximizing the area under the ROC for Multi_class classification problems. In: Proceedings 25th conference on AI, AAAI, pp 483–488

    Google Scholar 

  • Taylor JR (1982) An introduction to error analysis. University Science Books, Mill Valley

    Google Scholar 

  • Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  • Vapnik VN (2000) The nature of statistical learning theory. Springer, New York

    Book  Google Scholar 

  • Watson GS (1964) Smooth regression analysis. Sankhya 26:359–372

    MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Binghamton University, Bowie, MD, USA

    Walker H. Land Jr.

  2. Binghamton University, Binghamton, NY, USA

    J. David Schaffer

Authors
  1. Walker H. Land Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. David Schaffer
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Land, W.H., Schaffer, J.D. (2020). The Generalized Regression Neural Network Oracle. In: The Art and Science of Machine Intelligence. Springer, Cham. https://doi.org/10.1007/978-3-030-18496-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-18496-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-18495-7

  • Online ISBN: 978-3-030-18496-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation