Abstract
Neural network ensembles (some times referred to as committees or classifier ensembles) are effective techniques to improve the generalization of a neural network system. Combining a set of neural network classifiers whose error distributions are diverse can lead to generating more accurate results than any single network. Combination strategies commonly used in ensembles include simple averaging, weighted averaging, majority voting and ranking. However, each method has its limitations, dependent either on the application areas it is suited to, or due to its effectiveness. This paper proposes a new ensembles combination scheme called multistage neural network ensembles. Experimental investigations based on multistage neural network ensembles are presented, and the benefit of using this approach as an additional combination method in ensembles is demonstrated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tumer, K., Ghosh, J.: Error correlation and error reduction in ensemble classifiers. Connection Science: Special Issue on Combining Artificial Neural Ensemble Approaches, 8(3&4), 385–404, 1999.
Sharkey, A. J. C., Sharkey, N. E., Chandroth, G. O.: Neural nets and diversity. Neural Computing and Applications, 4, 218–227, 1996.
Breiman, L.: Bagging predictors. Machine learning, 24 123–140, 1996.
Krogh, A., Vedelsby, J.: Neural Network Ensembles, Cross Validation, and Active Learning. Advances in Neural Information Processing Systems, 7, MIT press, Editors: Tesauro, G., Touretzky, D.S. and Leen, T.K. pp. 231–238, 1995.
Partridge, D., Griffith, N.: Strategies for Improving Neural Net Generalisation. Neural Computing and Applications, 3, 27–37, 1995.
Wolpert, D. H.: Stacked generalization. Neural Networks, 5, 241–259, 1992.
Zeng, X., Martinez, T. R.: Using a Neural Network to Approximate an Ensemble of Classifiers. Neural Processing Letters, 12, 225–237, 2000.
Tumer, K., Ghosh, J.: Order statistics combiners of neural classifiers. In Proceedings of the World Congress on Neural Network, INNS press, Washington DC, 31–34, 1995.
Lincoln, W., Skrzypek, J.: Synergy of clustering multiple back propagation networks. Advances in Neural Information Processing Systems-2, Touretzky, D., (ed.), Morgan Kaufmann, 650–657, 1990.
Jacobs, R. A.: Methods for combining experts’ probability assessments. Neural Computation, 7, 867–888, 1995.
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Analysis and Machine Intell, 993–1001, 1990.
Al-Ghoneim, K., Kumar Vijaya B. V. K.: Learning ranks with neural networks. In Applications and Science of Artificial Neural Networks: Proceedings of the SPIE, 2492, 446–464, 1995.
Ho, T. K., Hull, J. J., Srihari, S. N.: Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(1), 66–76, 1994.
Lee, D. S., Srihari, S. N.: Handprinted Digit Recognition: A Comparison of Algorithms. Pre-Proc. 3RD International Workshop On Frontiers In Handwriting Recognition, Buffalo, USA, 153–162, 1993.
Kittler, J.: Combining Classifiers: A Theoretical Framework. Pattern Analysis and Applications, 1, 18–27, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, S., Browne, A., Picton, P.D. (2002). Multistage Neural Network Ensembles. In: Roli, F., Kittler, J. (eds) Multiple Classifier Systems. MCS 2002. Lecture Notes in Computer Science, vol 2364. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45428-4_9
Download citation
DOI: https://doi.org/10.1007/3-540-45428-4_9
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43818-2
Online ISBN: 978-3-540-45428-1
eBook Packages: Springer Book Archive