Abstract
This paper studies how an optimal Neural Network (NN) can be selected that is later used for constructing the highest quality delta-based Prediction Intervals (PIs). It is argued that traditional assessment criteria, including RMSE, MAPE, BIC, and AIC, are not the most appropriate tools for selecting NNs from a PI-based perspective. A new NN model selection criterion is proposed using the specific features of the delta method. Using two synthetic and real case studies, it is demonstrated that this criterion outperforms all traditional model selection criteria in terms of picking the most appropriate NN. NNs selected using this criterion generate high quality PIs evaluated by their length and coverage probability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
Lehtokangas, M.: Modelling with constructive backpropagation. Neural Networks 12, 707–716 (1999)
Garcia-Pedrajas, N., Hervas-Martinez, C., Munoz-Perez, J.: COVNET: a cooperative coevolutionary model for evolving artificial neural networks. IEEE Transactions on Neural Networks 14, 575–596 (2003)
Zhang, G., Eddy Patuwo, B., Hu, M.Y.: Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting 14, 35–62 (1998)
Lowe, D., Zapart, C.: Point-Wise Confidence Interval Estimation by Neural Networks: A Comparative Study based on Automotive Engine Calibration. Neural Computing & Applications 8, 77–85 (1999)
Hwang, J.T.G., Ding, A.A.: Prediction Intervals for Artificial Neural Networks. Journal of the American Statistical Association 92, 748–757 (1997)
Heskes, T.: Practical confidence and prediction interval. Advances in neural information processing systems 9, 176–182 (1997)
Nix, D.A., Weigend, A.S.: Estimating the mean and variance of the target probability distribution. In: IEEE International Conference on Neural Networks, pp. 55–60 (1994)
Khosravi, A., Nahavandi, S., Creighton, D.: Construction of Optimal Prediction Intervals for Load Forecasting Problems. IEEE Transactions on Power Systems 25, 1496–1503 (2010)
Chiu, C.-C., Kao, L.-J., Cook, D.F.: Combining a neural network with a rule-based expert system approach for short-term power load forecasting in Taiwan. Expert Systems with Applications 13, 299–305 (1997)
Ho, S.L., Xie, M., Tang, L.C., Xu, K., Goh, T.N.: Neural network modeling with confidence bounds: a case study on the solder paste deposition process. IEEE Transactions on Electronics Packaging Manufacturing 24, 323–332 (2001)
Khosravi, A., Nahavandi, S., Creighton, D.: A prediction interval-based approach to determine optimal structures of neural network metamodels. Expert Systems with Applications 37, 2377–2387 (2010)
Veaux, R.D.d., Schumi, J., Jason, S., Ungar, L.H.: Prediction Intervals for Neural Networks via Nonlinear Regression. Technometrics 40, 273–282 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Khosravi, A., Nahavandi, S., Creighton, D. (2010). Developing a Robust Prediction Interval Based Criterion for Neural Network Model Selection. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds) Neural Information Processing. Models and Applications. ICONIP 2010. Lecture Notes in Computer Science, vol 6444. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17534-3_89
Download citation
DOI: https://doi.org/10.1007/978-3-642-17534-3_89
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17533-6
Online ISBN: 978-3-642-17534-3
eBook Packages: Computer ScienceComputer Science (R0)