Abstract
Validation of an estimated model is not a trivial task because it depends on the purpose of the model, which usually defines the most important features of the model. Thus, in a validation process, the use of diverse tools that exploit different domains is recommended. Here, with this aim, a scale for model validation is proposed that combines the Normalized Root Mean Square Error (NRMSE) with two new indices: the coherence-based index and the fourth-order cross-cumulant index. The proposed scale was used for the validation of three models: the Logistic Map, the Duffing–Ueda oscillator, and the Buck converter. The results demonstrated that the proposed model validation scale produces a more complete validation process that takes into account both time and frequency information and provides robustness against Gaussian noise.
Similar content being viewed by others
References
Eriksson R, Söder L (2012) WAMS based identification for obtaining linear models to coordinate controllable devices. Electr Eng 94:27–36. doi:10.1007/s00202-011-0201-y
Aguirre LA (1996) A tutorial introduction to nonlinear dynamics and chaos, part II: modeling and control. SBA Control Autom 7:50–66
Aguirre LA, Billings SA (1994) Validating identified nonlinear models with chaotic dynamics. Int J Bifurcat Chaos 4:109–125. doi:10.1142/S0218127494000095
Penaranda JRC, Saavedra-Montes AJ (2012) Dynamic model validation via error indexes. In: IEEE 4\({{\rm th}}\) Colombian workshop on circuits and systems, Barranquilla. doi:10.1109/CWCAS.2012.6404063
Willmott CJ, Matsuura K (2005) Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res 30:79–82. doi:10.3354/cr030079
Routtenberg T, Tabrikian J (2010) Blind MIMO-AR system identification and source separation with finite-alphabet. IEEE Trans Signal Process 58:990–1000. doi:10.1109/TSP.2009.2036043
Bendat J, Piersol A (2000) Random data analysis and measurement procedures, 3rd edn. Wiley, New York
Miranda de Sá AMFL, Ferreira DD, Dias EW, Mendes EMAM, Felix LB (2009) Coherence estimate between a random and a periodic signal: bias, variance, analytical critical values, and normalizing transforms. J Frankl Inst 346:841–853. doi:10.1016/j.jfranklin.2009.07.009
Georgieva P, Ralescua A, Ralescub D (2007) Cross-cumulants measure for independence. J Stat Plan Inference 137:1085–1098. doi:10.1016/j.jspi.2006.06.034
Mendel JM (1991) Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications. Proc IEEE 79:278–305. doi:10.1109/5.75086
Gomis-Bellmunt O, Ikhouane F, Castell-Vilanova P, Bergas-Jané J (2007) Modeling and validation of a piezoelectric actuator. Electr Eng 89:629–638. doi:10.1007/s00202-006-0049-8
Tsang KM, Chan WL (2006) A search algorithm for the identification of multiple inputs nonlinear systems using orthogonal least squares estimator. Electr Eng 88:357–365. doi:10.1007/s00202-005-0293-3
Zhu Q, Yuan X, Wang H (2012) An improved chaos optimization algorithm-based parameter identification of synchronous generator. Electr Eng 94:147–153. doi:10.1007/s00202-011-0226-2
Orjuela R, Marx B, Ragot J, Maquin D (2013) Nonlinear system identification using heterogeneous multiple models. Int J Appl Math Comput Sci 23:103–111. doi:10.2478/amcs-2013-0009
Xiaojun Z, Chunhua Y, Weihua G (2014) Nonlinear system identification and control using state transition algorithm. Appl Math Comput 226:169–179. doi:10.1016/j.amc.2013.09.055
Tavoosi J, Suratgar AA, Menhaj MB (2016) Nonlinear system identification based on a self-organizing type-2 fuzzy RBFN. Eng Appl Artif Intell 54:26–38. doi:10.1016/j.engappai.2016.04.006
Tang Y, Han Z, Liu F, Guan X (2016) Identification and control of nonlinear system based on Laguerre-ELM Wiener model. Commun Nonlinear Sci Numer Simul 38:192–205. doi:10.1016/j.cnsns.2016.02.01
Aguilar-Leal O, Fuentes-Aguilar RQ, Chairez I, García-González A, Huegel JC (2016) Distributed parameter system identification using finite elemento differential neural networks. Appl Soft Comput 43:633–642. doi:10.1016/j.asoc.2016.01.004
Oukaci A, Toufouti R, Dib D, Atarsia L (2016) Comparison performance between sliding mode control and nonlinear control, application to induction motor. Eng Electr. doi:10.1007/s00202-016-0376-3
Scheinerman ER (1996) Invitation to dynamical systems. Prentice Hall, Upper Saddle River
Ueda Y (1985) Random phenomena resulting from nonlinearity in the system described by Duffing’s equation. Int J Nonlinear Mech 20:481–491
Aguirre LA, Billings SA (1994) Model reference control of regular and chaotic dynamics in the Duffing–Ueda oscillator, Part I. IEEE Trans Circuits Syst 41:477–480. doi:10.1109/81.298361
Aguirre LA, Donoso-Garcia PF, Santos-Filho R (2000) Use of a priori information in the identification of global nonlinear model—a case study using a buck converter. IEEE Trans Circuits Syst I 47:1081–1085. doi:10.1109/81.855463
Cardoso JF (1999) High-order contrasts for independent component analysis. Neural Comput 11:157–192. doi:10.1162/089976699300016863
Hyvarinen A, Karhunen J, Oja E (2001) Independent component analysis. Wiley, New York
Carter CG, Knapp CH, Nuttall AH (1973) Statistics of the estimate of the magnitude-coherence function. IEEE Trans Audio Electroacoust 21:338–389
Martins SAM, Nepomuceno EG, Barroso MFS (2013) Improved structure detection for polynomial NARX models using a multiobjective error reduction ratio. J Control Autom Electr Syst 24:764–772. doi:10.1007/s40313-013-0071-9
Acknowledgments
The authors would thank the National Council for Scientific and Technological Development (CNPq-Brazil) and the Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG-Brazil) for supporting this work.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ferreira, D.D., Nepomuceno, E.G., Cerqueira, A.S. et al. A model validation scale based on multiple indices. Electr Eng 99, 325–334 (2017). https://doi.org/10.1007/s00202-016-0430-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00202-016-0430-1