Skip to main content

Methods for the Invalidation/Validation of Simulation Models

  • Chapter
Testing and Validation of Computer Simulation Models

Part of the book series: Simulation Foundations, Methods and Applications ((SFMA))

  • 1970 Accesses

Abstract

In the context of simulation model development, the word “validation” is used to describe the process of establishing the extent to which a given model is an accurate representation of the corresponding real-world system. In practice, the emphasis should always be on “invalidation” since a model can never be proved to be, in any general sense, “correct”. This chapter provides an introduction to methods to be used for this process and includes quantitative methods, such as those based on system identification and parameter-estimation principles, barrier certificate methods, techniques based on concepts of model distortion and methods based on parameter sensitivity analysis. Methods of face validation, involving a more subjective approach and the opinion of experts, familiar with the real system, are also considered. That type of validation process is illustrated using an example of the development and testing of a complex simulation of a hydro-turbine generating system. Further sections of this chapter are devoted to approaches based on comparison with other models, the choice of data sets for model testing and validation, the validation of sub-models and generic models, issues relating to the validation of distributed parameter models and the validation of discrete-event and hybrid models. Before the final discussion section there is also a short review of issues arising in the acceptance or upgrading of models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Oberkampf WL (2007) Predictive capabilities in computational science and engineering. Presented at OASCR Applied Mathematics PI Meeting, Lawrence Livermore National Laboratory, 22–24 May 2007. http://science.energy.gov/~/media/ascr/pdf/workshops-conferences/mathtalks/Oberkampf.pdf. Accessed 10 June 2015

  2. Anderson J, Papachristodoulou A (2009) On validation and invalidation of biological models. BMC Bioinforma 10:132. doi:10.1186/1471-2105-10-132, http://www.biomedcentral.com/1471-2105/10/132. Accessed 10th June 2015

    Article  Google Scholar 

  3. Gore R, Diallo S (2013) The need for usable formal methods in verification and validation. In: Pasupathy R, Kim S-H, Tolk A et al (eds) Proceedings of the 2013 winter simulation conference. IEEE, Washington, DC, pp 1257–1268. doi:10.1109/WSC.2013.6721513

    Google Scholar 

  4. Heitmeyer CL (2007) Formal methods for specifying, validating and verifying requirements. J Univ Comput Sci 13(5):607–618

    Google Scholar 

  5. Gustavsson I (1972) Comparison of different methods for identification of industrial processes. Automatica 8(2):127–142

    Article  MATH  Google Scholar 

  6. Prajna S (2003) Barrier certificates for nonlinear model validation. In: Proceedings 42nd IEEE conference on decision and control 9–12 Dec 2003, vol 3. IEEE, Washington, DC. pp 2884–2889, doi:10.1109/CDC.2003.1273063

  7. Tischler MB, Remple RK (2012) Aircraft and rotorcraft system identification, 2nd edn. AIAA, Reston, VA

    Google Scholar 

  8. Knudsen M (2006) Experimental modelling of dynamic systems: an educational approach. IEEE Trans Educ 49(1):29–38

    Article  MathSciNet  Google Scholar 

  9. Gong M, Murray-Smith DJ (1993) Model reduction by an extended complex curve-fitting approach. Trans Inst Meas Control 15(4):188–198

    Article  Google Scholar 

  10. Balci O, Sargent R (1984) Validation of simulation models via simultaneous confidence intervals. Am J Math Manag Sci 4:375–406

    MATH  Google Scholar 

  11. McFarland J, Mahadevan S (2008) Multivariate significance tests and model calibration under uncertainty. Comput Methods Appl Mech Eng 197(29–32):2407–2479

    Google Scholar 

  12. Huynh DPB, Knezevic DJ, Patera AT (2012) Certified reduced basis model characterization: a frequentistic uncertainty framework. Comput Methods Appl Mech Eng 201:13–24

    Article  MathSciNet  Google Scholar 

  13. Rosenberg JR, Murray-Smith DJ, Rigas A (1982) An introduction to the application of system identification techniques to elements of the neuromuscular system. Trans Inst Meas Control 4(4):187–201

    Article  Google Scholar 

  14. Butterfield MH, Thomas PJ (1986) Methods of quantitative validation for dynamic system models-part 1: theory. Trans Inst Meas Control 8:182–200

    Article  Google Scholar 

  15. Cameron RG (1998) Model validation by the distortion method: linear state space systems control theory and applications. IEE Proc D 139(3):296–300

    Article  Google Scholar 

  16. Thomas PJ (1999) Simulation of industrial processes for control engineers. Butterworth-Heinemann, Oxford

    Google Scholar 

  17. Cameron RG, Marcos RL, De Prada C (1998) Model validation of discrete transfer functions using the distortion method. Math Comput Model Dyn Syst 4(1):58–72

    Article  MATH  Google Scholar 

  18. Gray GJ, Voon LK, Murray-Smith (1997) Application of the distortion method for model validation. In: Troch I, Breitenecker F (eds) Proceedings 2nd MATHMOS VIENNA IMACS symposium on mathematical modelling February 1997. Argesim, Vienna, pp 1033–1038

    Google Scholar 

  19. Butterfield MH, Thomas PJ (1987) Quantitative validation of dynamic models for use in fast reactor safety assessments. In: Proceedings international conference on science and technology of fast reactor safety, Guernsey (UK), 12–16 May 1986. British Nuclear Energy Society, London, pp 145–152

    Google Scholar 

  20. Butterfield MH, Thomas PJ (1986) Methods of quantitative validation for dynamic system models-part 2: applications. Trans Inst Meas Control 8:201–219

    Article  Google Scholar 

  21. Butterfield MH (1990) A method of quantitative validation based on model distortion. Trans Inst Meas Control 12:167–173

    Article  Google Scholar 

  22. Kleijnen JPC (1995) Verification and validation of simulation models. Eur J Oper Res 82:145–162

    Article  MATH  Google Scholar 

  23. Chattee E, Saam NJ, Möhring M (2000) Sensitivity analysis in the social sciences: problems and prospects. In: Suleiman et al (eds) Tools and techniques for social science simulation, chapter 3. Physica Verlag, Heidelberg

    Google Scholar 

  24. Bryce GW, Foord TR, Murray-Smith DJ, Agnew P (1976) Hybrid simulation of water turbine governors. In: Crosbie RE, Hay JL (eds) Simulations councils proceedings series 6(1), Simulation Councils, La Jolla, pp 35–44

    Google Scholar 

  25. Bryce GW, Agnew PW, Foord TR et al (1977) On-site investigation of electrohydraulic governors for water turbines. Proc IEE 124(2):147–153

    Google Scholar 

  26. The Mitre Corporation (2014) Verification and validation of simulation models. In: Mitre systems engineering guide, pp 461–469. www.mitre.org/publications/technical-papers/the-mitre-systems-engineering-guide. Accessed 10 June 2015

  27. Hemez FM (2004) The myth of science-based predictive modelling. In: Proceedings foundations’04 workshop for verification, validation and accreditation (VV&A) in the 21st century, Arizona State University, Tempe, Arizona, 13–15 October 2004. Report LA-UR-04-6829, Los Alamos National Laboratory, USA

    Google Scholar 

  28. Kocijan J, Girard A, Banko B et al (2005) Dynamic system identification with dynamic processes. Math Comput Model Dyn Syst 11(4):411–424

    Article  MathSciNet  MATH  Google Scholar 

  29. Tischler MB (1996) System identification for aircraft flight control development and validation. In: Tischler MB (ed) Advances in aircraft flight control. Taylor and Francis, London, pp 35–69

    Google Scholar 

  30. Tischler MB, Remple RK (2006) Aircraft and rotorcraft system identification. AIAA, Reston

    Google Scholar 

  31. Smith MI, Murray-Smith DJ, Hickman D (2007) Verification and validation issues in a generic model of electro-optic sensor systems. Def Model J Simul 4(1):17–17

    Google Scholar 

  32. Smith MI, Murray-Smith DJ, Hickman D (2007) Mathematical and computer modeling of electro-optic systems using a generic modeling approach. J Def Model Simul 4(1):3–16

    Google Scholar 

  33. Walmsley CW, McCurry MR, Clausen PD et al (2013) Beware the black box: investigating the sensitivity of FEA simulation to modelling factors in comparative biomechanics. PeerJ 1:e204, http://dx.doi.org/10.7717/peerj.204}. Accessed 10 June 2015

    Article  Google Scholar 

  34. Rizzi A, Vos J (1998) Towards establishing credibility in computational fluid dynamics simulations. AIAA J 36(5):668–675

    Article  Google Scholar 

  35. Thompson, DE (2005) Verification, validation and solution quality in computational physics: CFD methods applied to ice sheets, NASA/TM-2005-213453, NASA Technical Reports Server, 37 pp

    Google Scholar 

  36. Padfield GP, Du Val RW (1991) Application areas for rotorcraft system identification: simulation model validation. In: AGARD Lecture Series 178, Rotorcraft System Identification, 12.1-12.30, AGARD, Neuilly-sur-Seine, France

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Murray-Smith, D.J. (2015). Methods for the Invalidation/Validation of Simulation Models. In: Testing and Validation of Computer Simulation Models. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-15099-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-15099-4_7

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-15098-7

  • Online ISBN: 978-3-319-15099-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics