James C. Cavendish, General Motors Research and Development Center
Computer simulations and math models are used in industry to design, understand
and control various physical systems and processes and to predict the performance
of these systems and processes under various operating conditions. The two main
approaches to establishing credible confidence statements about the predictive
capabilities of computer models are verification and validation. Verification
is the process of determining that a computer model (that is, the operational
computer program or "code") is a correct (i.e. bug-free) and accurate
implementation of a conceptual model of a physical system or process. Model
validation is the process of determining the degree to which output from a computerized
model accurately represents reality. In this talk we present a framework that
enables the implementation of the validation process.
This framework is being developed as part of a Research Agreement between the National Institute of Statistical Sciences (NISS) and General Motors. Underlying the proposed validation framework is a systematic six-step procedure based upon a Bayesian statistical methodology. The Bayesian methodology is particularly suited to treating the major issues associated with the validation process: quantifying multiple sources of error and uncertainty in math models; combining multiple sources of information; and updating validation assessments as new information is acquired. Moreover, it allows inferential statements to be made about predictive error associated with model predictions in untested situations (that is, prediction beyond the validation comparisons between calculations and experimental outcomes). The framework is described with the help of a test bed model that provides context for each of the six steps in the proposed validation process.
See related publication: M.J. Bayarri, J.O. Berger, D. Higdon, M. Kennedy, A. Kottas, R. Paulo, J. Sacks, J.A. Cafeo, J.C. Cavendish, C-H lin, J. Tu (2002), "A Framework for Validation of Computer Models", Technical Report 128, NISS www.niss.org/downloadabletechreports.html.
Return to Program