[4] Verification and validation is an iterative process that takes place throughout the development of a model.
These include, but are not limited to, having the model checked by an expert, making logic flow diagrams that include each logically possible action, examining the model output for reasonableness under a variety of settings of the input parameters, and using an interactive debugger.
[3] Naylor and Finger [1967] formulated a three-step approach to model validation that has been widely followed:[1] Step 1.
[4] Face validity is tested by having users and people knowledgeable with the system examine model output for reasonableness and in the process identify deficiencies.
[1] For example, if a simulation of a fast food restaurant drive through was run twice with customer arrival rates of 20 per hour and 40 per hour then model outputs such as average wait time or maximum number of customers waiting would be expected to increase with the arrival rate.
For example, the number of servers in a fast food drive through lane and if there is more than one how are they utilized?
[6] The use of this assumptions must be restricted to assure that the model is correct enough to serve as an answer for the problem we want to solve.
Statistical hypothesis testing using the t-test can be used as a basis to accept the model as valid or reject it as invalid.
To perform the test a number n statistically independent runs of the model are conducted and an average or expected value, E(Y), for the variable of interest is produced.
Then the test statistic, t0 is computed for the given α, n, E(Y) and the observed value for the system μ0 If reject H0, the model needs adjustment.
Increasing the sample size decreases the risk of a type II error.
A statistical technique where the amount of model accuracy is specified as a range has recently been developed.
The operating characteristic (OC) curve is the probability that the null hypothesis is accepted when it is true.
[7] Confidence intervals can be used to evaluate if a model is "close enough"[1] to a system for some variable of interest.
To perform the test a number, n, statistically independent runs of the model are conducted and a mean or expected value, E(Y) or μ for simulation output variable of interest Y, with a standard deviation S is produced.
An interval, [a,b], is constructed by where is the critical value from the t-distribution for the given level of significance and n-1 degrees of freedom.
ASME V&V 10 provides guidance in assessing and increasing the credibility of computational solid mechanics models through the processes of verification, validation, and uncertainty quantification.
[9] ASME V&V 20 provides a detailed methodology for validating computational simulations as applied to fluid dynamics and heat transfer.