The Importance of Dedicated Experiments to Support Validation and Calibration Activities
Genetha Gray, (Sandia National Labs), firstname.lastname@example.org,
Monica Martinez-Canales, (Sandia National Labs), mmarti7@sandia,gov,
Cheryl Lam, (Sandia National Labs), clam@sandia,gov, and
Brian Owens, (Sandia National Labs), email@example.com
In recent years, numerical modeling and simulation have been used to augment and replace physical experiments in the study and design of complex engineering and physics systems. Moreover, the results of these simulations are often considered by decision makers in areas such as defense and national security. Therefore, validation and verification (V&V) activities have become critical for determining simulation-based confidence and predictive capabilities. For example, code verification must be used to confirm that the underlying equations are being solved correctly. In addition, validation processes should be applied to answer questions of correctness of the equations for the phenomena being modeled and the application being studied. Moreover, validation metrics must be carefully chosen in order to explicitly compare experimental and computational results and quantify the uncertainties in these comparisons.
Data is the driver of the V&V process. While existing experimental data is often used for validation activities, in some cases, existing data may be inadequate or inappropriate for comprehensive validation. For example, data may only be available in limited quantities or data sets might not be replicated making quantification of measurement variability within experimental factors impossible. Selecting new experiments requires tools that elucidate the behavior being studied as well as how it will be tested. The goal is to cover the space created by the experimental factors of importance as well as to test at extremes of testable space to ensure confidence in extrapolating the model to untestable regions. The variety of experimental conditions, experimental measurement errors, and part-to-part variation must also be considered.
Overall, the V&V process for modeling and simulation can provide the best estimates of what can happen and the likelihood of it happening when uncertainties are taken into account. In order to carry out the validation activities, experiments must be carefully planned and executed to provide adequate and appropriate data. We will describe this process for the validation activities related to Xyce, an electrical circuit simulator developed at Sandia. We will also discuss how V&V can be applied to the processes of decision making and risk analysis.