Detection of unknown uncertainties in simulation based systems verification process

Early detection of simulation inaccuracies is critical in system development for increasing result certainty, focusing physical experimentation, and avoiding incorrect conclusions about system performance. This research develops a quantitative methodology to detect “unknown uncertainties“ or “unknown unknowns” within system simulations, in the presence of “known uncertainties“ or “known unknown”. The work introduces three methods: Uncertainty Level Extrapolation and Trend Analysis, Multi-Scenario Aggregated Normalized Goodness-of-Fit (GOF), and Probability Integral Transform (PIT) Aggregated Moment Matching. By translating engineering validation processes into statistical inference problems, these methods utilize statistical tools to identify unmodeled discrepancies and structural errors. The methods are described and demonstrated across different system architectures, aimed at offering systems engineers with tools for verifying simulation accuracy.

This work is towards a Ph.D. degree under the supervision of Prof. Gil Yudilevitch of Aerospace Engineering, Technion, and Dr. Arkady Lichtsinder, RAFAEL – Advanced Defense Systems.