By Dr. Barna Szabó
Engineering Software Research and Development, Inc.
St. Louis, Missouri USA

Mathematical models have become indispensable sources of information on which technical and business decisions are based. It is therefore vitally important for decision-makers to know whether relying on the predictions of mathematical models is justified. When properly used, numerical simulation can be a major corporate asset. However, it can become a major corporate liability if the reliability of predictions is not guaranteed.

Resource Allocation

Project management is responsible for allocating resources to numerical simulation and physical experimentation.  Consider the two extreme cases: (a) If the decision is not to use numerical simulation, just rely on experimentation, then management is adopting the methodology of the Wright brothers.  (b) If the decision is not to do experiments, and just rely on finite element modeling, then management will risk repeating the costly mistakes of the F-35 program.  The correct balance depends on the justified degree of confidence in the predictive performance of numerical simulation [1].

Simulation Governance

Simulation governance is a managerial function concerned with the assurance of reliability of information generated by numerical simulation. The term was introduced in 2011 and specific technical requirements were addressed from the perspective of mechanical design in 2012 [2]. At the 2017 NAFEMS World Congress in Stockholm, simulation governance was identified as the first of eight “big issues” in numerical simulation.

A plan for simulation governance has to be tailored to fit the mission of each organization or department within an organization.  We consider three types of mission in the following.  A summary of the main points is presented in the table below.

• If the mission is the application of established design rules, then the goal is to verify that a quantity of interest  does not exceed its allowable value .  Simulation governance is concerned with standardization of recurring numerical simulation tasks through the creation of smart applications. Smart applications, also called “simulation apps”, are expert-designed in such a way that the use of those applications does not require expertise in numerical simulation. The preservation and maintenance of corporate know-how and institutional knowledge are among the important objectives of simulation governance. The productivity of newly hired engineers significantly increases if routine simulation procedures are standardized so that applications consistently produce certifiable results. Economic benefits are realized through improved productivity and improved reliability.

• If the mission is the formulation of design rules, for example, to establish allowable values for a new material system, then the plan should focus on the collection, maintenance, and documentation of experimental data, management of solution and data verification procedures, revision and updating mathematical models in the light of new information collected from physical experiments and field observations. Economic benefits: Substantial savings through reduction of the number of tests on the sub-component, component, sub-assembly, and assembly levels.
• If the mission is to support condition-based maintenance (CBM), then the goal is to determine the probability that the number of cycles to failure  is smaller than a given number of cycles .  The main activities are: Collection, maintenance, and documentation of fatigue data and unit-specific information on service data, standardization of recurring analysis tasks. Economic benefits: Substantial savings through improved disposition decisions.

Observe that solution, data, and code verification are common to all three types of mission.  Validation and uncertainty quantification are performed in model development projects.

Recognizing that technology advances and the available information increases over time, planning must incorporate data management and systematic updates of simulation practices so as to take advantage of new data, ideas, and technology.  Model development projects are open-ended.  Validation and definition of the domain of calibration is conditional on the available data.  Since the available data increases over time, and new ideas are likely to be proposed, there will be opportunities to revise and update mathematical models.  No one has the final word in model development [3].

Questions for Management

1. Does the engineering team possess the technical expertise and software tools to perform numerical simulation projects effectively?
2. Are mathematical models being defined independently from the way numerical approximations are obtained? – A common mistake is to conflate model definition with its numerical solution, as in “finite element modeling”.
3. Are the errors in numerical approximation properly estimated, controlled, and reported?
4. Physical testing is necessarily tied to the specific test conditions.  Generalization to a larger set of conditions requires a mathematical model.  Testing without a plan to generalize the results does not make sense. – Are physical testing projects properly planned, executed and analyzed in your organization? 
5. Do decision-makers have sufficient confidence in the predictive performance of mathematical models to reduce the number and complexity of physical tests through reliance on numerical simulation?
6. Are the experimental data properly documented and archived?
7. How well are the various professional skills needed for successful execution of numerical simulation projects coordinated?
8. Are mathematical models properly calibrated and their domains of calibration properly defined and documented?
9. How are new data incorporated into model updates when the data fall (a) within the domain of calibration and (b) outside of the domain of calibration?
10. Are the procedures of verification, validation, and uncertainty quantification (VVUQ) properly and consistently applied?
11. Have opportunities to improve design workflows through standardization been fully explored?
12. What is the estimated economic value of your current numerical simulation activities? – Without simulation governance, that value can be a large negative number.

[1] B. Szabó and R. Actis, Planning for Simulation Governance and Management: Ensuring Simulation is an Asset, not a Liability. Benchmark, July 2021, pages 8-12.

[2] B. Szabó and R. Actis, “Simulation governance: Technical requirements for mechanical design,” Computer Methods in Applied Mechanics and Engineering, vol. 249, pages 158-168, 2012.

[3] B. Szabó and I. Babuška, “Methodology of model development in the applied sciences,” Journal of Computational and Applied Mechanics, vol. 16, no. 2, pp. 75-86, 2021 (open source).