By Dr. Barna Szabó
Engineering Software Research and Development, Inc.
St. Louis, Missouri USA

In the engineering sciences, mathematical models are based on the equations of continuum mechanics, heat flow, Maxwell, Navier-Stokes, or some combination of these. These equations have been validated and their domains of calibration are generally much larger than the expected domain of calibration of the model being developed. In the terminology introduced by Lakatos [1], the assumptions incorporated in these equations are called hardcore assumptions, and the assumptions incorporated in the other constituents of a model are called auxiliary hypotheses. Model development is concerned with the formulation, calibration, and validation of auxiliary hypotheses. 

Assume, for example, that we are interested in predicting the length of a small crack in a flight-critical aircraft component, caused by the application of a load spectrum. In this case, the mathematical model comprises the equations of continuum mechanics (the hardcore assumptions) and the following auxiliary hypotheses: (a) a predictor of crack propagation, (b) an algorithm that accounts for the statistical dispersion of the calibration data, and (c) an algorithm that accounts for the retardation effects of tensile overload events and the acceleration effects of compressive overload events.

The auxiliary hypotheses introduce parameters that have to be determined by calibration. In our example, we are concerned with crack propagation caused by variable-cycle loading. In linear elastic fracture mechanics (LEFM), for example, the commonly used predictor of crack increment per cycle is the difference in the values of the stress intensity factors between subsequent high and low positive values, denoted by ΔK.

The relationship between crack increment per cycle, denoted by Δa, and the corresponding ΔK value is determined through calibration experiments. Various hypotheses are used to account for the cycle ratio. Additional auxiliary hypotheses account for the statistical dispersion of crack length and the retardation and acceleration events caused by loading sequence effects. The formulation of auxiliary hypotheses is a creative process. Therefore, model development projects must be open to new ideas. Many plausible hypotheses have been and can yet be proposed. Ideally, the predictive performance of competing alternatives would be evaluated using all of the qualified data available for calibration and the models ranked accordingly. Given the stochastic nature of experimental data, predictions should be in terms of probabilities of outcome. Consequently, the proper measure of predictive performance is the likelihood function. Ranking must also account for the size of the domain of calibration [2]. The volume of experimental information tends to increase over time. Consequently, model development is an open-ended activity encompassing subjective and objective elements.

Example: Calibration and Ranking Models of Crack Growth in LEFM

Let us suppose that we want to decide whether we should prefer the Walker [3] or Forman [4] versions of the predictor of crack propagation based on experimental data consisting of specimen dimensions, elastic properties, and tabular data of measured crack length (a) vs. the observed number of load cycles (N) for each cycle ratio (R). For the sake of simplicity, we assume constant cycle loading conditions

The first step is to construct a statistical model for the probability density of crack length, given the number of cycles and the characteristics of the load spectrum. The second step is to extract the Δa/ΔN vs. ΔK data from the a vs. N data where ΔK is determined from the specimen dimensions and loading conditions. The third step is to calibrate each of the candidate hypotheses. This involves setting the predictor’s parameters so that the likelihood of the predicted data is maximum. This process is illustrated schematically by the flow chart shown in Fig. 1.

Finally, the calibration process is documented and the domain of calibration is defined. The model that scored the highest likelihood value is preferred. The ranking is, of course, conditioned on the data available for calibration. As new data are acquired, the calibration process has to be repeated, and the ranking may change. It is also possible that the likelihood values are so close that the results do not justify preferring one model over another. Those models are deemed equivalent. Model development is an open-ended process. No one has the final say.

Opportunities for Improvement

To my knowledge, none of the predictors of crack propagation used in current professional practice have been put through a process of verification, validation, and uncertainty quantification (VVUQ) as outlined in the foregoing section. Rather, investigators tend to follow an unstructured process, whereby they have an idea for a predictor, and, using their experimental data, show that, with a suitable choice of parameters, their definition of the predictor works. Typically, the domain of calibration is not defined explicitly but can be inferred from the documentation. The result is that the relative merit of the ideas put forward by various investigators is unknown and the domains of calibration tend to be very small. In addition, no assurances are given regarding the quality of the data on which the calibration depends. In many instances, only the derived data (i.e. the Δa/ΔN vs. ΔK data), rather than the original records of observation (i.e. the a vs. N data) are made available. This leaves the question of whether the ΔK values were properly verified unanswered.

The situation is similar in the development of design rules for metallic and composite materials: Much work is being done without the disciplined application of VVUQ protocols.  As a result, most of that work is being wasted. 

For example, The World Wide Failure Exercise (WWFE), an international project with the mission to find the best method to accurately predict the strength of composite materials, failed to produce the desired result.  See, for example, [5].  A highly disturbing observation was made by Professor Mike Hinton, one of the organizers of WWFE, in his keynote address to the 2011 NAFEMS World Congress [6]: “The theories coded into current FE tools almost certainly differ from the original theory and from the original creator’s intent.”  I do not believe that significant improvements in predictive performance occurred since then.

In my view, progress will not be possible unless and until VVUQ protocols are adopted for model development projects.  These protocols play a crucial role in the evolutionary development of mathematical models. 

References

Related Blogs:

• Where Do You Get the Courage to Sign the Blueprint?
• A Memo from the 5th Century BC
• Obstacles to Progress
• Why Finite Element Modeling is Not Numerical Simulation?
• XAI Will Force Clear Thinking About the Nature of Mathematical Models
• The Story of the P-version in a Nutshell
• Why Worry About Singularities?
• Questions About Singularities
• A Low-Hanging Fruit: Smart Engineering Simulation Applications
• The Demarcation Problem in the Engineering Sciences