What is model validation?

Model validation is the set of checks that tell you whether a model can be trusted to answer the question it was made for. It does not prove a model is true, only that it is adequate for its purpose. It matters because decision makers act on model results, and unchecked models can mislead.

How it works

Validation combines several tests. Structure tests ask whether the stocks, flows, and loops match what experts know about the real system. Unit checks confirm that each equation balances in units. Extreme-condition tests set inputs to zero or very high values and check that the model behaves sensibly, for example that nothing goes negative. Behavior tests compare the output with data, ideally data the model was not calibrated on.

Two further checks are common. The time step test halves the step and confirms that the result barely changes. Sensitivity analysis shows whether the conclusions survive reasonable changes in uncertain inputs. Validation is never finished, because trust is built by accumulating evidence, and it should be repeated whenever the model or its purpose changes. A model checked against the reference mode is a start, not an end.

Example

Say a team builds a model of warehouse stock and has 12 months of data. They calibrate on months 1 to 8. For months 9 to 12 the real stock is 480, 450, 470, 520, and the model gives 470, 440, 480, 500. Errors are under 5 percent, and when they set incoming orders to zero the stock falls to zero and stays there, as expected. The model passes for the purpose of monthly planning. This is an illustration with round numbers.

How to model it in Machinations

Chart the Pool that represents the quantity you have real data for, and keep the recorded series beside it, for example in a spreadsheet. Play the diagram with the calibration inputs, then with new inputs, and compare. For extreme tests, move a Lever to zero or to a very large value. For random models, run a Forecast to see whether the real data falls inside the range.