What is parameter sweep?

A parameter sweep is a systematic series of simulation runs in which one or more inputs take a set of values across a chosen range. Each run records an output, and the results are compared. It matters because it shows the shape of the relationship between inputs and results, including tipping points, that a single run cannot show.

How it works

First pick the parameter, the range, and the number of steps, for example a price from 10 to 50 in steps of 10. Run the model for each value, with everything else held fixed, and record one output per run, such as the final stock or the peak. Plot the output against the input. A sweep over two inputs gives a grid and is often drawn as a heat map. The number of runs multiplies with each added parameter, so sweeps over many parameters get expensive fast.

A sweep is the engine behind sensitivity analysis, behind scenario analysis of many cases, and behind a coarse calibration. For stochastic models, repeat each setting several times and average. If you want the best setting rather than the full picture, use simulation optimization, which searches with fewer runs.

Example

Say a subscription model has a monthly churn rate that is uncertain. Sweep churn from 2 to 10 percent in steps of 2 percent and record the customer count after 24 months from a start of 1,000 customers with 100 new each month. Results are about 2,540 at 2 percent, 1,940 at 4 percent, 1,520 at 6 percent, 1,220 at 8 percent, and 1,000 at 10 percent. The first step costs 600 customers and the last only 220, so the effect is strongest at low churn. This is an illustration with round numbers.

How to model it in Machinations

Put the parameter in a Register and step it with a Lever between Plays, noting the outcome each time. When the model has randomness, run a Forecast at each value: it runs many Plays at that one setting and shows the range of results. Chart the Pool that holds the outcome. Repeat for each value and compare the end points.