Building Confidence in Models: Introduction [Systems thinking & modelling series]

This is part 39 of a series of articles featuring the book Beyond Connecting the Dots, Modeling for Meaningful Results.

When used correctly, the modeling techniques presented in this interactive learning environment (ILE) result in models that are powerful and persuasive tools. As with any model, however, concerns and questions will invariably be raised that could cause users to doubt the results. You can use a number of techniques to help preemptively address these concerns and increase an audience’s confidence in your model.

The idea of building confidence in a model is traditionally tied to the standard concept of model verification and validation. We dislike this conceptual approach to assessing models, as it implies that a model can go through a process to get a big fat “VALID” or “VERIFIED” stamp on it. Returning to Box’s quote that “all models are wrong, but some are useful”, in reality, all models are wrong and none of them are completely valid – period. However, models can be useful, especially narrative models in which the audience has confidence.

We favor the conceptual approach put forth by Forrester and Senge¹. This approach states that no single test or suite of tests will verify or validate a model, and that validity should instead be thought of as a function of confidence. This view differs from that held by some modelers and laypeople. As Forrester and Senge note, “the notion of validity as equivalent to confidence conflicts with the view many seem to hold, which equates validity with absolute truth.” We share their belief that confidence in a model is built from a variety of tests that, though they cannot prove anything, together comprise a persuasive case for the quality of a model.

Confidence needs to be developed in three distinct areas:

1. That the model itself is well designed.
2. That the model is implemented correctly.
3. The conclusions drawn from the model are accurate.

In the next articles we will look at each of these areas in detail. We will explore the different tests and tools that can be used to build confidence for each area.

Next edition: Building Confidence in Models: Model Design.

Article sources: Beyond Connecting the Dots, Insight Maker. Reproduced by permission.

Header image source: Beyond Connecting the Dots.

Reference: