Building Confidence in Models: Confidence and Philosophy [Systems thinking & modelling series]
This is part 43 of a series of articles featuring the book Beyond Connecting the Dots, Modeling for Meaningful Results.
From an evaluator’s view, the quality of a model is significantly influenced by the evaluator’s world-view and philosophical orientation (if any). Broad world-views or epistemologies1 exist. One key divide in various epistemological theories continues to be between those based on a strong belief in a concrete true reality that our knowledge can accurately capture and those based on a belief that our knowledge is partially or wholly independent from reality.
Epistemological theories that are primarily in the first camp are those such as positivism or empiricism. Theories in the latter camp include constructivism and idealism. Constructivism is a popular theory that claims knowledge is constructed with social context and historical time. Our presentation of confidence building for narrative models in this chapter is implicitly in line with a constructivist theory of knowledge.
In our discussion of confidence building we repeatedly refer to matching the beliefs of the audience. We recommend creating simulations and behavior in our models that match an audience’s expectations for the behavior of the system. This is distinct from saying that you should match the reality of true systems. Ideally, true behavior of the system and an audience’s mental models of the system should be equivalent, but in practice they may well differ. Although confidence in a model will be boosted by strictly matching the mental models of an audience, a truly effective narrative model should be persuasive enough to change the mental models of an audience.
Our discussion of predictive models from the previous chapter does not fall within a constructivist world-view, as we are claiming that we can obtain objective “outside-ourselves” measures of predictive accuracy. It should go without saying that predictive models may not be accurate reflections of reality, even in their own terms. The mathematics of a predictive model may be unrelated to the true system that is being modeled, yet is may still create accurate predictions. As such, our discussion of predictive models is not really a positivist or empiricist one. Instead, this discussion would fall under the epistemological theories of pragmatism or instrumentalism. These claim that a theory or model should be assessed on how well it predicts, which may be independent of the truth of the theory itself.
| Exercise 5-7 |
|---|
| You are asked to evaluate a model simulating the growth of an endangered species in its habitat. What tests and demonstrations would you like to see in order to trust the model and recommend its use in practice? |
| Exercise 5-8 |
|---|
| You are asked to evaluate a model simulating the potential adoption for a new product at your company. The basic results of the model are very encouraging for the product, suggesting it would make a significant return on investment.
What tests and demonstrations of the model would you like to see in order to recommend production based on the model results? |
Next edition: The Process of Modeling: Introduction.
Article sources: Beyond Connecting the Dots, Insight Maker. Reproduced by permission.
Header image source: Beyond Connecting the Dots.
Notes:
- From the Greek word “epistēmē” meaning “knowledge” or “understanding”, epistemology is the branch of philosophy describing how we understand or come to know the world around us. ↩
