Demystifying systemic thinking for evaluation in post-normal times
This article is part of an ongoing series of articles on KM in international development.
When you suggest to someone that you’re interested in using systems thinking in your project, you’ll often hear “we tried that once…” or “I like the idea of it but I’m not really sure how I would apply it” or “I get what it is, but I don’t really get what it is!” These are all comments that I have heard over the last three years when asked, “So what are you working on?”
I’m a systems thinking theorist and for the last three years I’ve been working with my colleagues Ellen Lewis (Ethos of Engagement Consulting) and Shravanti Reddy (UN Women’s Independent Evaluation Service) to write a comprehensive field guidance on systemic evaluation. The Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized voices (ISE4GEMs) Guide is now on the BetterEvaluation website.
The three of us have embraced systemic thinking and wanted to provide guidance that demystifies systemic thinking and complexity science concepts, walks one through the process of evaluation using systemic thinking, and provides tools, examples and activities, to ‘be’ an inclusive systemic evaluator.
As Bob Williams … said about the ISE4GEMs guide on the Pelican listserve recently:
What makes it almost unique is that it is essentially a soup to nuts evaluation manual rather than a general description of a particular evaluation approach.
So what exactly is ‘systemic evaluation’? In the guide, we make an important differentiation between ‘systematic’ and ‘systemic’. Put very simply, to think systematically, one engages in a process that can be thought of as, predictable, thorough and controlled, as a way to think about parts of a system that may interact. While to think systemically encourages us to be more critical about how those parts interact. What are the opportunities, constraints and relationships of parts within a system that tell us much about the system as a whole? We write:
Systemic evaluation asks questions to capture the conditions and changes relevant to an intervention, the changes it produces and opportunities for learning and empowerment. In the course of an evaluation, one may uncover what else was going on—the external effects, spill over of other efforts or policies, uncontrolled events (such as political conflict or environmental disasters), or unexpected facilitators or inhibitors of change that may or may not have been part of the original plan. (ISE4GEMs, p. 8)
We think that these skills are apposite for working in uncertain times, which both characterises and clichés life in the 21st Century, but more specifically, if we are to meet the goals of the 2030 Agenda: the [Sustainable Development Goals].
Professor Thomas Schwandt summed it up perfectly in a captivating presentation he gave at the 2018 European Evaluation Society conference. He argued that we live in ‘post-normal’ times. Drawing on the seminal work by Silvio Funtowicz and Jerome Ravetz, post-normal science contrast to traditional problem-solving strategies by accounting for complexity, emergence and plurality. In a democratic mode it responds to what might be appropriate, rather than what should be done. He suggests we think about evaluation in new ways, as we work in a post-normal evaluation era. One that calls for us to use systemic thinking to make sense of new-normal levels of social, political, economic and ecological complexity.
And this is what, we believe as authors, we are endeavouring to achieve with the release of the ISE4GEMs. More than a description of systems thinking, it provides a primer in systemic evaluation for practice. Our intention is to be your thought partner, to help you reflect on the work you are being asked to do as you unfold your understanding of the evaluation. We then provide guidance with the detailed steps one might take, to do a systemic evaluation. We created tools to work with and resources to draw on. And, further to this, we’ve used the approach, as we were writing it. So many examples from our practice and others are embedded in the guide.
Why is this different? Because there are not too many guides that have reviewed the literature and carefully chosen techniques, strategies, ideas and concepts from across the disciplines of evaluation, systems sciences and feminism, and spill over into a range of social and environmental science disciplines as well. We have selected key ideas and established a framework that generates practitioners’ knowledge and capacity to ‘do’ systems thinking, rather than just have a better general awareness of the theory.
Get involved
We invite all evaluators, emerging or experienced, who are interested in using the ISE4GEMs to have a go and let us know as your thought partners what works, what doesn’t and how we can make the approach better. We have an active research program to investigate inclusive systemic evaluation. We want people to experience the approach and tell us how it needs to be improved, that includes the ISE4GEMs toolkit, theory (Part A) and guide (Part B). Our community of practice will shape the ISE4GEMsV2 and together we can make better sense of our complex world.
Article source: Better Evaluation, CC BY-NC 3.0.
Header image source: authors.
