Typically, when you’re asked to conduct an evaluation, you start by looking at background documents to orient yourself to the project. Ideally, those materials include a logic model, results chain diagram, or theory of change illustration—key pieces you need to plan that baseline, midline, summative, or process evaluation you’re about to conduct.
But what if you don’t have this verbal or pictorial map? What’s your way forward? Before you despair, consider this: the lack of a theory or logic model can actually be an opportunity—one that allows you to conduct a better evaluation than ever!
At EnCompass, our collaborative, appreciative approach to evaluation is grounded in systems thinking. Using Appreciative Inquiry with clients and stakeholders as part of the evaluation, we co-learn about the logic behind the project along the way, applying the key principles of systems thinking in evaluation at every step. This is a topic near and dear to my heart, as an evaluator and as a member of the AEA Topical Interest Group on the subject.
The Principles of Systems Thinking in Evaluation
These principles—of interrelationships, perspectives, boundaries, and dynamics—support the development and implementation of a sound and relevant systems-based evaluation that involves, and is transparent to, clients and stakeholders.
Following the four systems thinking principles, our project and evaluation logic development is iterative, involving the evaluation team and our clients and stakeholders in a process of inquiry and refinement of a theory of change. Any of the phases depicted below can occur multiple times, depending on options, preferences, and timeline for the evaluation.
How the Systems Thinking Principles Work
Here’s how the systems thinking principles work throughout this iterative evaluation process:
- Taking stock of clients’ and other stakeholders’ understanding about the project, evaluation, and their roles uncovers and includes diverse perspectives, broadening evaluation thinking beyond that of “experts” and challenging our mental models, as Peter Senge advises us to do. Doing this also supports culturally responsive and gender-responsive evaluation designs.
- Working collaboratively to understand the theory of change for the purposes of evaluation enables transparent identification of project and evaluation system boundaries, as well as the consequences of setting these boundaries.
- Drafting and revising a theory of change for the evaluation provides an avenue to consider the dynamics influencing the system that flag a need to adapt evaluation processes. Understanding dynamics affecting project variables can reveal hidden structures, feedback loops, or influences that cause a seemingly intractable or inexplicable problem or provide momentum toward a solution.
- Projects take place in complex, changing environments, with often unpredictable interrelationships—influencers, results, time lags, and feedback loops. Evaluators who understand these interrelationships can adeptly plan and time evaluation activities and design ethical evaluations, mindful of consequences related to these factors
Incorporating theory of change design or refinement in your evaluation processes gets you all this! And, clients appreciate the relationship-building these exchanges foster. They also often find this development process helps them improve the project design—all in advance of the evaluation or (if working developmentally with the client), alongside it. This collaborative, appreciative, systems-oriented approach to evaluation is a win-win for evaluators, stakeholders, and clients.
Feeling inspired? Share your stories of systems thinking in evaluation in the comments below, by tweeting @EnCompass_World, or by connecting with EnCompass team members at Evaluation 2018 and the SETIG session on November 2.
Photo by Ryoji Iwata on Unsplash