Evaluating with No Theory or Logic Model? No Problem!

Written by: Kimberly Norris

Typically, when you’re asked to conduct an evaluation, you start by looking at background documents to orient yourself to the project. Ideally, those materials include a logic model, results chain diagram, or theory of change illustration—key pieces you need to plan that baseline, midline, summative, or process evaluation you’re about to conduct.

But what if you don’t have this verbal or pictorial map? What’s your way forward? Before you despair, consider this: the lack of a theory or logic model can actually be an opportunity—one that allows you to conduct a better evaluation than ever!

At EnCompass, our collaborative, appreciative approach to evaluation is grounded in systems thinking. Using Appreciative Inquiry with clients and stakeholders as part of the evaluation, we co-learn about the logic behind the project along the way, applying the key principles of systems thinking in evaluation at every step. This is a topic near and dear to my heart, as an evaluator and as a member of the AEA Topical Interest Group on the subject.

The Principles of Systems Thinking in Evaluation

These principles—of interrelationshipsperspectivesboundaries, and dynamics—support the development and implementation of a sound and relevant systems-based evaluation that involves, and is transparent to, clients and stakeholders.

Following the four systems thinking principles, our project and evaluation logic development is iterative, involving the evaluation team and our clients and stakeholders in a process of inquiry and refinement of a theory of change. Any of the phases depicted below can occur multiple times, depending on options, preferences, and timeline for the evaluation.

Image shows iterative, four-phase cycle: Inquire (use Appreciative Inquiry to learn more about the project from our client); Learn (interview other stakeholders to learn how the project works); Draft (Use the information to draft or modify a theory of change for the evaluation); Refine (Share with our client and stakeholders for feedback and revisions)

How the Systems Thinking Principles Work

Here’s how the systems thinking principles work throughout this iterative evaluation process:

  • Taking stock of clients’ and other stakeholders’ understanding about the project, evaluation, and their roles uncovers and includes diverse perspectives, broadening evaluation thinking beyond that of “experts” and challenging our mental models, as Peter Senge advises us to do. Doing this also supports culturally responsive and gender-responsive evaluation designs.
  • Working collaboratively to understand the theory of change for the purposes of evaluation enables transparent identification of project and evaluation system boundaries, as well as the consequences of setting these boundaries.
  • Drafting and revising a theory of change for the evaluation provides an avenue to consider the dynamics influencing the system that flag a need to adapt evaluation processes. Understanding dynamics affecting project variables can reveal hidden structures, feedback loops, or influences that cause a seemingly intractable or inexplicable problem or provide momentum toward a solution.
  • Projects take place in complex, changing environments, with often unpredictable interrelationships—influencers, results, time lags, and feedback loops. Evaluators who understand these interrelationships can adeptly plan and time evaluation activities and design ethical evaluations, mindful of consequences related to these factors

Incorporating theory of change design or refinement in your evaluation processes gets you all this! And, clients appreciate the relationship-building these exchanges foster. They also often find this development process helps them improve the project design—all in advance of the evaluation or (if working developmentally with the client), alongside it. This collaborative, appreciative, systems-oriented approach to evaluation is a win-win for evaluators, stakeholders, and clients.

Feeling inspired? Share your stories of systems thinking in evaluation in the comments below, by tweeting @EnCompass_World, or by connecting with EnCompass team members at Evaluation 2018 and the SETIG session on November 2.

Photo by Ryoji Iwata on Unsplash

Post Tags :

Kimberly Norris

Senior MEL Specialist

Kimberly Norris, Senior MEL Specialist, is highly skilled in evaluation, learning, and facilitated strategic planning. Kim has more than 20 years’ experience collaboratively leading evaluations and building evaluation and learning capacity in Africa, Europe, Asia, and Latin America and the Caribbean. Dr. Norris applies whole systems evaluation and adaptive management skills to lead deeply integrated mixed methods baseline assessments, developmental evaluations, and performance and impact evaluations. She brings organizational capacity strengthening skills that support more collaboratively oriented monitoring, evaluation, and learning systems. She has applied evidence-informed and values-based Appreciative Inquiry and participatory approaches to evaluations across sectors, including agriculture, food security, child nutrition, education, environmental adaptation and ecological monitoring, governance and corruption, and human rights. This includes multiple recent evaluations addressing anti-human trafficking efforts in Asia and Latin America and the Caribbean. Her work has served USDA, USAID, the Department of State, and the Department of Health and Human Services, as well as the MacArthur Foundation and Margaret A Cargill Philanthropies. She joined EnCompass after working for several years at the U.S. Peace Corps headquarters and in eSwatini. Dr. Norris is the current Co-chair of the American Evaluation Association Systems in Evaluation Topical Interest Group (SETIG). In 2007, she received a commendation on the floor of the U.S. House of Representatives for directing food and nutrition education programs across Vermont and bolstering Farm to School programs nationally. In 2014, she received an award from then U.S. Peace Corps Director, Carrie Hessler-Radelet, for collaboratively designing an evaluation and learning system across Peace Corps offices. Kim holds a PhD in Wildlife and Fisheries Sciences from Texas A&M University. She received her Master's degree in Education and Bachelor’s degree in Natural Resources Conservation and Management from the University of Florida.

Leave a Reply

Skip to content