The Three “Pause Points” of Evaluation Management

Tessie at NIH
February 15, 2018

Evaluation management is an essential competency for evaluators, evaluation team leaders, and commissioners of evaluation, often embedded in other evaluation courses. I’ve been teaching evaluation management as part of courses on developing a theory of change, reframing evaluation through Appreciative Inquiry, gender-responsive evaluation, and other topics. Today, I invite us to think about the intersection of good management and good evaluation, because both are essential to the success of any evaluation. 

I can hear your immediate question: What does evaluation success mean? Good question! It’s one I ask participants in evaluation management classes—most recently in the Evaluation Management and Oversight class I teach at The Evaluators’ Institute and the Managing Decentralized Evaluations class I teach for the World Food Programme’s Evaluation Learning Programme. To synthesize the participants' responses, successful evaluation is relevant to important issues, planned well, conducted with high technical quality by evaluators who retain independence while engaging all stakeholders respectfully, and managed in a collaborative way with an eye to the use of results for learning, decision making, and program improvement.

In other words, successful evaluation is not only about getting the methodology right, but also about negotiating agreements and working with others to plan and organize every step of the evaluation, including its use. Thus, evaluation is “an intervention and a process”1 that requires facilitation and management.

In this post, I share one of my secrets: the three pause points of good evaluation management.

We never begin at the beginning

Evaluators always come in during the middle of the story, and we need to understand what happened before we arrived, the direction things are going, and the different visions of the future that key evaluation participants may have, and then situate our evaluation within that context. Commissioners of evaluation arrive first at the scene, and we must look to them to manage a productive process for coming up with terms of reference that embed values and make the purpose of the evaluation clear. This is helpful to the evaluation team. Still, when the evaluators get hired, they need to revisit the terms of reference, understand what is meant, and make clear agreements around the evaluation purpose, questions, process, timeline, and other factors. Managing this “beginning”—or these multiple beginnings—is an important pause point and worth everyone’s attention.

What did we miss?

A good time to think about this question is early in data collection. That’s when the rubber hits the road and you get useful feedback about what you got right and what you missed. Don’t wait 2 weeks to learn that no one responded to your survey. Check in during the first week. Check in with those conducting data collection in site visits, and with their hosts. Early check-ins help you adjust your questions and scope to include something important that you overlooked, and give you a chance to speak with your client about it in case other adjustments are necessary. 

What does it all mean?

The third pause point worth your attention is data analysis. Many evaluators rush this part. There were inevitable delays in data collection, the client pushed for “just three more interviews with really important people,” and here you are: behind schedule and tempted to just start writing. Stop. You put in all this effort, and your data deserve respect. Plus, you promised triangulation in your proposal. So, take the time to see what your data tell you, compare findings from different data sources and methods, step back, and reflect. What does it all mean? What findings do you trust to bring forward into the evaluation report?

Of course, every step in the evaluation is important. But my advice is to prioritize start-up, early points during data collection, and data analysis. And, if you are convinced that evaluation management competencies are a priority for you, we can continue this conversation in my course on project management and oversight for evaluators. I look forward to hearing about your own pause points. 

[1] Catsambas, T. T. (2016). Facilitating Evaluation to Lead Meaningful Change. In R. S. Fierro, A. Schwartz, & D. H. Smart (eds.), Evaluation and Facilitation: New Directions for Evaluation 149: 13–23.

Image property of Tessie Tzavaras Catsambas. All rights reserved.

Comments

Mhlonishwa

As a person who works in the evaluation space, I can relate to this. I agree on the importance of giving data so much respect. We put so much efforr and sometimes rush ot analyse data before we even take inti cognisance the factors that may lead to inaccurate results. This has been a good read, thanks Tessie

Bhabatosh Nath

<p>The Key statement you mentioned, "We never begin at the beginning". This is so true for the evaluators! A learning for us. I think during 'managing the beginnings', in addition to internalize the Terms of Reference we also need to know about the 'readiness' of the implementing organizations and the donor whether they are also reday to face the evaluation!</p>

Add new comment

About the Author

As co-founder of EnCompass, Tessie Tzavaras Catsambas has led the development of responsible...

All content © 2012-2018 EnCompass LLC

Contact Us | Sitemap | Privacy Policy