navigating unfamiliar territory
February 17, 2021 Lynne Franco No Comments

Navigating Unfamiliar Territory to Get the Most from Evaluation

Written by: Lynne Franco

This piece was originally published as part of the John D. and Catherine T. MacArthur Foundation’s Learning as We Go evaluation Perspectives series, with co-authors Anne Gienapp, Lindsay Hanson, and Hallie Preskill. The blog introduces a case story on The MacArthur Foundation’s Approach to Evaluation and Learning

In October 2018, MacArthur convened a meeting of its six evaluation and learning partners (ELPs), one for each of its Big Bets and Enduring Commitments. While all the ELPs were experienced evaluators with foundation clients, the meeting represented a new and unique opportunity. It was the first time any of us had met with a client alongside our client’s other consultants for the explicit purpose of sharing the successes and ongoing tensions of our respective engagements—something that does not often occur in our field.

However, with the encouragement from the Office of Evaluation, we recognized that the benefits of being open about our experiences outweighed the potential downsides or uneasiness of speaking about our challenges and revealing what we sometimes safeguard as exclusive content and tactics. First, it offered us an opportunity to draw on each other’s work to gain a deeper and more comprehensive understanding of how to best support MacArthur’s approach to strategy, evaluation, and learning. Moreover, it lent legitimacy to our roles by enabling us to model the behavior and thinking that we are collectively working toward within MacArthur and across the philanthropic sector. As evaluators, we should lean into the discomfort and perceived risk of making our work visible and open to scrutiny, so we can reflect on our decisions, invite feedback, and ultimately improve upon what we are doing.

In fact, the energy we all derived from this exchange spurred our decision to establish a community of practice among the ELPs. We meet regularly, without MacArthur’s participation, to highlight challenges and successes about our work with, swap ideas, and learn from each other’s experiences.

One of the early tasks set by this community of practice was to find venues to more broadly share our experience of working as an ELP within MacArthur’s approach to evaluation and learning. The community of practice considered a range of options, including writing a peer-reviewed article for journal publication, presenting at field-wide conferences, and developing a series of blog posts. In the end, we settled on this case story. It is based on interviews with MacArthur staff (leadership, program, and evaluation) and the ELP community of practice; our review of extant literature on the philanthropic sector; and written materials about MacArthur’s approach to evaluation and strategy.

This case story is not intended to present MacArthur’s approach as the way to do evaluation; there are many ways to evaluate programming, each one being more or less appropriate for any given context. Instead, we are sharing what we learned to inform efforts within philanthropy and beyond to link evaluation with strategy, drive informed decision making, and promote transparency and accountability in the social sector.

We have seen how MacArthur’s approach to evaluation is rooted in values, especially in the value of diversity of thought that truly challenges assumptions and invites criticism of their strategies. That invitation for dynamic feedback, however, brings inherent tensions. Being a “critical friend,” the role ELPs are supposed to fill, requires a delicate balance. We must have deep knowledge of MacArthur’s complex strategies while leading evaluations that test the logic and coherence of those strategies.

To manage this balance, our role requires bringing a high level of rigor to the methodologies we employ so we can be confident in the findings we present. There is a personal factor that is essential to the work, too. To drive uptake of our evaluation findings, there must be strong working relationships that are grounded in mutual respect and trust between the ELPs, Evaluation Officers, and the program teams. The attention that goes into building those relationships is critical to the utility of evaluation, ensuring that program teams can translate learnings into evidence-based decisions about their work. And, this mutual respect and trust requires ongoing cultivation and commitment from us as ELPs, the program teams, and the Evaluation Officers serving a critical intermediary role.

Through the process of interviews and writing this case story, our ELP community of practice gained insights into the dynamics of MacArthur’s approach to evaluation. We confirmed the importance of relationship building and management as a critical pillar of ELPs’ engagements with MacArthur and that building a strong support network of evaluators has improved the value of our respective work. We hope that this take-away and the others that are highlighted in the case story can be helpful to foundations and others that, like MacArthur, continue to grapple and experiment with ways to get the most out of evaluation and learning.

Lynne Franco

Special Advisor

Dr. Lynne Miller Franco, Special Advisor, is an expert in evaluation and learning, with 35 years of experience in international development, covering over 30 countries in Africa, the Middle East, Eastern Europe, and Latin America. She has held long-term positions in Benin, Malawi, and Mali. Dr. Franco brings her strong evaluation and facilitation skills to both single evaluations and to EnCompass’ role as evaluation partner for organizations seeking to develop and implement ongoing monitoring, evaluation and learning (MEL) efforts. She has led evaluation projects for USAID, PEPFAR, WHO, the MacArthur Foundation, the Margaret A. Cargill Foundation, the Bill & Melinda Gates Foundation, and Save the Children among others. Prior to joining EnCompass, Dr. Franco served as Director of Research and Senior Quality Improvement Advisor for University Research Co., LLC. In 2011, Dr. Franco was among the first recipients of USAID’s Excellence in Evaluation Award for her work on the evaluation “Results of Collaborative Improvement: Effects on Health Outcomes and Compliance with Evidence-based Standards in 27 Applications in 12 Countries.” She teaches workshops on data analysis and theory of change at the American Evaluation Association, and social network analysis through The Evaluators' Institute (TEI). Dr. Franco holds a ScD in International Health Systems and an MHS in Health Planning from Johns Hopkins University School of Hygiene and Public Health. She is fluent in English, French, and Italian.

Leave a Reply

Skip to content