Utilization-Focused Evaluation: A Network Analysis of Organizations Working to Prevent Stillbirth
October 01, 2016
What is utilization-focused evaluation and how does EnCompass use this approach?
Utilization-focused evaluation, developed by evaluation consultant Michael Quinn Patton, is an approach to evaluation based on the principle that an evaluation should be judged on its usefulness to its intended user. In his seminal book, Utilization-Focused Evaluation (2008), he argues that evaluations should be conducted in a way that engages the intended user throughout the evaluation to enhance the utilization of findings once the evaluation is complete. This includes facilitating the use of the evaluation by presenting actionable, evidence-based recommendations, as well as developing a dissemination strategy for the evaluation that promotes use of the findings among all end users.
EnCompass takes utilization-focused evaluation very seriously and is continuously exploring new and innovative ways to engage our clients, and make the findings of our evaluations easily accessible and useful for all intended users. This is evident in the design phase when, in collaboration with our clients, we formulate research questions and data collection tools that best capture data our clients can immediately use for decision making and planning; in the reporting phase when we prepare clear and concise reports that have actionable, realistic recommendations; and the report dissemination phase when we organize webinars and live events to bring stakeholders to discuss, reflect, and plan around the evaluation data.
Organizational Network Analysis
The most recent example of this commitment was in our work with Saving Newborn Lives (SNL). Funded by the Bill & Melinda Gates Foundation, SNL seeks to reduce global neonatal mortality by working in partnership to develop packages of effective, evidence-based newborn care interventions and to implement these innovations at scale. As an evaluation partner to SNL, we conducted an organizational network analysis (ONA) on the community of organizations working on stillbirth issues. ONA is a technique that allows us to better understand social structures by mapping interactions and relationships among organizations and stakeholders, visually showing the strength and direction of those relationships and what role each stakeholder plays within a community.
EnCompass first worked with SNL to conduct an ONA of the community of organizations working to improve newborn health in 2014. Our report on the ONA enabled the community to identify appropriate actions to enhance future sustainability of newborn health and survival, and specific organizations that could play key roles in sustaining the momentum over time.
Based on this success, SNL asked EnCompass to conduct an ONA of the network of organizations working to prevent stillbirths. This network is made up of a diverse group of stakeholders who identify their work across the continuum of maternal, newborn, and child health. As part of the ONA, EnCompass interviewed multilateral organizations, professional associations, academic/research organizations, donors, nongovernmental organizations, partnerships, parent groups, and the private sector. EnCompass used the ONA to investigate the dynamics of this community, provide evidence of the community’s strengths and weaknesses, and what partnerships could be intensified across the community to increase attention and action toward ending preventable stillbirths.
Ensuring Utilization Through Effective Dissemination
We then developed a suite of client-focused products, and conducted a dissemination event to share and discuss the results.
Initially, some of the results of the ONA were featured in two papers of the Lancet's Ending Preventable Stillbirth Series. The papers used the ONA data to support key points regarding intentional leadership and the way forward for the community working to prevent stillbirths. However, space limitations prevented a full discussion of other important findings from the ONA.
To ensure that organizations working tirelessly on stillbirths could use all findings of the ONA, Encompass conducted additional analysis and captured the data in a SlideDoc, which is a visual tool that bridges the gap between a long-form document and a presentation. The benefit of a SlideDoc is that it uses graphics alongside qualitative text to explain key messages. The SlideDoc was intended to deepen an understanding among the community of organizations working on stillbirth issues and further assist stakeholders to operationalize the next steps charted in a “Call to Action,” which was presented in the Lancet papers. EnCompass sent the SlideDoc to all stakeholders who participated in the ONA and their networks.
Birds' eye view of the Stillbirth ONA SlideDoc
Reaching All Stakeholders with the Findings in Written Products
To ensure that all the implicated stakeholders working on stillbirths engaged with and used the results of the ONA, EnCompass collaborated with the Partnership for Maternal, Newborn and Child Heath (PMNCH) to host a 90-minute webinar. The webinar was attended by 26 people representing leading multilateral, donor, and nonprofit organizations working on stillbirth issues, and included presentations by Joy Lawn of the London School of Hygiene and Tropical Medicine and Besty McCallon of the White Ribbon Alliance.
After the presentations, participants held a vibrant 45-minute discussion on how to use the ONA data to push the community forward. Highlights from the discussion included creating an action plan that linked the existing initiatives and funding in maternal, newborn, and child health care with stillbirth work; the importance of appointing a person within one of the key multilateral agencies such as the World Health Organization (WHO) or the United Nations Children’s Fund (UNICEF) who would be specifically focused on stillbirths; building a knowledge hub that includes stillbirth resources and agreed-upon definitions for the community; and creating a catalogue of parent groups for stillbirth advocacy throughout the world. The call ended with a proposal from PMNCH to put stillbirths on the next agenda of the PMNCH board meeting, thus creating a platform for moving this conversation into an action plan for the stillbirth community.
The way the Lancet papers, the SlideDoc, and webinar reached and engaged a broad cross section of the entire stillbirth prevention community demonstrated how important it is for organizations to not only conduct quality research and evaluations, but to be purposeful and strategic about how to share the findings so that they will be fully utilized in the future.