Written by: Jonathan Jones, Director, MEL

At EnCompass, our Monitoring, Evaluation, and Research team uses data gathered through rigorous data collection approaches and appreciative learning techniques to help ensure our clients are drawing on evidence to inform strategic decisions. Over the years, we have led research and evaluation efforts, large and small, for numerous clients around the world, including USAID (Headquarters, Regional Missions, and Country Offices), U.S. Department of State, Democracy Fund, UN Women, World Health Organization, MacArthur Foundation, Bill & Melinda Gates Foundation, and many more.

Often in evaluation, there is immense focus on data collection without fully considering the crucial analysis and synthesis part of the process. All evaluators have probably, at one time or other, found themselves with a mountain of data and struggling to figure out how to get to a strong evaluation report. Over the years, we have developed an approach that efficiently and rigorously integrates and interprets data. We call it the Data Analysis, Integration, and Synthesis (DAIS) process. This participatory approach allows us to integrate complex data streams, often collected and analyzed by multiple evaluators. The DAIS process is interactive and iterative. During a DAIS workshop, we put analyzed data visually in front of the evaluation team (using cards or sticky notes), and we work together to integrate it into initial findings. We have found that this process can overcome the challenge of moving from analysis to report writing and can help lead to a well-structured report with strong findings, a clear narrative reflected in the conclusions, and recommendations grounded in data.

The idea of a DAIS started in 2013, with a virtual session among the EnCompass evaluation team engaged in an evaluation of the USAID Program Cycle to discern and articulate clear findings from our large mixed methods dataset. By 2014, we had developed a somewhat standardized process, which we have continued to tweak and adapt. Over the years, we have used the DAIS approach for in-person, virtual, and blended virtual/in-person sessions. We have used it for large and small evaluations and evaluations with data derived from a single methodology or complex mixed method data collection efforts.

To go from data analysis to a meaningful evaluation report that tells the right story, we divide the process into three phases: preparation, the DAIS session itself, and preparing the report process. In the simplest terms:

1. Preparation: Evaluation team members develop data analysis summaries. Data analysis summaries consist of short emerging theme statements derived from the data and a summary of the data used to develop the statement. Often, we have dozens of emerging theme statements to consider during a DAIS workshop.
2. DAIS workshop: Emerging theme statements are placed on cards. We discuss them and look for affinity to develop initial findings. A finding is typically supported by several emerging theme statements, often from different data streams. During the DAIS, we also brainstorm conclusions and recommendations based on the initial findings, ensuring that all conclusions and recommendations are supported by evidence. In some cases, we co-create recommendations with the key evaluation audience.
3. Preparing the report process: We determine the narrative arc for the report, develop a report outline based on the DAIS workshop outputs, make writing assignments, and establish a timeline to complete the report.

The timeline for the full DAIS process depends on the complexity of the evaluation, the number of different methods, the amount of data collected, and the vision of the final deliverable(s). We have found that this participatory process provides our evaluation teams and, ultimately, our clients with confidence in the findings, conclusions, and recommendations that arise.

Benefits of the DAIS process include:

• Empowerment: Participatory data analysis can give the evaluation team a sense of ownership over the data and the analysis, leading to an increased confidence in the findings. Thus, the team feels more empowered to recommend action (if appropriate) based on the evidence.
• Rigor: By pausing to have everyone on the evaluation team deeply engage with the data during a DAIS workshop, a more nuanced understanding of the evidence emerges, and findings are grounded directly in the evidence gathered. The DAIS also ensures that the analysis and findings are grounded in the experiences of those involved in data collection.
• Transparency: By involving the evaluation team in the DAIS process, the team can be confident that findings, conclusions, and recommendations are grounded in the evidence and presented appropriately.
• Co-learning: Participatory data analysis can facilitate a co-learning process among evaluation team members. Each can learn from the other’s expertise and perspectives.

Because we have used the DAIS so successfully, we now share the process with others through our EnCompass Learning Center course. To learn more about the course or to join our next program offering, please email learningcenter@encompassworld.com. We have also trained on this approach several times in skill-building workshops at the American Evaluation Association annual conference.

To learn more about our past and ongoing work at EnCompass, please visit the project page on our website.