August 6th was the kick off date for a conference put together by leading private foundations in Brazil to address the role of evaluation in the social programs they fund. The Conference, entitled Evaluation of Investment in the Social Private Sector: an Organizational Strategy, is part of a larger effort to embed evaluation in civil society in Brazil. The conference provided a forum to engage peer-to-peer exchanges and involve evaluation specialists in relevant discussions, and to build partnerships with new actors such as universities, the Brazilian evaluation sector, and others, to develop of the evaluation field in Brazil.
The event was sponsored by several important foundations including Fundação Itaú Social (a major bank in Brazil); Fundação Roberto Marinho (established by a Brazilian media company); Fundação Maria Cecilia Souto Vidigal (leading work in early childhood development); Fundação Santillana (supporting ibero-american education and culture); MOVE and COMEA (two private evaluation consulting firms); and the Group of Institutes, Foundations, and Enterprises (GIFE)—the Brazilian council of foundations.
The meeting began with a gathering in Sao Paulo with members of GIFE discussing the role of evaluation in Brazil’s foundations’ social programs. We then paid a visit to the “Abraço do Tigre”, a community program in Rio de Janeiro to explore a cutting edge approach to social program evaluation. The capstone of the three days was a one-day conference on evaluation with 300 attendees from the nonprofit sector in Rio de Janeiro.
Strategic and Operational Challenges
In the GIFE meeting, representatives of Brazilian foundations raised many issues including:
- Brazilian foundations are bombarded by evaluation terms, and as engage in the conversation, we are becoming uncertain of the meaning of evaluation words.
- There are significant weaknesses in monitoring, and this limits our ability to conduct evaluations.
- It is hard to come up with measures of social capital.
- It is challenging to explain evaluation in our foundations.
- What is the right mix of efforts to evaluate implementation and impact?
- What alternatives to quasi-experimental designs are there for impact evaluations?
Using Evaluation to Communicate Respect in Communities
Once in Rio, we visited the Instituto Abraço do Tigre, a community initiative that runs after-school programs for children and vocational training programs for adults using, among other strategies, a methodology developed and promoted by the Roberto Marinho Foundation. An interesting learning moment was when the program’s director Cecilia (or would it be Simone?) told us about an online evaluation survey they are doing to receive feedback from program participants. We questioned the wisdom of spending extremely scarce resources in doing evaluation, and asked about it. We expected them to answer that it was required by the funder, but instead, the director said, “Because people feel respected. It matters for them to know that their opinion matters.”
Feeling the Power of Evaluation in Brazil
Conference participants took part in all three workshops offered by Thomas Cook, Patricia Rogers and me. My workshop was entitled, “Evaluating Our Way to Smarter Organizations,” and I learned a great deal from engaged, vocal participants.
As I reflect, on the 3-day discussions, I am struck by the power of Brazil’s social sector to lead through evaluation. The foundations that hosted us are backed by strong, intelligent entrepreneurs who manage empires. They have a high demand for quality and low tolerance for ambiguity and delays. The conference itself was managed to perfection by Fundação Itaú Social with excellent organization and state-of-the art equipment. The Roberto Marino Foundation mobilized the media: TV Globo and Globo News covered the evaluation event in the evening news, and Canal Futura— an educational TV channel— had special reportsduring their programming that week. Maria Cecilia Foundation has placed evaluation close to its leadership team informing the CEO and supporting the Foundation’s accountability to its Board. The power of this formidable group of foundations linked through GIFE to influence how evaluation is embedded in the social sector was palpable!
EvalPartners and the global evaluation community should pause and reflect on what we can learn from Brazil’s emerging model of building evaluation capacity and evaluation use. And the Brazilian Government’s Secretary for Evaluation and Information Management (SAGI) with the leadership of Paulo Jannuzzi, would find it rewarding to talk about evaluation with these important foundation partners. Both will benefit immensely, and the international community will be watching and learning with them.