We are at an important moment in evaluation. All kinds of people are engaged in evaluation, and sometimes they do not even call it evaluation. FastCompany named Nate Silver as 2013 #1 Most Creative Person in Business. In this interview, Silver—followed because of his excellent predictive algorithms based on large data sets—said:
“There’s a lot of ways to take a lot of data, mangle what you’re doing with it, not ask good questions, and get yourself in trouble.”
Evaluation’s commitment to mixed methods is precisely because numbers without the genius of context and interpretation do not have useful meaning; it also explains why it is so easy to lie with numbers by manipulating what they mean. So as @BetterEval and others so eloquently advocate, we should start with clarifying the purpose of evaluation, and get the questions right. “People blame the data,” Silver says in the article, “when they should be asking better questions.”
Big Data is exceedingly useful. For example, without important data such as @MEASUREDHS, we would not have a reliable foundation for better health policy decisions. The only point here is that, without systems thinking, questioning assumptions, methods that invite meaningful questions and local knowledge in interpretation, Big Data can result in a high level of confidence in the wrong conclusions.