Thursday, September 02, 2010

If you can flip a coin, can you be an expert?

With support from the Rockefeller Foundation, we recently ran an experiment at GlobalGiving that had shocking results.  We asked people in four communities in Kenya to tell stories* about the development issues most important to them.  This was their answer:

In parallel, we asked experts (both local and foreign) with experience in those communities to predict what they thought the story would be about.  Here is the shocker:  Only 1 of the 65 experts and implementers correctly predicted the most common theme.

Personally, I found it surprising that 42% of the stories were about social relations, but on the other hand, I don't work in those communities so how should I know?  But I would expect the experts to know, wouldn't you?  Alas, the experts predicted only three of the top six concerns of the community.  In other words, they did about as well as flipping a coin.** 

The old style of aid is for experts to study the situation and decide what people need.  It is tempting to say that we should simply reverse this and let the people decide.  Exciting new technologies will enable beneficiaries to have a far greater voice in the coming years, and that is long overdue.  But the best system will likely  provide a balance of the two.  It will create an ongoing, iterative conversation between beneficiaries and experts about what is needed, what works and what doesn't, and what that implies about priorities and initiatives in subsequent rounds.

* The story telling part of this experiment was done with the help of Cognitive Edge and Irene Gujit. They were introduced to us by the Rockefeller Foundation, which is leading some of the most innovative work in the field I have seen in years.

** There were a total of twelve themes to choose from.


April said...

very cool Dennis.
one concern I have is that not all issues are equally susceptible to social policy actions by gov't (with or without donor support). Social relations and self-esteem stand out to me to be to be issues that individuals and local actions may be much better suited address than public official or aid agency activities. So I think I would rather ask people what things they'd like outsiders (or some other workd) help with. Or gov't help with or something. Then compare that ranking to that of expert predictions.

Security comes out at the top in many countries list of concerns also in the Gallup World Poll. I'm always torn on this one. I find it hard to imagine aid agencies being effective on this. But sad that we do so little if it is seen as so important.

Dennis Whittle said...

April, agreed that not all issues are equally susceptible to outside aid. The nature of this exercise was just to understand the things that people care most about, so we did not ask about outside help. The experts - the majority of whom were local by the way! - still missed the boat hugely...

Tom Watson said...

Dennis - fascinating. How was "social relations" defined for the purposes of the study? What did the term include?

Dennis Whittle said...

Tom - send me your email and I will send you the form itself.

Random Surfer said...

The Kenya study corroborates documented experience in the field and that experience in the field, in turn, corroborates your conclusion about not running too far afield and ceding all decision making to the beneficiaries of aid. Calling for healthy, respectful collaborations might be the moral of this story.

Consider the fascinating iterative process that led to a self-sustaining* hospital in rural Ecuador. Had it been more collaborative -- more respectfully collaborative -- from the beginning, it might not have had to have been so iterative.

In this 2009 article, "The Rural Hospital in Ecuador," published in the journal Health Affairs by the director of Andean Health & Development** ( ), we see that listening to the local people in combination with outside expertise can be an excellent model for the efficient delivery of aid:

As Dr. David Gaus writes on page 7 of the article, "I have learned that the community had the answer. They even told me so, though I ignored it at first. But they left it up to me to figure out the 'how.' It was a bottom-up process, just as the founders of primary care suggested."

*/ The goal of a self-sustaining hospital was important, as this is different from a missionary hospital and other efforts that collapse when the missionaries depart or a central figure exits the scene. The operating costs of the hospital are not dependent on charity or grants or aid that can be fickle and dry up overnight. (Capital improvements, however, are always welcome.)

**/ Andean Health & Development is currently part of the September 2010 Global Giving Open Challenge:

Dennis Whittle said...

Dear Random-

That is a nice paper.

Random Surfer said...

I wonder if you regard this as another example of collaboration...though, to be sure, it's not so much a collaboration in terms of expertise as it is a collaboration of power-sharing:

Dennis Whittle said...

Random -

The Newark educations initiative funded by Mark Z will definitely bears watching. The mayor seems to be highly respected, the bipartisanship and power sharing by the governor is to be lauded. Let's see how it goes and what lessons are learned.