Counting Mosquito Nets

If you think it’s simple to measure a nonprofit’s effectiveness, think again.

I’ve written before about the failings of Charity Navigator and other rating systems, which resort to vastly oversimplified measurements to evaluate charities – often in misleading or even detrimental ways. But can’t some truly in-depth analyses work better? Sure, but even the most concerted and thoughtful efforts to evaluate nonprofits are riddled with ambiguity and challenges.

Let’s take the example of GiveWell, an organization created by a group of veteran hedge fund managers as a way of advising donors on the best nonprofits to support. The GiveWell folks attempt to apply the same intellectual rigor to evaluating nonprofits that they’ve used in examining for-profit investment opportunities. GiveWell scours the globe for worthy charities and comes up with a handful (truly – only three or four) that they recommend. And the story of their involvement with one of those organizations is instructive.

A recent New York Times piece explains how a few years ago GiveWell gave its endorsement to Against Malaria Foundation, a London-based nonprofit. Against Malaria Foundation has a simple formula for fighting the disease. Every five dollars provided by donors allows Against Malaria Foundation to buy an insecticidal mosquito net for people in Africa to use as protection at night. Those five dollars can save a person’s life – and they’ve bought and distributed nearly seven million nets, so the reach is considerable. It’s a compelling cause, and, because Against Malaria Foundation tracks the results by studying malaria rates in the areas where they work, there’s also a high degree of accountability.

After GiveWell’s endorsement, a flood of donations arrived – and then Against Malaria Foundation had a problem. They now had buckets of money for buying mosquito nets, but their trusted distribution network of local organizations that put the nets to good use couldn’t absorb the additional supply. Against Malaria Foundation had the option of working to distribute the nets through less tested means, such as governmental agencies, but many of these agencies were rife with corruption, and there was a good chance that the nets would be stolen and sold for profit on the black market. So Against Malaria Foundation piled up cash instead of putting all the money to immediate use.

At which point GiveWell pulled its endorsement from Against Malaria Foundation. You see, GiveWell now considered that Against Malaria Foundation was inefficient in the way it applied the charitable dollars. Or, more exactly, how it didn’t apply them. (We’ll leave aside the irony that GiveWell was itself responsible for the flood of cash that Against Malaria Foundation had trouble digesting.)

I have four reactions to this story.

  • The first is that sometimes nonprofits are damned if they do and damned if they don’t. Had Against Malaria Foundation worked with the less trusted governmental agencies and had their nets stolen by profiteers, they would have been criticized for wasting their donors’ money and feeding corruption. Instead, they’re accused of hoarding the funds and not putting them to charitable use.
  • The second is that even simple organizations are hard to evaluate. Against Malaria Foundation is a much less complex organization than most nonprofits. Against Malaria Foundation is using a single strategy – distributing insecticidal mosquito nets – to combat a single disease. Though the mission is very important – literally life and death – and though no work in developing countries is easy, the operation is less complex, and its outcomes simpler to measure, than, say, a social service organization working to help families struggling with multiple problems, such as poverty, disease, substance abuse, violence, substandard housing, lack of opportunity, and failing schools. I’m sure the simplicity and clarity of its mission and methods explain why Against Malaria Foundation rose to the top of the GiveWell list: it’s an organization with a mission that is easy to see, to understand, and to measure. And yet, even with Against Malaria Foundation, it’s tough to measure with absolute certainty the effectiveness and efficiency of the charitable gift.
  • The third is that, as I’ve written before, going to scale is a challenge. Most nonprofits are not very deep administratively. The more efficient they are at getting funds to work for the mission, the more challenging it may be for them to double or triple their impact in a short period of time.
  • And finally, this story reveals a class-based double standard in how we evaluate nonprofits. No one is asking America’s elite cultural and educational institutions to prove their effectiveness. Donors don’t check with GiveWell or Charity Navigator before sending millions to Princeton or the Metropolitan Opera. Funders don’t hassle the Boston Museum of Fine Arts to prove the effectiveness of its exhibits or the efficiency of its use of donated dollars. But for the social service organizations that are trying to ameliorate conditions for vulnerable people in a badly broken world, the expectations can be set unreasonably high.

So should we as donors work as hard as we can to be sure the charitable organizations we support are effective? Of course. Should nonprofits look for intellectually rigorous ways of testing their impact? Absolutely. But should we expect a third-party evaluator to have all the answers and to reduce complexity to a simple number or rating? No, we shouldn’t. And we should be suspicious of those who claim to do so.

Copyright Alan Cantor 2014. All rights reserved.

Share this: Facebooktwittergoogle_plusredditlinkedintumblrmailFacebooktwittergoogle_plusredditlinkedintumblrmail

2 thoughts on “Counting Mosquito Nets”

  1. The bellweather S & P 500 aveage has outperformed Hedge Funds as an investment class in 10 of the past 11 years. Hard to argue with…or say more than that. Emperor’s clothes?

  2. Interesting example of unintended consequences. Reminds me of a story I heard that, instead of protecting cork trees, synthetic corks have made cork trees less profitable so they are now more likely to be replaced by real estate development in some cases.

    Your post also reminds me of the debate between active and passive investors, e.g. Fidelity versus Vanguard. Instead of looking for the most worthy charity, why not invest in a portfolio of them and try to mimic the market. Now if GiveWell would only come up with a Dow Jones index for non-profits…

Leave a Reply

Your email address will not be published. Required fields are marked *