Some General Concerns About GiveWell
In my previous post, I explained why I was surprised by GiveWell’s recent recommendation of GiveDirectly. In this post, I’ll take the opportunity to explain some other general concerns I have with GiveWell. As before, I’ll note that these just represent my view, rather than those of GWWC as a whole, and that such criticism should be taken as a sign of respect for an organisation which I generally think very highly of. We are making these suggestions in part because we draw heavily on GiveWell’s own research and want it to be as compelling as possible.
I’ll also note that many of these concerns can also be applied to GWWC itself. This list therefore also represents a list of improvements we would like to make to our own research and publications in future.
Some specific points
Long run effects
Different countries vary dramatically in their rate of economic growth. In 2011, India’s economy grew by 7.8%, whereas Namibia’s economy grew by just 3.8%. In the long-run, insofar as your donation generates some jump in the productivity of the labour force, compounding effects will cause it to snowball into substantially more income if used in India than in Namibia. On this basis one might argue that donations to India constitute a better ‘investment’ than donations to Namibia. On the other hand, one might reason that, because we should expect people to be richer in India in the future, they will get less benefit from any extra income in the future. The economic trajectory of the country that you try to help could potentially be a ‘crucial consideration’ concerning which charity you should give to, but it hasn’t yet been addressed by GiveWell.
This is not even considering the magnitude and persistence of an intervention’s impact on economic productivity. This is another crucial consideration which we understand GiveWell has looked into and found very difficult to answer.
External Evaluation
As far as I know, GiveWell haven’t commissioned a thorough external evaluation of their recommendations. (GiveWell have done some external evaluations in the past, but mainly in 2010 or 2011, and mainly done by volunteer undergraduates or recent graduates.) This surprises me. Whereas businesses have a natural feedback mechanism, namely profit or loss, research often doesn’t, hence the need for peer-review within academia. This concern, when it comes to charity-evaluation, is even greater. If GiveWell’s analysis and recommendations had major flaws, or were systematically biased in some way, it would be challenging for outsiders to work this out without a thorough independent evaluation. Fortunately, GiveWell has the resources to, for example, employ two top development economists to each do an independent review of their recommendations and the supporting research. This would make their recommendations more robust at a reasonable cost.
Billionaires and Room for More Funding
To put it simply: If GiveWell’s top recommended charities are so good, why hasn’t their room for more funding been used up?
GiveWell’s view is that there are relatively few good giving opportunities, because the best ones tend to be snapped up quite quickly. But they have provided very public information on their top-recommended causes, some of which have been on the list for years. And we know that major philanthropic foundations (such as the Gates Foundation, and CIFF) are aware of their research. So it’s surprising, in their view, that the room for more funding for their top-recommended organisations wasn’t used up rather more rapidly. This either suggests that their model of how philanthropy works is wrong, or that their recommendations aren’t believed by the big players in the ‘philanthropic market’.
Even if we accept that their model of an efficient market in philanthropy is inaccurate, and so could attribute the failure of big foundations to fund their recommended charities to irrationality or different values, there remains cause for concern. GiveWell functions in an advisory capacity to GoodVentures, which has very similar aims and attitudes to those of GiveWell. GoodVentures' endowment could easily use up the room for more funding at GiveWell’s top charities. So it raises the question why GoodVentures doesn’t donate to GiveWell’s top charities: even if not exhausting their room for more funding, at least using up a considerable proportion of it. The most plausible explanation in my mind is that GiveWell thinks that GoodVentures can better spend its money at a later date after further research. In which case, it’s hard to see why individual donors shouldn’t save and donate later, or even try to give to GoodVentures. Perhaps that’s wrong, but it’s worthy of discussion.
What does GiveWell mean by ‘good’
GiveWell claim that they want to maximise good done per dollar invested, but they haven’t explained what they think ‘good’ consists in and why. Do future people count? If so, how much, and how far into the future? Do they have a discount rate? How much priority should they give to the worst off? Is an infant death worse, or better, than the death of a 20 year old? How do they weigh improvements in health against extensions in health? What do they think is the relationship between economic growth and wellbeing?
These are just of theoretical interest — varying any of the parameters could lead to different charitable recommendations. For example, when they discuss AMF’s cost-effectiveness, they focus on under-5 mortality. But they don’t discuss the fact that most people think that an infant death is not as great a loss as a young adult death; an intuitive view that has good theoretical grounds. (This is something that will likely be taken into account in the next generation of the DALY metric, in the third edition of the Disease Control Priorities Project.) Depending on how, exactly, one thinks one should weigh saving infant lives against saving adult lives, this issue could make AMF several times less cost-effective relative to SCI than it is currently judged to be. This one methodological issue could therefore dramatically alter their recommendations.
To their credit, GiveWell has moved towards helping people to tailor their choice to their values, by comparing the cost of bed nets delivered, children dewormed, and cash delivered. This can be done using spreadsheets and explanations provided in some recent blog posts. In its current form this requires more work than most donors would be willing to engage in. Making this easier, and guiding people through discovering the implications of their moral values would be worthwhile improvements. This kind of tailored advice is only possible for the minority of charities GiveWell has looked into in great detail, which means it is still important to be explicit about the value judgements that generated that shortlist.
Once they allow the moral view or ‘judgment calls’ of the donor to be an input, providing a #1 #2 #3 ranking of charities becomes misleading. For example, I wrote previously that I was surprised by their recommendation of GiveDirectly. Most of my surprise would have been alleviated if, rather than ranking them #2, they gave them a conditional recommendation: “If you have such-and-such moral view, then GiveDirectly is an excellent option.” Indeed, there are many reasonable views based on which giving to GiveDirectly could be the best option. Perhaps you do not want to be paternalistic towards the poor, even if being paternalistic would do more to improve their wellbeing in the long run. Or perhaps the reason you give is because you think you should redistribute your wealth as a matter of justice, and it’s not your right to say how that money is spent. In either case, it’s very useful to know that one has a good option to put wealth directly into the hands of the poor. But, if that’s the motivation for recommending GiveDirectly, ranking them above SCI and below AMF disguises the fact that it is being judged on a different set of criteria.
Interested?
To find out more or show your support, please sign up for our monthly updates:
Subscribe by RSS
Comments
First off, taking India is a bad example. India is starting to reject aid from different countries and´focusing on a pool of cash from some big givers. Fundamentally and in my opinion (!) giving, when push comes to shove, should go to those who need it the most. The article mentions India's growth as being a "return of investment" but then again, taking growth as an indicator of anything other than the richness of some seems like a poor indicator of anything at all. There is a huge amount of extreme poverty in India that simply won't improve for many many years, quite possibly several generations. The economic trajectory of a country should have next to no impact on your giving since the social situation is what, at least I hope, most are trying to address.
I can't do anything but agree with the need for external assessments of activities in an organisation, not least those purporting to be experts in the area of giving. Whenever a think-tank like GW speaks badly of an organisation it could have a massive impact on their ability to function. They need to be incredibly careful and back up their work, and ensure that no outside interests are influencing their results. I don't know them well enough to comment.
everyone's definition of good is different, and for good reason. We all have passions. Particular injustices we are trying to rectify, particular catastrophic situations we are trying to fix. That's the bread and butter of any organisation, and most would try their hardest to define what this is without alienating givers. NO ORG is ever going to say "the death of a baby is more tragic than that of a twenty year-old". It's fundraising suicide, and understandably so. GW cannot do what the author is asking in the way he demands it. They can set priorities, maybe, but that would defeat the point of the org since what they're trying to do is advise givers on the best "bang for the buck" in giving, the holy grail for any recipient.
Smart giving is hard, but no matter how much data and information you have I would imagine a gut-feeling is there that will ultimately decide on who gets what. If you're approaching giving in an emotionally detached way I will have difficulty understanding you. There are many good organisations out there, you as a giver must have a priority, GW shouldn't be deciding for you.
There are many other possible reasons why more people don't contribute to GiveWell's top-ranked charities. For example, I don't give to them for the simple reason that they're not tax-deductible in my country. I don't make charitable contributions in order to get a tax deduction, but tax-deductibility factors into my decision on which charities to support because I can use my tax refund to increase my giving. If I give only to non-deductible charities I can't afford to give as much as I can if I can claim a deduction. None of the charities ranked highest by GiveWell (or Giving What We Can) are listed as tax-deductible charities in my country (Canada).
"tax-deductibility factors into my decision on which charities to support because I can use my tax refund to increase my giving"
Personally I would give to givewell's top charity even if they weren't tax deductible because a tax deduction (for me, in the US) only makes giving about 30% cheaper and I think that the AMF is more than 30% better than the next best charity out there.
@ Jeff: It's not so much that a tax deduction makes giving 30% cheaper, it's that a tax deduction allows me to give 30% more. I actually do give some money to AMF even though it's not deductible (the biggest single donations I've made in my life have all been non-deductible, in fact), but there are thousands of charities whose effectiveness hasn't yet been evaluated; I'm not sure we can yet say with confidence that the top-ranked GiveWell or GWWC charities are more effective than the next-best charities out there.
Just for the record, AMF is tax-deductible in both the U.S. and Canada: http://www.givewell.org/about/FAQ/research#Arethetopcharitiestaxdeductib...
Alexander, thank you for pointing that out! Most of the US-based charities I've given to in the past were not registered in Canada so I assumed the same was true for AMF; this will have a big impact on my giving for next year. I didn't give them much this year but will make them my primary charity going forward. Thanks for that!
Goodventures have just announced grants to the three Givewell recommended charities - see the link below
http://www.facebook.com/notes/good-ventures/giving-season-grants-to-give...
Will, thanks very much for this post. We're always happy to get feedback on what we could be doing better.
A few responses:
On external evaluation. Will's point is a good one, and this is a topic we've been discussing internally for quite a while; it may deserve a blog post of its own at some point. In a nutshell, though, the short story is that
(a) Getting a qualified individual to do a meaningful external evaluation is a very challenging undertaking. There are questions around how to find the right person (many of the most "qualified" people tend to have their own affiliations that may bias their assessments, and are also the hardest people to get to spend time on this sort of work); how to structure the evaluation (it isn't practical for someone to go over all of our work with a fine-toothed comb; the higher-status the person, the more of an issues this becomes); how to compensate people for their time (we don't want to create a pro-GiveWell bias by paying, but not paying further limits how much time we can ask); etc. Given the time investment these sorts of activities require on our part, we're hesitant to go forward with one until we feel confident that we are working with the right person in the right way and that the research they're evaluating will be representative of our work for some time to come.
(b) We believe we have been getting more engagement and critique of our work from the outside, in a more piecemeal way. This is a very good thing.
(c) Because of (a) and (b), we currently have made external evaluation a low enough priority that it hasn't happened in a while. We hope to return to this priority at some point and agree that GiveWell's research would be more credible, all else equal, if we did so; however, the capacity we'd use to work on this area can also be used on many other tasks that also improve our research.
On why the "room for more funding" hasn't been used up. This is a very good question that certainly merits its own blog post at some point. In brief, we've optimized our research to date for individual funders. Most major donors don't believe that they can maximize their good accomplished by supporting our top charities, and we largely agree with them, which is part of the reason we're working on broadening our research beyond the criteria we've used to date. Note that Good Ventures supports our top charities partly to signal support of GiveWell and its criteria (more at this Good Ventures post).
On our definition of "good." We haven't spelled out exactly what we mean by "good"; in fact we don't have full consensus within GiveWell and we probably wouldn't even be able to fully, precisely articulate our own personal definitions of "good." However, what we have spelled out is our criteria and process for forming our shortlist of recommended charities. (These rely less on a particular conception of "good" than on a decision to emphasize the role of empirical evidence of effectiveness, among other factors.) Within our shortlist, as you note, we have made it possible for donors to plug their own assumptions into our sheets, and encouraged them to do so. I agree that doing so is not as easy as it could be; our focus is on providing the highest-quality research we can such that sufficiently motivated individuals can engage with whatever aspect of it they're curious about, and we generally put fairly little effort (though not zero) into improving digestibility and ease of use. (Note that this prioritization has been debated heavily throughout GiveWell's history, including at board meetings whose audio and materials are available online and in annual review/plan documents.)
Why we rank charities. Some donors seek to engage heavily with our research; some seek to spot-check limited aspects of it; and some are simply looking for a quick bottom-line recommendation. We present charity rankings and recommended allocations with the latter group in mind. By doing so, we make it possible for someone to decide, "I trust GiveWell enough, and my time is limited enough, that I'll just give as GiveWell suggests"; we believe this leads to more giving to our top charities. People who don't take this attitude generally do not take our rankings at face value, and instead make their own judgment calls.
On long-run effects of giving in different countries. This is one of many conceptually relevant issues that we haven't addressed. Our goal is to use our limited person-hours with maximal efficiency from the perspective of finding the best possible giving opportunities; at the point where the "India vs. Namibia" question is crucial to a giving decision, we will address it, but so far we have found it more efficient to form our shortlist of recommended charities by applying our criteria (which, again, emphasize considerations such as strength of evidence rather than a particular conception of the good), and when deciding between shortlisted charities we haven't found this issue to be a major consideration.
A final note. The question of "where should I give?" is a huge question, and GiveWell is a small organization. Therefore, we don't think it's surprising that there is a very large number of issues we haven't addressed and services we haven't provided; we wouldn't characterize this situation as "concerning."
What we would consider "concerning" would be (a) suboptimal use of our resources - cases in which we not only would be better off (all else equal) for having done X, but in which we would be better off sacrificing some other activity Y in order to do X.
That said, we appreciate all feedback on what we would ideally be doing that we aren't currently doing, and feel that many of the points raised in this post are worthy of consideration.
On the point of external evaluation: I do not know if this sort of organisation exists in the US yet, but here in the UK we have http://www.probonoeconomics.com/. They exist precisely to match up economists with charities on a pro bono basis, so that charities can be externally evaluated on their effectiveness. It seems like their interests are pretty well aligned with organisations like GiveWell and GWWC, as they're pretty focussed on bang-for-buck, so collaborating with them might avoid some of the problems you raise in (a) above, Holden. If GW can't take up that opportunity since PBE don't operate in the US at the moment, then GWWC definitely should look into it. These are leading economists who are into charity effectiveness, after all -- what a great resource!
There has to be an element of compassion and intuition in any charity giving and this leaves it up to the individual as to what they regard as their particlar moral imperatives. We can't just be ruled by what is 'efficient'. However, I think that we all want to see the most effective use of our donated funds and tools such as those provided through Give Well are useful as guidance in these decisions. I feel Will's point on external ecvaluaton is well worth further consideration.
Hi Holden,
Thanks so much for this; I appreciate it. I especially liked your responses on external evaluation, ranking charities, and long-run effects. I'll just mention a couple of other things.
On the final note:
I'm not sure if it's just a linguistic tic of mine (and spending time in the US has made me very aware of how much linguistic discrepancy there is between the US and the UK), but I'd think of "I have some concerns about GiveWell's recommendations" as much weaker in force than "I find GiveWell's recommendations concerning". I said the former, but you read me as saying the latter. The former I'd translate as "I have some reservations about GiveWell's recommendations"; the latter I'd translate as "I think there are major problems with GiveWell's recommendations". At any rate, it was the former I meant, and I apologise if it read like the latter. I think, perhaps, that the title was (unintentionally) misleading - the title "Some concerns about GiveWell's recommendations" would have been better.
In particular, I could agree that GW has spent its time optimally, while still having many concerns about the recommendations. E.g. The long-run economic effects argument could be a game-changer, and it's as yet unstudied. So that provides reason, for example, to save and donate later, when there's better evidence, rather than donate now. So these issues are still relevant even if GW has spent its time optimally (which, of course, it's impossible for any organisation to do).
On room for more funding of top charities:
If GW's recommendations are tailored for individuals, but foundations think that they can do more good per dollar by funding larger projects, wouldn't it make more sense for individuals to give to GoodVentures? This probably would be more cost than help for $25 donations, but many GW donors give upwards of $10k, where the financial benefit would exceed the transaction costs.
On the definition of "good":
You said that, though you don't have a conception of good, you do have your criteria, one of which is empirical evidence of effectiveness. But in order to judge "effectiveness", we've got to have a conception of 'good'. (Burning coal is a very effective way of releasing CO2 into the atmosphere, but a coal-burning charity wouldn't be a good one, even if its desired aim was to increase atmospheric CO2 levels).
Now, many (most?) issues that philosophers argue about - e.g. utilitarianism versus egalitarianism - seem to me to be of relatively little consequence, in practical terms, for GW's recommendations. But other issues could be game changers: How much do future individuals count? How much do non-human animals count? I've gotten the sense that GW gives non-human animals low weight. But I'm really not sure what it's view on future individuals is. And that would greatly affect how I understand GW's recommendations.
For both the definition of good issue, and the long-run economic effects issue, perhaps they are unlikely to affect recommendations of SCI vs AMF vs GD (though I think some, like infant deaths, are). But they would plausibly affect which new cause areas and charities GW chooses to look into - and whether we should think that there are other charities out there that are much better than AMF. So these issues are certainly relevant.
Hi Will,
Regardless of the phrasing around it, "concern" seems to me to be stronger than "room for improvement" or "reservations," with "concern" implying that something has gone wrong. But I understand better what you meant now, and appreciate the clarification.
I do think there's a case to be made for saving to give later, and for donating to Good Ventures (or to GiveWell with the money restricted for regranting). Ultimately, I've made my own giving decision with the points made at our post on giving now vs. later in mind, which means I've aimed to get money out the door (i.e., in the hands of an organization other than GiveWell or Good Ventures, which I can then observe and learn from) with some regularity. I think some of the donors who use our research are thinking similarly; others may simply be skeptical of the typical large philanthropist's approach, and looking for something more direct.
On our definition of "good" - I think our page on impact analysis (linked from our criteria guide) is relatively clear about which outcomes fit into our definition of "good," though it lacks a highly conceptual framework. I agree with you that as we broaden our research, philosophical issues that have been of minor importance may become much more important, and accordingly we will be writing about such issues as they become more relevant to the recommendations we're making.
Not sure if GiveDirectly's donations are enough for this to matter, but isn't there some concern that large amounts of money going to one place (in GiveDirectly's case, Kenya) could contribute to Dutch Disease:
http://en.wikipedia.org/wiki/Dutch_disease
?
'NO ORG is ever going to say "the death of a baby is more tragic than that of a twenty year-old". '
Of course! DCPII clearly holds, in its methodology, that twenty years olds are worth more than babies.
Hi Matt, it's an interesting point and one I've thought about from time to time. One thing to keep in mind is that an appreciating exchange rate is bad for exporters, but good for importers (which includes many businesses that use intermediate inputs). It can also be offset by a central bank lowering interest rates, or a government reducing spending. Perhaps for these reasons it is not clear that Dutch disease is actually bad for a country's development, though it will have distributional effects: http://www.voxeu.org/article/why-worry-about-real-exchange-rates-missing...
Hi Matt, it's an interesting point and one I've thought about from time to time. One thing to keep in mind is that an appreciating exchange rate is bad for exporters, but good for importers (which includes many businesses that use intermediate inputs). It can also be offset by a central bank lowering interest rates, or a government reducing spending. Perhaps for these reasons it is not clear that Dutch disease is actually bad for a country's development, though it will have distributional effects: http://www.voxeu.org/article/why-worry-about-real-exchange-rates-missing...
Add new comment