Stay up to date with Giving What We Can's blog. Featuring the latest news about our community, how to donate effectively, charity evaluation, community events and more!

Blog

Giving What We Can's blog, featuring the latest news from our community.

footerNavigation

About us

Contact us

Our team

Our values

Our members

Our events

Transparency

Our mistakes

Our impact

Donate to Giving What We Can

Careers

About

Our best charities

Take a giving pledge

Get involved

Make a bequest

Videos, books, podcasts

FAQs

Sitemap

Learn more

Press kit

Media coverage

Case studies: People who pledge to give

Media

Footer Navigation

topNavigation

Recommended charities

High-impact causes

Why give to funds

How much to donate

Why donate publicly

Recommendations

Comparing charities

Choosing a cause

Inclusion criteria

Trusted evaluators

Our approach

Videos, books, podcasts and essays

Hesitations about charitable giving

How rich am I?

Can money buy happiness?

Resources

Research

Team

The Pledge

Trial Pledge

Further Pledge

Bequests

Why pledge

Pledges

Newsletter

Podcast

Videos

Sharing our ideas

Events

Giving games

Charity elections

Pledge drive

Fundraisers

Community

Donate now

Best charities

Manage my donations

Tax deductibility

Donate

My pledge

My donations

My income

My impact

Pledge

My giving

Top Navigation

Giving What We Can

A CONVERSATION WITH HOLDEN KARNOFSKY, CO-FOUNDER OF GIVEWELL-PART II

Recently, Giving What We Can staff sat down with Holden Karnofsky, the co-founder of GiveWell. The goal was to try and critically explore GiveWell's methodology, research approach, and recommendations. The 

Giving What We Can's aim was to act as lawyers, not judges, to help GiveWell smooth over any potential flaws and become more accurate. In playing the 'devil's advocate', Giving What We Can might come across as more negative and skeptical than is their actual view of GiveWell.

Ultimately, these conversations are valuable because GiveWell relies on people to challenge them but, often enough, there are not enough feedback loops in the world of philanthropy. Giving What We Can's recommendations will also be improved, since we rely heavily on GiveWell's research. Likewise, it should benefit the thousands of people who donate based on GiveWell's recommendations and the hundreds of people who have taken Giving What We Can's 

You can find the first part of the interview 

Rob: As GiveWell is moving into different kinds of interventions (migration, catastrophic risk, etc.), do you worry that you might bring over too much skepticism because of the work you’ve done in the past on completely different areas?

Holden: There is a risk on both sides and we’re aware of it. And I think the bigger risk is on the other side -- not bringing in enough skepticism. For example, global health has a really good track record -- it’s quantifiable, measurable, and accountable. The track record of people trying to do things that rely on specific views on how very low probability events will affect the very far future has a really bad track record.

Donors are afraid that GiveWell is good at understanding interventions with lots of evidence and accountability, but that when GiveWell moves into evaluating politics or asteroid impacts, one can’t know nearly as much, and GiveWell will underestimate their own uncertainty because they’re still thinking about global health.

I’m excited about politics and existential risk reduction. I probably could give to more of these “blue sky” things when we have better recommendations, but I’m concerned that I’m just being crazy and overconfident.

Rob: Do you think there is a 50% or higher chance that GiveWell will find something twice as high impact as malaria within the next, say, five years?

Holden: I do, though again I’m worried about being crazy. I don’t think we can make a really quantified or irrefutable case, so we expect people to use their own judgement calls. GiveWell will move to just putting up information about different areas and explaining what Holden thinks and where he gives his money, but we’ll tell people they’ll have to decide for themselves because it will come down to very intangible judgements.

I personally think this highly measurable stuff is just not as high impact as this “blue sky” stuff, as long as the “blue sky” stuff is done right. I assume it’s easily to do badly and therefore gets a bad reputation, but it could be done well and be at least an order of magnitude better than global health interventions. But that’s just my guess, and I don’t know if Elie Hassenfeld would agree with me.

Rob: My impression is that Elie probably doesn’t. But I basically share that outlook. Therefore, I’m personally holding on to my money rather than donating it, because I expect that soon we’ll find a much more high impact place to put it. I’d be curious to know if you are holding your money as well, and whether you think others should do the same, at least in the short-term.

Holden: I’m not holding on to my money. I’m donating at a regular pace. We wrote about this in “

”. I think giving itself is a mechanism for learning.

Let’s think about this two ways. First, you’re a big donor with your own foundation and your own staff. At that point, giving regularly allows you to continue to gain access to the things you are giving to and continually re-examine your hypotheses in a way you wouldn’t be able to do otherwise. But I agree that foundations shouldn’t be giving out nearly as much now as they ought to when they finally understand enough.

A small donor won’t have this level of access. But they can give to GiveWell’s top charities because it allows GiveWell to gain that level of access. See our post that “

Giving to GiveWell’s Recommended Charities Helps GiveWell

”, because people will engage with GiveWell much more if GiveWell can demonstrate more money moved, and this will greatly improve our ability to find higher impact interventions.

Rob: I agree that’s true, but my guess is that consideration is outweighed by the chance of us finding something many times better. Though there’s value in learning, there’s probably more value in holding on to your money. Aren’t we kind of biased toward giving people advice that they should give more, because it’s better for our organisations.

Holden: There are issues about psychology when making recommendations about giving -- for example, people who save might not always follow through. I’m not concerned about this for myself, but I can see it being an issue for some people. GiveWell takes a strategy of always saying what we think and not adjusting our advice for this psychology.

I disagree with your reasoning because if the whole effective altruist community were thinking your way, GiveWell’s input would be zero, and we couldn’t find opportunities better than the Against Malaria Foundation. And most of our donors are considering this and won’t just continue to give regardless.

Rob: I definitely think that some people should be giving. But given that you’re moving potentially $10M a year, I don’t think that adding an extra $100K is worth the cost in terms of not being able to later fully fund even better interventions.

Holden: I think it is worth the cost. Of course, it’s not quite linear -- the first few million is definitely more important. But when you’re recruiting or talking to a charity, the bigger the amount of money GiveWell moves, the better.

And there’s still a concern that if you think this and restrict your money, others will follow, and GiveWell’s money moved will go down.

Paul: One response to these issues of psychology and having a credible impact down the line so charities will engage with you is to encourage people to donate to donor advised funds with a governance structure that charities can understand. If GiveWell moved a few million dollars less directly to top charities, but had that few million dollars in a donor advised fund for “blue sky” projects, it’s not clear to me that this would decrease your ability to evaluate these “blue sky” projects.

Holden: I think that’s a reasonable observation, but it gets into questions of psychology. If GiveWell could replace all the donations moved to top charities with donations to a donor advised fund, and emphasized the size of our assets rather than the size of our money moved, we would do this. But if we could only do this for half our donations, things get more complicated.

Also, many people don’t understand what donor advised funds are. It would need to belong to an organization in order to be useful and this is difficult to explain to people and it’s not very credible. Keeping it simple is better.

Paul: How has the $1M for GiveWell Labs worked in gaining access to the sort of charities you talk to?

Holden: The $1M didn’t matter much at all because we started working closely with Good Ventures. Good Ventures is accomplishing what you’re saying because they give to show they are serious, but they’re giving a small fraction of what they’ll eventually be giving. And this is credible because people know about the size of their endowment.

As long as our relationship with Good Ventures stays strong, we’ll be able to gain access. But being able to say that the “effective altruism” movement is a real movement and being able to say that the movement is recruiting lots of people and moving millions of dollars from individuals is very helpful to GiveWell.

Peter Hurford

Should We Tell People That Giving Makes Them Happier?

It seems strange to even ask. Most people would point to the fact that they’re altruistic and want to 

. Others are concerned with inequality and justice. Another group points to the concept of “paying it forward” or repaying a debt to society. Other explanations cite various religious or social reasons.

Not too many people cite the fact that giving makes them happier. Even if people agree this is true, I don’t often hear it as people’s main reason. Instead, it’s more like a beneficial side effect. In fact, it seems pretty odd to me to hear someone boldly proclaim that they give only because it makes them happier, even if it might be true.

, should we be promoting that publicly and loudly? Perhaps it might make a great opportunity to tap into groups who wouldn’t consider giving otherwise or have misconceptions that giving would make them miserable? I’m a bit worried about how it might affect people’s incentives.

In this essay, I follow the evidence provided in the Harvard Business School working paper “Feeling Good About Giving: The Benefits (and Costs) of Self-Interested Charitable Behavior” by Lalin Anik, Lara B. Aknin, Michael I. Norton, and Elizabeth W. Dunn. Overall, in light of potential incentive effects, I think caution and further investigation is warranted when promoting the happiness side of giving.

Giving What We Can has published its own review of research on happiness and giving and find a pretty strong connection. And it’s true -- lots of evidence confirms the connection and even indicates that it’s a causal relationship rather than a misleading correlation. In fact, it goes in both directions -- giving makes people happier and happier people are more likely to give[1].

Neurological studies of people found that people experienced pleasure when they saw money go to charity, even when it wasn’t their own, but experienced even more pleasure when they gave to charity directly[2], a conclusion that has been backed up with revealed preference tests in the lab[3, 4].

This connection has also been backed up in numerous experimental studies. Asking people to commit random acts of kindness can significantly increase self-reported levels of happiness compared to a control group[5]. Further research found that the amount people spent on gifts for others and donations to charity correlates with their self-reported happiness, while the amount they spent on bills, expenses, and gifts for themselves did not[6]. Additionally, people given money and randomly assigned to spend money on others were happier than those randomly assigned to spend the same amount of money on themselves[7].

People generally believe that spending on themselves will make them much happier than spending on others[6], which, given that this isn’t the case, means there is plenty of room for changing people’s minds. However, any social scientist or avid reader of Freakonomics knows that altering incentives can create unintended effects. So is there a potential harm in getting people to do more giving via advertising self-interested motive?

The classic example is that of the childcare center that had problems with parents who were late to pick up their children. They reasoned that if they charged fines, parents would stop being late, because they would have an economic incentive not to. They found instead, however, that introducing a fine actually created even more tardiness[8], presumably because what once was seen as rude and bad faith now could be made up for with a small economic cost. More surprisingly, the amount of lateness did not return to pre-fine levels even after the owners stopped the policy[9].

Other studies have found similar effects. A study of 3-5 year old nursery students who all initially seemed intrinsically interested in various activities were randomly put into three groups. One group made a pre-arranged deal to do a one of the activities in which they seemed interested in exchange for a reward, another group was surprised with a reward after doing the activity in question, and the third group was not rewarded at all. Those who were given an award upfront ended up significantly less intrinsically interested in the task than the other groups after the study was finished[10]. A similar study found that students who were interested in solving puzzles stopped solving those puzzles after a period ended where they were paid to solve puzzles[11].

In general, money and reminders of money tend to make people less pro-social[12]. This has also been found to some degree specifically in the world of charity. In a randomized field experiment, donors were encouraged to donate to disaster relief in the US and were randomly either enticed with an offer of donation matching or not. The study found that while people donated more often with the promise of donation matching, their contributions after the donation matching dropped below the control group, ending with a negative net effect overall[13].

Another study found that when gifts were sent out to donors, larger gifts resulted in a larger response rate of returned donations, but yielded a smaller average donation[14], though I suppose this could just be because more people who usually would give nothing were giving a small amount, bringing the average down. More importantly, this study found no net decrease in future donations after gifts were no longer sent out; instead, donations returned to their normal levels[14].

And certainly it’s worth noting some times when appeals to self-interest are successful. I couldn’t find any studies where this was the case. However, there is one anecdotal example: as Nick Cooney points out in “Self-Interest Can Make the World a Better Place -- For Animals, At Least”, reduction in people eating factory farmed meat is coming almost entirely from people motivated not by concern for animal cruelty, but concern for their own health. Could advocating self-interested donations be the same as advocating health-motivated vegetarianism?

It’s not very good to just let things be unclear if they don’t have to be, and I think we can resolve this issue with more scientific study. For example, one could randomly select one group to receive information about giving and happiness, another group to receive other standard arguments for giving, and a control group to receive no arguments or information about giving at all, and track their donation habits in a longitudinal study. This study would have it’s complications for sure, but could help see if information about giving and happiness backfires or not.

Or perhaps one could perform a field experiment. You could set up a booth asking people to donate to your cause and randomly include information about giving and happiness or not in your pitch and see how this affects immediate and long-term contributions. Doing this would have added advantages of being much quicker to run and not leading to people donating only because they think they’re being observed.

[1]: Anik, Lalin, Lara B. Aknin, Michael I. Norton, Elizabeth W. Dunn. 2009. 

“Feeling Good about Giving: The Benefits (and Costs) of Self-Interested Charitable Behavior”

. Harvard Business School Working Paper 10-012. [2]: Harbaugh, William T. 2007. 

“Neural Responses to Taxation and Voluntary Giving Reveal Motives for Charitable Donations.”

 Science 316: 1622-1625. [3]: Andreoni, James, William T. Harbaugh, and Lise Vesterlund. 2007. 

. New Palgrave Dictionary of Economics. [4]: Mayr, Ulrich, William T. Harbaugh , and Dharol Tankersley. 2008. 

“Neuroeconomics of Charitable Giving and Philanthropy”

. In Glimcher, Paul W., Ernest Fehr, Colin Camerer, and Russel Alan Poldrack (eds.) 2009. 

Neuroeconomics: Decision Making and the Brain

. Academic Press: London. [5]: Lyubomirsky, Sonja, Kennon M. Sheldon, and David Schkade. 2005. 

“Pursuing Happiness: The Architecture of Sustainable Change.”

 Review of General Psychology 9 (2): 111–131. [6]: Akin, Lara B., et. al. 2010. 

“Pro-social Spending And Well-Being: Cross-Cultural Evidence for a Psychological Universal.”

 National Bureau of Economic Research Working Paper #16415. [7]: Dunn, Elizabeth W., Lara B. Aknin, and Michael I. Norton. 2008. 

“Spending Money on Others Promotes Happiness.”

 Science 319: 1687-1688. [8]: Gneezy, Uri and Aldo Rustichini. 2000a. 

 Journal of Legal Studies 29: 1-18. [9]: Gneezy, Uri and Aldo Rustichini. 2000b. 

 Quarterly Journal of Economics 115: 791-810. [10]: Lepper, Mark R., David Greene, and Richard E. Nisbett. 1973. 

“Undermining Children's Intrinsic Interest with Extrinsic Reward: A Test of the ‘Overjustification’ Hypothesis.”

 Journal of Personality and Social Psychology 28(1): 129-137. [11]: Deci, Edward L. 1971. 

“Effects of Externally Mediated Rewards on Intrinsic Motivation.”

 Journal of Personality and Social Psychology 18(1): 105-115. [12]: Vohs, Kathleen D., Nicole L. Mead, and Miranda R. Goode. 2006. 

“The Psychological Consequences of Money”

. Science 17 (314): 1154-1156. [13]: Meier, Stephan. 2007. 

“Do Subsidies Increase Charitable Giving in the Long Run? Matching Donations in a Field Experiment”

. Federal Reserve Bank of Boston Working Paper #06-18. [14]: Falk, Armin. 2005. 

A Conversation with Holden Karnofsky, co-founder of GiveWell-Part I

A Conversation with Holden Karnofsky, co-founder of GiveWell

Recently, Giving What We Can staff sat down with Holden Karnofsky, the co-founder of GiveWell. The goal was to try and critically explore GiveWell's methodology, research approach, and recommendations. The 

Ultimately, these conversations are valuable because GiveWell relies on people to challenge them but, often enough, there are not enough feedback loops in the world of philanthropy. Giving What We Can's recommendations will also be improved, since we rely heavily on GiveWell's research. Likewise, it should benefit the thousands of people who donate based on GiveWell's recommendations and the hundreds of people who have taken Giving What We Can's 

Rob Wiblin: It seems that GiveWell has a pessimistic view of charities, assuming the worst of them unless there is strong evidence otherwise. Is that an accurate representation of GiveWell?

Holden Karnofsky: No, and this is where a lot of confusion comes from. We should distinguish between a "pessimistic attitude" and a "regressive attitude". A pessimist attitude would say that charities are not doing good unless otherwise proven. A regressive attitude is that when you see a claim that someone is an outlier having massively more positive impact (or negative impact) than the average charity, they are probably closer to the average than they claim to be. This "regressive" attitude, not the "pessimistic" one, is the attitude GiveWell has.

We at GiveWell are looking for the best opportunities. Most charities are doing good work. But when we run across something that looks really good, the most likely explanation is a combination of exaggeration, error in estimation, or missing key pieces of information. However, as you gain more evidence and better knowledge in the field, this becomes much less of an issue.

In most areas of life you assume that things are "too good to be true" unless otherwise proven. You don't just jump at any "get rich quick" scheme -- when you see something you don't understand, you don't take immediate action. Rather, you are unsure and investigate it further. There might be reasons why this analogy isn't perfect for charity, but the general principle applies to many parts of life.

And throughout our history at GiveWell, this approach has been confirmed as something that works - it pushes our biases in the right direction relative to our initial estimate. It's not about being skeptical that charities are doing good.

Rob: Maybe a key disagreement between GiveWell and Giving What We Can is that GiveWell thinks we should shift our estimates downward in light of low evidence more than Giving What We Can thinks we should. What empirical evidence do you have to back up this claim? And how do you quantify how much we should regress?

Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased)

 and I published a spreadsheet there that gives a sense of how GiveWell thinks regression should be done. There also was a follow up called 

Maximizing Cost-Effectiveness via Critical Inquiry

The basic idea is that when you have a "back of the envelope calculation" (where the 95% confidence interval includes the estimate that the charity is no more cost-effective than average), it's hard for your estimate to get to or above the two standard deviations from average range, no matter what your explicit estimate was -- even if you initially estimated that your charity had an impact that was thousands of standards of deviations better than average.

Owen Cotton-Barratt: GiveWell seems to imply that if average is within the 95% confidence interval, the estimate will always turn out to be only two standard deviations above average at best. Is this always the case?

Holden: This is not always the case. It's only for estimates that are sloppy and involve a lot of guesswork. I wouldn't want to make this claim about things we understand much better, and that's why GiveWell places such a high priority on understanding things better.

Consider the concept of "broad market efficiency". Imagine you're looking at a field of research you don't understand very well and lots of people understand it better than you. If you find something really great, it's more likely the experts are ignoring it for a good reason.

But if you're the expert, can see things from all angles, and you know all the relevant evidence, and you've checked everything out, it's a completely different story. And some really good work has been done, so GiveWell isn't saying that if you think something's a good idea you're wrong. And this is all consistent with what the math says.

GiveWell has never once gone looking to "knock down" an estimate. Our ideal situation is that we find massively excellent cost-effective interventions that are far better than anything else you could get. But we have found in our history that we've revised our estimates downwards. For example, very early in our history, we heard that the Fred Hollows Foundation was curing blindness for $50 and we hoped it was true. But when we looked in depth into the numbers, we found that the number was not well grounded.

When VillageReach was our top charity, we estimated them at under $1000 per life saved. But with further investigation to learn more, we kept finding that estimate going up, and ultimately we had to say that we weren't even sure if VillageReach was having an impact. And that was really tough for GiveWell because we were going against our first "big bet".

Thirdly, there is deworming. We were suspicious about the figure that deworming saves a year of healthy life (DALY) for every $3.41 spent. This number never looked right based on what we knew. When we looked into the spreadsheets for the number, we found five spreadsheet errors that involved overestimates and underestimates in cost-effectiveness. When those errors were all corrected, the "$3.41" estimate was off by a factor of 100. 

And we published all this work for the public

Fourthly, a major foundation told us that Mothers2Mothers was having a huge impact, serving 20% of all eligible mothers in Sub-Saharan Africa. But when we looked into this, we found the charts didn't make any sense. Furthermore, if their claim were true, we should be able to see trends in country-level figures, and we did not. The country-level data flat out contradicted this claim, and 

I could give more examples, but this is a constant pattern: we find something, we get very excited, we look into it, and it doesn't come out looking as good as we thought.

New Updates to Education Research

Interested in how we can improve education in the world? Giving What We Can has been looking into more opportunities to make a difference in this area and have found some intriguing conclusions. Thanks to research from Anna Maarova, Giving What We Can recently updated its page on education interventions. Here's all about what changed and why.

Giving What We Can has considered a broader range of interventions and has made significant use of the latest research from the 

 (J-PAL), an organization centered at the Economics Department at the Massachusetts Institute of Technology. J-PAL does randomized controlled trials to test interventions to helping with extreme poverty and they have a lot of academic credibility in economic development.

We looked specifically for organizations that were using J-PAL’s recommended approaches which are shown on our 

. However, our search only turned up one organization, Pratham, which we already were aware of. 

 back in March 2012 and concluded they were a promising organization, but ultimately not as promising as the Against Malaria Foundation or their other recommendations. That being said, Pratham looks like the best bet for someone who wants to focus specifically on developing world education. They are responding to evidence about the best ways of improving education in a very poor country, and collecting data on what is working for them.

Our new page now offers a more balanced appraisal of the potential benefits and costs of various interventions in education, and the benefits of higher education, where perhaps previously we focused too much on papers that were more pessimistic than the literature on the topic.

Why We Don’t Recommend Education Interventions

However, we still believe that interventions in health are more promising than interventions in education. The reasons are:

they have a stronger base of evidence that points toward major impact

health interventions are easier to scale with a predictable impact

health projects get more reliable feedback, and so have have been able to improve delivery over time

there are few education charities that seem to operate as effectively as health organizations like AMF.

This is generally believed to be the case not only by us, but also by 

More importantly, the evidence suggests that interventions in children’s health may be even better at improving education than investment directly in education. Donald Bundy, writing for the World Bank, 

 “deworming has proven to be one of the most cost-effective interventions to get children into school”. Nicholas D. Kristof wrote a column for the New York Times called 

, saying that “the cost of keeping a kid in school for an additional year by building schools or by subsidizing school uniforms is more than $100, while by deworming kids, the cost drops to $3.50.”

Why might this counter-intuitive fact be true? It’s because poor health is a key reason that many children aren’t able to go to school, or don’t develop physically or mentally to their full potential. Helping them become healthier removes a significant barrier to attending and benefiting from school.

Nonetheless, there are various scenarios in which we might see ourselves recommending education projects in the future:

evidence supporting low-cost services such as online education

the best value global health problems lack room for more funding

the arrival of stronger evidence on what works well in education, or a charity as streamlined as AMF.

Image Credit: GiveWell's Site Visit with GiveDirectly. Used with permission.

Blog

A CONVERSATION WITH HOLDEN KARNOFSKY, CO-FOUNDER OF GIVEWELL-PART II

Should We Tell People That Giving Makes Them Happier?

A Conversation with Holden Karnofsky, co-founder of GiveWell-Part I

New Updates to Education Research

Project Healthy Children - Nurturing the next generation in Africa

Don’t Call Us a Charitable-Industrial Complex

Measuring What Matters: Two key indicators for the Successor Development Goals

GiveWell interview: on causes outside global health, GiveWell Labs and fundraising

Just how rich are you?

Don't be shy, donate to SCI!