Stay up to date with Giving What We Can's blog. Featuring the latest news about our community, how to donate effectively, charity evaluation, community events and more!

Blog

Giving What We Can's blog, featuring the latest news from our community.

Donate

Pledge

footerNavigation

About us

Contact us

Our team

Our values

Our members

Our events

Transparency

Our mistakes

Our impact

Donate to Giving What We Can

Careers

Leave feedback or make a complaint

About

Our best charities

Take a giving pledge

Get involved

Make a bequest

Videos, books, podcasts

FAQs

Learn more

Press kit

Media coverage

Case studies: People who pledge to give

Media

Footer Navigation

DONATE

topNavigation

Best charities

High-impact causes

Why give to funds

How much to donate

Why donate publicly

Recommendations

Our research & approach

Comparing charities

Evaluating evaluators

Our approach

Giving guides

Videos, books, podcasts and essays

Hesitations about charitable giving

How rich am I?

Can money buy happiness?

Resources

Research

About Us

Our governance structure

Team

Newsletter

Videos

Take the Pledge

Connect with our community

Share our ideas

Events

Giving games

Charity elections

Pledge drive

Fundraisers

Social Shareable Images

Community

What is the Pledge?

Why pledge instead of just donating?

Share the Pledge (inspire others!)

Pledge signers

My Pledge

🔹Trial Pledge

Company Pledge

Further Pledge

Bequests

Have a 1-on-1 call about pledging

Donate now

Manage my donations

Tax deductibility

Top Navigation

Ben Hoskin

Decision Making under Uncertainty

Sadly, there are no certainties in life. In charity research, as in other decisions, we have to deal with the fact that we can only ever have 'probabilistic' knowledge: there is always some chance we might be wrong. Indeed, the available information on intervention cost-effectiveness leaves us with considerable uncertainty about the true figures.

A naive way of dealing with this would be to simply work out what the most likely outcome is, and focus on that. So if we think a drug will probably save a life, we count the drug as saving a life. If it probably won't, we consider it as not saving a life.

However, this is clearly a dangerous approach. My house probably won't burn down this year - but I still buy house insurance. I probably won't be hit by a car as I hit the road - but I should still check for cars. When the stakes are high enough, we definitely need to take into account low-probability events.

On the other hand we shouldn't simply fixate on the best or worst scenarios. We take reasonable precautions against being involved in a car accident - but we still cross the road. Instead, we need to take into account both the severity of the consequences of a possible outcome, and the likelihood of that consequence.

The mathematical procedure for doing this is using 

. We multiply the probability of each outcome by its magnitude, add them together, and produce an expected value.

Expected Value = (Probability of outcome 1)

(Magnitude of outcome 1) + (Probability of outcome 2)

(Magnitude of outcome 2) + ... + (Probability of outcome n)*(Magnitude of outcome n)

This approach has many advantages; those interested in the mathematics should see the Wikipedia articles on 

The Expected Value approach requires we be able to assign probabilities to each of the possible outcomes. But this can be very hard, or even seem impossible - how can we assign a probability to something that only occurs once, like the Duke of Wellington winning at Waterloo?

We deal with this by treating probabilities as 

. A probability doesn't express something objective about the world like frequencies or propensities. Instead, it expresses a mental fact about our confidence in a proposition; a probability of 1 means utter certainty, and a probability of 0 means utter disbelief.

In practice we never have the infinite amounts of evidence requires to achieve total certainty - our probabilities lie between 0 and 1. And this is exactly why we need to use expected values.

In some cases it's easy to assign a credence - if half the population is female, we can happily assign a 50% probability to a randomly chosen person being female. Some cases are more difficult - how can we put a probability on the Democrats winning the next US presidential election?

However, while this is hard, we just don't have a choice. We have to make decisions, and if we don't use probabilities - or something equivalent - we run the risk of behaving 

. Our actions will always imply credences, even if we don't consciously hold them. While gut instinct works well for familiar everyday decisions about our local environment, or decisions that don't have big consequences, humans don't generally have well-trained intuitions for the effectiveness of different approaches. Using credences on the other hand and embracing the uncertainty that they make explicit, offers us the best chance to help the most people.

Owen Cotton-Barratt

Making practical estimates in cases of large uncertainty

A major part of our research is about producing quantitative estimates in order to help inform our decisions and recommendations. That could include which charities to recommend, what to research in more depth, or which career to choose. This post explains some methodological issues around making these estimates, and how we are addressing them.

We want to justify our estimates, and to make them as accurate as we can. However just about everything has some uncertainty attached to it, and it would be hopelessly counterproductive to ignore all estimates we are not completely confident about. But nor should we forget about our uncertainty, as it impacts how we use those estimates. So we'd really like to keep track of the uncertainty, along with the other parts of the estimate.

This is all well known, and those making estimates are generally aware of it. Typically people keep track of upper and lower bounds, as well as a best guess. For example if 

estimating the number of piano tuners in Chicago

, you might say your best guess is 125 but also give a lower bound of 30 and an upper bound of 500. While this is much better than ignoring uncertainty, we've noticed that there are some problems with this approach.

Firstly, it's not clear what is meant by "bound". Since it's very rare for us to be completely certain something lies above or below a given figure(1), the bounds presumably play a similar role to a confidence interval. But different people may have different definitions in mind. For example, one author could use "lower bound" to mean the level they are 90% certain the true value lies above (the 10th percentile in their subjective estimate), a second the same term to mean the 5th percentile, and a third for the 1st percentile. The numbers produced are not going to be meaningfully comparable. This is tolerable for back-of-the-envelope calculations, but when we are using such judgements to guide decisions, it is worth standardising. Similarly, it can matter whether the "best guess" is supposed to represent a median or a mean (the distributions we are considering often have fat tails which can cause these two to diverge substantially).

A second issue is easiest to explain with an example. Suppose we want to estimate the impact 

 of a campaign to persuade more people to volunteer for GWWC. We can break down 

 is the number of volunteers it generates, and 

 is the average (mean) impact of each volunteer. This expression may be useful because we have a better idea of how to estimate 

 individually. Let's assume now that we have some best guesses E(

. We multiply these together to get a best guess E(X) = E(

(2). It seems we should also be able to multiply our upper bounds 

. But if we choose a given meaning for upper bound-let's say the 90th percentile-we can see the problem with this approach.

There's only a 1 in 10 chance that the true value of 

, and a 1 in 10 chance that the true value of 

. Since it seems possible for one of these to happen without the other, they aren't perfectly correlated. If very high or low values for A and B don't tend to occur together, there's much less than a 1 in 10 chance that they're both extreme enough that the true value of 

 should be a bit higher than the product of the lower bounds for 

To be more precise about how to find an upper bound on 

, we can build a model of how our estimation error is distributed. We are developing such a model. It also aims to help with the first issue discussed above by standardising procedure and usage.

I'll give an overview of the model while ignoring some technicalities. Our model is for cases where we can break up the value 

 of other values, like in the example above. We assume a log-normal distribution for 

. The justification for this assumption is that if a lot of variables are multiplied together, by the central limit theorem we expect the product to converge to a log-normal distribution(3).

To work out the parameters for the log-normal for 

, we just need to keep track of a couple of values associated to each of the 

. We can normally infer values for these if we have a best guess and a confidence interval, which is what people are used to providing. We want this method to be accessible, so we'll provide a spreadsheet to take care of the calculations so that you can use it without having to follow all of the technical details.

Finally, what should we use these numbers for? Let's assume we're evaluating the cost-effectiveness of different interventions, in DALYs/$. We might use different aspects of the figures for different purposes. If we want to recommend an intervention we are sure is doing good, we might look for a high lower bound. This will tend to favour cases where are estimates have low uncertainty. In other cases we might care more about the expected DALYs/$, which is likely to be higher than our best guess, and in cases with great uncertainty significantly higher because of the fat tail of the log-normal distribution. We should be careful when doing this, because we are likely to see regression to the mean especially out in the tail; if we have lots of estimates with large uncertainty, some of them are likely to be substantially overestimated by chance, and we risk focusing our attention on these in appropriately. We're still considering the appropriate way to deal with this. One thing which is clear is that these estimates can be very helpful in identifying where further research is likely to be most useful. Things with high expectation and high uncertainty are the best candidates. You can also see which parts of the calculation are contributing the most uncertainty and warrant further analysis. In most cases, we'd expect this research to reduce our expectation of the intervention's cost-effectiveness, but when it increases it can do so by a large amount, and these cases can be of huge potential importance.

(1) Bounds of 0 and 1 on probabilities are a notable exception.

(2) Aside for the technically minded: this works if "best guess" means expectation; it also works for medians if our subjective probability distributions for the true values are log-normal[a].

(3) Though note convergence can take a long time in the tail, something we hope to explore more in future.

Michelle Hutchinson

Giving What We Can in the new year

The end of 2012 was a busy time for Giving What We Can - we ran our first sixth-monthly review process. This involved presenting a progress report and a plan for the future to our advisory committee (composed of GWWC members from a variety of backgrounds) and to the trustees. Following on from that, I'd like to share with you a bit about our plans for the upcoming months, and the advisory committee's feedback. We've already 

, so I'll be concentrating on other areas. If you'd like to see more detailed versions of our plans, retrospective review, and advisors' assessment, please 

GWWC is a community of people who care deeply about helping those in the developing world as much as possible. Therefore, GWWC's ultimate aim is to make sure that as much money as possible is given to the most cost-effective charities we can find. Our plan for the coming year has attempted to specify the ways in which we intend to do this in broad outline, as well as describe more precise plans for the next few months.

In fulfilling our ultimate aims, we have two key sub-goals:

1. To provide a complete resource for those seeking to help people in the developing world as much as possible …

… by providing high-quality information about poverty, and the most cost-effective ways of alleviating it, through our website, lectures, and publications.

… by creating a supportive community of givers.

… by providing a fast and reliable means of donating to the best poverty-related charities.

2. To get more people using this resource in guiding their giving …

… by increasing the number of members in gwwc, the number of chapters, the amount of money pledged, and the amount moved.

… by raising awareness of effective giving amongst the general public, governments, and NGOs.

We also want to support other organisations focussed on effective giving and effective altruism. This includes our sister organisations, 

The research team will continue to provide high quality information on charitable interventions and methodology, which we'll regularly share on the blog and the main website. We'll initiate a members' survey, to find out what information our members would most like to see on the website, and in what form. We will also be asking about how they first got involved, and any obstacles they faced, so that we can help others overcome similar problems.

Our web team is working on updating the members' forum. When the new forum is up and running, the research team will use it as their primary place for sharing drafts and discussing projects. As well as being handy for the research team, this will allow members to read about what we're working on, give comments, and suggest topics.

We recently polled our members to see whether they would be interested in GWWC running a donor advised fund, and since they were, we are currently looking at how to set one up. This will allow people to donate regularly and indefinitely to the same fund, which will then pay out to whichever charity is currently considered to be the most cost-effective. For busy people, we hope this would be useful because they wouldn't have to regularly check to see if our recommended charities have changed. It will also make it easier for people to leave money in their will to whichever charities are in future the most cost-effective.

GWWC is still in the fairly early stages of trying to find the best ways to spread the message of effective giving. So in the coming months we'll be doing some "market research", to try to find out how best to interest people in the idea. We'll also be studying the paths readers take through the website and working out in more detail which pages interest people, and lead to them reading on. That information will help us to improve the website such that more of the people who find it stay to learn more. For example, the charity quotations page gets a large number of hits from Google searches, and therefore we'll be experimenting with different ways to make it more engaging, and make it clear where to navigate to from there. Teams from Oxford Student Consultants will help us with this research.

The communications team has been working on using social media to make people aware of how much good they could do. For example, producing interesting infographics, which can be easily shared with others on Twitter and Facebook.

GWWC chapters, meanwhile, will be trying out different kinds of events. They will monitor how well each event works, and what resources can be provided centrally in order to help other Chapters run such an event.

The advisors felt that GWWC was growing rapidly, and that the 6 month plan contained a rather ambitious number of activities. They worried that having too many projects might mean losing sight of our overall goals. To avoid these dangers, we'll concentrate on prioritising carefully between activities, and working out how each of our actions links with our goals.

GWWC's roots lie in the University of Oxford, and, as such, many of our members are students, and many of our chapters are student chapters. The advisors were concerned about that leading to too much emphasis being placed on appealing to students. To address this, we will be concentrating on ensuring that we provide adequate and appropriate support for our non-student chapters, and to all our members.

The advisors discussed the relation between GWWC and similar organisations such as GiveWell. They questioned where GWWC would position itself relative to such organisations, in order to avoid duplication of work, and to do as much good as possible. This is a difficult question, which we are still in the process of answering. In research we're planning to focus to a greater degree on methodological considerations in the coming months, since that's where our comparative advantage lies. In a couple of months we will reassess whether this seems to be the best approach. We'll continue to think carefully about the best ways to work alongside other organisations, to make sure we're helping each other rather than competing.

In all our activities, we'll aim to monitor progress and to find out to what extent particular activities increase the amount of money going to the best charities. We're keen to keep evolving, and trying to do as much good as we can. If you'd like to help us in this endeavour, whether for 

Bastian Stern

The $/DALY Illusion: Part II - Averaging

In the first post, I discussed how we can be misled when we compare improvements in cost-effectiveness between organisations whose cost-effectiveness is reported in $/DALY.

The $/DALY-DALY/$-contrast also becomes important when we start talking about averages. For instance, suppose that you want to consider how much better it is donate to, say, charity D, compared to spending equal proportions of one's money on charities C and D(1). In this case, you would want to compare C and D's average cost-effectiveness to the cost-effectiveness of D.

To obtain C and D's average cost-effectiveness, it is tempting to simply take the familiar arithmetic mean(2) (which is what we typically mean by "the" average) of the figures for C and D's cost-effectiveness in $/DALY reported in the table in the previous post (20 $/DALY and 10$/DALY respectively), to obtain (20 $/DALY+10 $/DALY)/2= 15 $/DALY

However, this would deliver the wrong result. To see why, suppose I donate $10 dollar to each of C and D. I would thereby avert $10/(10 $/DALY)+$10/(20 $/DALY)=1.5 DALY . Given that I donated $20 in total, the average cost-effectiveness of my donations would be (15 DALY)/$20=0.075 DALY/$, or, reported in $/DALY, ~13 $/DALY.

The reason for this in general is this: Suppose we want to average ratios (e.g. cost-effectiveness reported in $/DALY) across a number of instances (e.g. different charities, whose cost-effectiveness the ratios measure). In that case, the arithmetic mean must be used to average the ratios only when the denominator is held constant and the numerator variable across the different instances. For instance, this is the case when the ratios are reported in $/DALY, the amount of DALYs averted by each charity is meant to be held constant and the amount of $ donated to each charity variable . Accordingly, when we naively took the arithmetic mean between C's and D's cost-effectiveness reported in $/DALY, we were effectively calculating the effectiveness of splitting one's money between C and D in such a way that the amounts of DALYs that are averted by our donations to each of these charities are held constant (which is clearly not something we are typically interested in). But what we instead wanted to hold constant across instances was the amount of money that flows into the charities .

This problem can be remedied by taking the harmonic mean(3) rather than the arithmetic mean of C's and D's cost-effectiveness reported in $/DALY, i.e. 2/(1/(20 $/DALY)+1/(10 $/DALY))=~13 $/DALY.

In general, when we are averaging ratios across a number of instances, we have to use the harmonic mean exactly when the numerator is held constant and the denominator variable across the instances, e.g. when, as in our case above, the ratios are reported in $/DALY, the amount of $ donated to each charity is meant to be held constant across instances and the amount of DALYs averted by each charity is variable.

Note that effectively, taking the harmonic mean amounts to converting the cost-effectiveness estimates for C and D into DALY/$, taking their arithmetic mean, and then converting back into $/DALY. Accordingly, had we reported C's and D's cost-effectiveness in terms of DALY/$ at the outset, we would not have been misled in using the straightforward arithmetic mean.

What the considerations in post I and this post suggest is that it is usually wise to report cost-effectiveness in terms of DALY/$ rather than $/DALY, because then

the comparison of differences in cost-effectiveness that we are about are more conspicuous, as the differences could be obtained by simple subtraction and;

the most often relevant averages of cost-effectiveness could be reached simply by taking the familiar arithmetic mean, rather than the more cumbersome harmonic mean.

Accordingly, it would be helpful if it became conventional to report cost-effectiveness in terms of DALY/$ rather than $/DALY. Until this is the norm, we must take care not to be misled when averaging and comparing differences in cost-effectiveness estimates.

(1) Or similarly, that you are interested how much better D is then choosing between C and D at random, with an equal probability of choosing either. (2) The arithmetic mean of n values a1, a2, a3, … is given by ((a1+a2+⋯+an))/n. (3) The harmonic mean of n values a1, a2, …, an is given by n/(1/a1+1/a2…+1/an).

Julia Wise

When is your help special?

I've heard the argument that we should "think globally, act locally" because we understand the needs of our own communities best. I'm willing to accept this for some situations.

I think it boils down to where your special help would be useful. If you pass a car accident, yes, your physical presence means you have a unique ability to help.

Likewise, when it comes to personal interactions, people are not interchangeable. Getting a card or letter from a stranger is not so comforting as hearing from someone you love. We evolved to interact with people we know in real life, and this still satisfies us more than some abstract kindness from a stranger.

So recognize the areas where you can be uniquely helpful: being kind with your family and friends. Sudden emergencies where you are physically present. Being a good neighbor.

But here's where I think people go askew with this logic: they feel that financial help should also work this way. After all, don't I understand the needs in my own community better than anyone? So I should fund projects in my own community, and other people should take care of theirs.

But rich people live in communities with other rich people, and poor people live near poor people. Your average American probably has several relatives or neighbors who have a few thousand dollars in their bank accounts. Your average Nigerian does not know any such people. When both rich and poor people give within their own communities, the opera gets a lot more funding than the maternal health clinic in Nigeria.

Of course, lots of first-worlders have given misguided aid because they misunderstood the needs of people in other countries. But you can misunderstand the needs even in your own community.

Last summer when I was studying in Ecuador, several of my fellow travellers volunteered in a local orphanage. The place was in bad shape, and it was hard to see the kids living in deprivation without wanting to do something. So some Americans decided to raise money for the orphanage from their friends at home. After all, they saw these kids every day and had seen the need first-hand.

Except they found out the reason the place was falling to pieces was that the owner was embezzling money. Those donations would probably never have reached the kids. Americans weren't alone in being duped by this guy. Every Christmas he had a big fundraiser and lots of Ecuadorans gave money, clothes, and toys. The locals were just as misinformed, despite it being in their own community, because they hadn't seen the financials.

This can happen in any country. At home in the US, I saw a lot of misguided community help during the summer I worked in a domestic violence shelter. We had a garage overflowing with blankets, because people had this idea that battered women need blankets. What they actually needed was a child's car seat so the children could safely ride to doctors' visits. The fact that donors were local didn't give them special insider knowledge about the residents' needs. It would have been better for them to donate cash so that we could use it for what was most needed (the car seat, not more blankets).

There are good and bad charities working in all parts of the world. Find ones that will use your money well, and that are doing important work. (And if you live in a rich part of the world, the greatest need probably isn't local.) Then donate money, which will help more than your blankets, old clothes, or volunteering.

And then find someone you love and give them the hug or the kind word that only you can give.

Rob Wiblin

Our research goals for the next few months

Giving What We Can recently produced a strategic plan for the next six months, including for its research. This plan has been discussed and improved after meetings with an external advisory committee and our trustees.

We are blessed with a wide range of promising research questions, which means the challenge has been picking out the best ones. Those we considered properly are listed in full at the bottom of this post.

Some highlights for what we hope to achieve over the next two to four months are:

 to coordinate and build our growing research team, and encourage greater autonomy among participants. This will complement a new discussion board for members.

Produce and run several training sessions for volunteers to help them understand the approach we use, including its strengths and weaknesses.

the strengths and weaknesses of existing analyses of climate change

making estimates when you are uncertain about many things

why we don't think we should have any 'pure time preference'

the expected returns on financial investment, and others valuable additions to our advice on 'giving now vs giving later' (yet to be determined)

the kinds of strategies that provide 'leverage', their likely strengths and weaknesses, and how they can be identified.

write on whether we should place less weight on the death of infants relative to older children

finish, or at least start, a survey of the meta-charities, like GWWC, that exist including their methodologies and recommendations

Revise our education, TB, hygiene and sanitation, migration, climate change and maternal health pages, and upload new pages on biomedical research and meta-charity.

Update the data and method used in our 'how rich are you' calculator to accommodate the latest research.

Maintain the flow of at least 1 research focussed blog post each week.

Come to a conclusion on whether to recommend 

Blog

Decision Making under Uncertainty

Making practical estimates in cases of large uncertainty

Giving What We Can in the new year

The $/DALY Illusion: Part II - Averaging

When is your help special?

Our research goals for the next few months

Some General Concerns About GiveWell

The $/DALY Illusion: Part I - Comparing Improvements

The Way It Was

Can we evaluate political advocacy?