Giving What We Can's blog, featuring the latest news from our community.

Stay up to date with Giving What We Can's blog. Featuring the latest news about our community, how to donate effectively, charity evaluation, community events and more!

Blog

footerNavigation

About us

Contact us

Our team

Our values

Our members

Our events

Our mistakes

Transparency

Careers

Donate to Giving What We Can

About

Our best charities

Take a giving pledge

Get involved

Make a bequest

Videos, books, podcasts

FAQs

Sitemap

Learn more

Press kit

Media coverage

Case studies: People who pledge to give

Media

Footer Navigation

topNavigation

Recommended charities

High-impact causes

Why give to funds

How much to donate

Why donate publicly

Recommendations

Comparing charities

Choosing a cause

Inclusion criteria

Trusted evaluators

Our approach

Videos, books, podcasts and essays

Myths and misconceptions

How rich am I?

Can money buy happiness?

Resources

Research

Team

The Pledge

Trial Pledge

Further Pledge

Bequests

Why pledge

Pledges

Newsletter

Podcast

Videos

Sharing our ideas

Events

Giving games

Charity elections

Pledge drive

Fundraisers

Community

Donate now

Best charities

Manage my donations

Tax deductibility

Donate

My pledge

My donations

My income

My impact

Pledge

My giving

Top Navigation

William MacAskill

As you may have noticed, change is afoot in GWWC.

In the past, we feel that our research has been lacking in a few ways, including: a) We haven't made it clear, publicly, what our methodology has been for assessing charities; b) We haven't made it clear, publicly, what role GWWC has, relative to other charity evaluators; c) The website has not been updated regularly, so there's been a disconnect between our research - what we've known - and what's been broadcast publicly; d) We have spent fewer man-hours on research than was desirable. To a large extent, this has been because, until very recently, GWWC was run on an entirely voluntary basis. Now that we have full-time staff, we hope to be able to rectify these issues over time.

As a first step, this blog post and the next will explain our methodology in a little more detail.

The crucial question for any charity evaluator is: what metric to use? GWWC is solely focused on finding the most 

 charities: that is, the charities that have the highest ratio of good produced per dollar given to the charity. Along with 

, we are the only charity evaluators that are solely concerned with cost-effectiveness so understood. (1)

Explanation of what 'producing good' consists of.

The grain of truth in that question is that different people's conceptions of the good vary. Some people think that scientific knowledge is of value in itself; other people think that scientific knowledge is only valuable insofar as it improves the lives of others.

However, the sum total of human wellbeing is a component of any plausible moral view, and, on most moral views, increasing human wellbeing is the main way to do good in the world. For these reasons, it's this aspect that we focus on. If this is not a component of your moral view, then our research will not be of use to you (but then, if you aren't concerned by increasing human wellbeing, it's unlikely that giving to global poverty alleviation is your top priority). In future posts, we'll explain how to measure increases in human wellbeing.

Good question. A mistake to avoid is to think of producing good as to mean 

 good. To illustrate, consider two charities that one could give to. Charity A will save 10 lives with every $10 000 given to it; Charity B will save 5 lives with every $10 000 given to it. However, if you don't give $10 000 to Charity A, it will fill up its room for more funding from other donors (who wouldn't have given at all otherwise). Whereas if you don't give the $10 000 to Charity B, it will not fill up its room for more funding from additional donors. In this case, you should donate to Charity B rather than Charity A: though you directly cause 10 lives to be saved by donating to Charity A, you don't really do any additional good, because those lives would have been saved anyway. In contrast, you do make a difference (of 5 lives saved) by donating to Charity B.

Making this more precise: what producing good means is as follows. First, you consider two possibilities: (i) where you give $n to Charity X; (ii) where you burn the money. Now think: how good is the world in which (i) you give the money? And how good is the world (ii) in which you burn the money? The amount of good you produce by giving $n to charity X is the difference in the goodness of the world in which you do give, minus the goodness of the world in which you burn the money.(2)

Importantly, the above method means that all consequences are considered. So 'good produced' does not just mean "impact provable in a few years"; long-run consequences are just as relevant to cost-effectiveness as short-run consequences. Nor does 'good produced' mean 'the effect on the direct beneficiaries of an action'; effects on third parties are just as relevant as effects on direct beneficiaries.

But surely you can never know what organisation does the most good per dollar?

That's exactly right - we'll never know what the best charity is. However, that doesn't prevent charity recommendations from being possible. It just means that we need to think in terms of expected good. To see this concept in action, consider two donation opportunities. Donating $10 000 to charity A will certainly save one life. Donating $10 000 to charity B will probably (80% likelihood) save two lives; but has some chance (20%) of producing no benefit.

In general, to work out the expected value of an action X, you look at each possible outcome that could result from X; for each possible outcome, you multiply the probability of that outcome occurring by the goodness of that outcome, if it does occur; and you then take the sum of this product across all possible outcomes. So the expected value of donating $10 000 to charity A is 1 life x 100% = 1; whereas the expected value of donating $10 000 to charity B is 2 lives x 80% = 1.6.

The concept of expected value becomes especially important in situations of great uncertainty, such as charity evaluation. Further posts will discuss in more detail how to correctly apply this concept.

In principle, we're open to evidence from any source, and if certain sources prove to be particularly reliable or unreliable, we will alter which sources we rely on. In the past, we've used the latest peer-reviewed research from health and development economics, information from larger cost-effectiveness-focused academic projects such as the Disease Control Priorities in Developing Countries (2nd edition) and the Abdul Latif Jameel Poverty Action Lab, and on research from other charity evaluators, in particular from GiveWell. Like a growing number of organisations, we place particular weight on randomised controlled trials where they are available.

In the near future we will also begin publishing draft reports on topics and passing them around in order to to solicit additional information from experts and the public.

Sometimes, we have to rely more on 'back-of-the-envelope' calculations, and first-order research. This is especially true for some of the more conceptual, but potentially game-changing considerations.

What do you plan to work on, going forward?

We'll work on whatever we think will generate the most benefit with our additional research hours. This can include areas where:

We think that there are large gains from small amounts of work

We think there might be a game-changing or 'crucial' considerations, that could increase the cost-effectiveness of one's donations by several times.

Where we think we have particular skills that are lacking elsewhere.

Particular questions we will address in the future include:

Should one give now, or invest and give later?

This is an issue that satisfies (1), (3) and (4). It's potentially game-changing because it's at least plausible that one can have significantly greater impact with one's donations by investing one's money, and donating when we have better evidence regarding charity cost-effectiveness.

Should one give to 'meta-charity' such as charitable fundraising or cost-effectiveness research?

This is an area that satisfies at least conditions (1)-(3). The average return on investment from charitable fundraising is 5:1 - far higher than the typical market return on investment - and the highest-return fundraising can significantly outstrip that. This suggests that 'investment' in fundraising could be extremely cost-effective.

As well as funding fundraising, one could fund additional research. The current state of knowledge regarding cost-effectiveness is poor, and, in general, where knowledge is poor, the value of gaining new information becomes very high. In the case of cost-effectiveness evaluation, it also has great potential to affect large future donation streams.

What are the best sources of information out there to donors?

This is an area that satisfies conditions (1) and (2). There is a proliferation of charity evaluators out there - which should one trust, and why?

What are the most promising cause-areas for further research?

This is an area that satisfies (1) and (3); it's also an area that we feel it's important to explore in order to explain to givers why we focus on the areas that we do. Our primary question is: are there any cause areas that could plausibly contain interventions with expected cost-effectiveness considerably greater than developing world health? Examples of such cause areas include climate change, medical and other research, political lobbying and economic empowerment.

In the near future, we don't plan to do further in-depth evaluation of developing world health charities, as we've found GiveWell's research in this area to be of high quality, and we don't think that marginal research hours from us would provide much additional benefit. This kind of work does not appear to be our '

' in the charity effectiveness community. However, we will remain as an independent group in this area, and will check GiveWell's research before relying on it, as we would do with other sources of information.

Many of these questions will be tackled in a new format for us - 

A periodically updated list of possible research topics for new volunteers is 

Below are some of the people involved in our research team.

. As of late September, Rob has been our Director of Research. He graduated top of his year in Applied Economics from the Australia National University, and has worked as a research economist in several Australian Government agencies including the Productivity Commission.

. Sally is a graduate student in Development Studies (Research) at the London School of Economics. Prior to this, she read Philosophy, Politics, and Economics at Oxford, and spent a year working for NGOs in India, Cambodia, Malawi and the UK. She currently manages several volunteers in London, with a primary focus on micronutrients.

. Owen is a College Lecturer in pure mathematics at Oxford.

. Nic is a PhD student in machine learning at Oxford University. He graduated with First Class Honours from Cambridge, and received in 96% in his Master's degree in Applied Statistics at Oxford.

. Bastian is a postgraduate student in philosophy at Oxford University. He recently graduated top in his year with a double starred first from Cambridge, with one of the best undergraduate performances on record.

Michael Peyton Jones (intern and volunteer)

. Michael has background in computer programming, and recently graduated with First Class Honours from Oxford in Physics and Philosophy.

. Rossa is doing a PhD in development economics at the University of Pennsylvania, on a Fulbright Scholarship. He graduated with First Class Honours in economics from Oxford, where he won a prize for the best undergraduate thesis. He is currently taking time out from Giving What We Can's research to focus on his studies.

. Eliz is currently reading Clinical Medicine at Oxford University with an interest in tropical and infectious disease. During her preclinical studies she was awarded a Casberd Scholarship by St. John's College. She is currently investigating biomedical research and contagious diseases.

The research team also receives input from the Trustees and the Executive Director of Giving What We Can and external sources.

(1) GiveWell also places significant weight on 

 such as transparency and accountability, which they believe to be good proxies for cost effectiveness (2) Ignoring any effects from changing the amount of currency in circulation!

Researching What We Should

Krisztina Csortea

Since we published our post about developments in the field of malaria research earlier this week, the results of another interesting study have been made public. Nature magazine 

eported on Friday about phase III of the trial of a potential new malaria vaccine. Follow this link to 

New Malaria Vaccine Trial Provides Variable Results

Callum Calvert

Diseases in the developing world cause an enormous amount of suffering. Controlling them requires vast sums of money. Could funding the development of a vaccine, ending the problems caused by the specific disease once and for all, be an immensely cost-effective way to donate? The case for this type of charitable giving is obvious: giving money to people doing this research and development (R&D) will lead to an earlier vaccine-day (the time at which a working vaccine is ready for service). Bringing forward this point by even a day could prevent a serious number of DALYs from being inflicted. We've been looking at a few ways we could model the cost-effectiveness of funding a general vaccine development charity. This post will explore the first of these models.

The Sabin Vaccine Institute is a good example of this type of charity. Sabin is conducting non-profit R&D for a number of different neglected disease vaccines, but GWWC are particularly interested in their work on schistosomiasis (also knows as bilharzia, bilharziosis or snail fever), a vacc ine for which is scheduled to enter phase 1 clinical trials next year. The aspiration of this series of blog posts will be to get a reliable cost-effectiveness estimate of donations to this schistosomiasis vaccine development program.

To begin with we’ll assume that a vaccine will definitely be produced eventually, either by the program we donate to or by a different one. Then with any model there's a natural way to split the calculation up into 2 parts. Firstly we want to know just how much our donation will bring forward V-day. Secondly we want to find out how bringing forward this momentous occasion by 1 unit of time translates to positive effects in the developing world (measured in DALYs). Then we’ll combine these to get a figure for positive effect per donation.

So how much will our donation affect V-day? Say the charity is at a certain stage of its research, and plans to begin its next stage at some particular point in the future 

 they fill up a funding pot, of size G. If not, they will postpone to another date T years after the first date. We’re only concerned with whether my donation (of size D) makes the difference between the research going ahead and being postponed; so how effective we think it is to donate will depend a lot on how much we think the charity will fundraise elsewhere. If we think that they'll fill their pot and then some, or that everyone else will stay away, then it’s unlikely my donation will make much difference to the outcome. The problem of course is that we don't know how much everyone else is going to donate! To get around this we can try to estimate a distribution of how much we think everyone else will give in total. Plausibly, the total amount donated (excluding our donation) is distributed normally [The reason for this is that F is the sum of lots of other people’s donations. Assuming (for simplicity) that the individuals' donations choices are independent, the central limit theorem says that their sum will be approximately normal]. In that case, the probability that our donation will make a difference is P[G - D < F < G]; this is the probability that the total amount donated (excluding our donation) will be within D of filling the funding pot, so that our donation fills the pot (this probability is represented by the grey area in the graph below). So our expected time impact on V-day will be P[G - D < F < G]

p where p is the probability that the program successfully produces a vaccine (If the program fails to produce a vaccine then our funding has not brought forward the usage of any vaccine, and unfortunately, 

Now for part 2! There are 2 important positive effects which might come from having a vaccine available earlier.

Firstly people who are at risk from the disease will benefit from the vaccine being ready to use earlier. For the moment we'll assume that eradication is achieved exactly T years earlier if the vaccine is successfully developed without the delay in research due to lack of funding. (There are possible reasons why this might not be true, for example that the vaccine might not be the limiting factor in the eradication effort). Assuming that eradication is achieved eventually in all cases and happens in the same way whenever the eradication effort begins then by looking at the graph below we can see that our impact depends on the global burden of disease (DALYs caused by the disease annually) between T1, the time at which the vaccine will be ready if funding is reached, and T2, the time at which the vaccine will be ready if the next stage of development is delayed due to lack of funding. So if the GBD of the disease is B DALYs/year then the expected benefit of our donation is the expected time impact of our donation multiplied by B. This is B* P[G - D < F < G]

The second benefit is because the annual cost of controlling a disease could be massive. If we assume that the money being spent on control would otherwise be diverted to other health interventions in the developing world then potentially this would avert vast numbers of DALYs. Let the cost of controlling the disease be $C/year. If the average cost-effectiveness of health interventions in the developing world is X DALYs/$ then for each dollar diverted the benefit is C

X DALYs/year. Therefore multiplying as before gives expected benefit of our donation due to diverted dollars as C

Thus our first basic benefit estimate for a donation of size D to a vaccine development charity is the sum of the 2 benefits of our donation, (B+C*X)*P[G - D < F < G]

p DALYs. Assuming some fairly conservative numbers for the inputs and the parameters of the distribution of funding suggests that vaccine charities have the potential to be quite cost effective.

There is a other factors still to be taken into consideration. For instance, some vaccines are only partially effective and some diseases might be controlled by other means before a vaccine is developed. Allocating scarce biomedical researchers to vaccine research will take them away from other R&D programs. In future blog posts, as we explore some other models, we hope to find some ways to deal with these things.

How Can We Model Vaccine Development? 

Toby Ord

Several recent articles have reported on interesting developments in the fight against malaria. See below for a brief summary of these (with links if you are keen to do some further reading).

Efforts to eradicate malaria have significantly increased over the past decade. In 2011 alone, the three main donors, the WHO’s Roll Back Malaria Partnership, the Global Fund and the President’s Malaria Initiative, spent 

 on treatment and prevention. The increase in funding appears to have had a positive effect on the fight against the disease, according to WHO estimates there has been a 26% reduction in death rates since 2000. The UN has called for an even greater increase in funding. A $6.7 billion investment between 2012 and 2015 could prevent 640m malaria cases and 3m deaths. However, the prospects of malaria eradication, even given extra funding, might be less rosy than this picture suggests.

First of all, a recent study published in the 

 has found that the WHO’s estimates of deaths from malaria might be too conservative. Based on statistics gathered from 105 countries since 1980, the study found that malaria accounted for 1.24m deaths in 2010, nearly double as many as the 655 000 reported by the WHO. Even if the WHO did significantly underestimate the number of deaths caused by malaria, this of course doesn’t mean that efforts to fight the disease haven’t been successful, but it might lead us to question the amount of money donors would need to invest in order to eradicate it.

 in the Scientific American highlights a second potential concern. One of the principle ways of preventing malarial infections, bed nets, might not be as successful as it seems. Nets are treated with synthetic insecticides which repel mosquitoes and kill the ones that land. Bed nets also provide a mechanical barrier between the sleeper and insects. One of the reasons why bed nets work so well is that the periods of the day when they are in use coincide with periods when mosquitoes, including malaria carrying ones, are the most active as mosquitoes tend to be nocturnal. It has been known for some time that mosquitoes can evolve to resist the insecticide that the bed nets are coated with, however a recent study has found that some mosquitoes have started adapting their feeding times too. The study found that over a period of three years mosquitoes changed their feeding times from around 2 to 3 am to 5 am. The study was fairly comprehensive, it was conducted at two different locations in Benin and samples were collected before, during and after bed nets were distributed. What are the potential consequences of this for humans? Worryingly, people are more likely to be out and about during the new feeding times, and therefore are more likely to be bitten and potentially infected. On the other hand, mosquitoes that feed on humans during daytime are also more likely to be spotted by the victim. Furthermore, mosquitoes feeding at dawn might also be faced with new predators, such as frogs and lizards. Given the uncertainties surrounding the consequences of the findings of this study, bed nets still remain to be one of the most effective ways of fighting malaria.

Even if the mechanical effectiveness of bed nets is reduced by behavioural changes in mosquitoes, luckily bed nets are not the only card up the sleeve of those battling malaria. The BBC 

 last week that a recent study in the Lancet has come up with encouraging results when researching the effectiveness of a new anti-malaria drug scheme. The Affordable Medicine Facility for malaria set up by the Global Fund, subsidises drugs in 7 African countries. The study has found, that although the scheme had a low effect in Niger and Madagascar, it reduced the price of drugs in the other countries it was trialled in. 

, the scheme has been criticised, most notably by Oxfam, for limitng access and for diverting attention from more established methods of fighting the disease.

Malaria in the News

Michael Peyton Jones

Warning: We're now going to start assuming some serious statistics understanding! Not everyone will be able to follow the guts of the modelling, but the bottom line should still make sense. As before, we're using R to do most of the work.

, I explained a simple statistical model for assessing the effectiveness of donating to DCP. In this post I would like to describe some improvements to the model. The most important of these is allowing for the possibility of error in DCP's assessment process. Some of the variation in the DCP's results are due not to real differences in the cost effectiveness of treatments, but rather simply measurement error on their part. We will run simulations where we vary how large this error is, and see how much it reduces the value of DCP's work. The conclusion is that given our guesses about the size of the errors and the money their research moves, DCP still appears much better to fund than the best treatment they identify. Some objections to this model are then noted.

Estimating the Effectiveness of DCP2 - Post 2

Simon Jenkins

This blog post is going to be a kind of summary of the recent and very exciting launch event for Giving What We Can: Birmingham. Summaries are rarely exciting, and I would have said “review” but I can’t really review an event that I had a hand in organising without being accused of bias. So, rather than trying to be guarded against bias by carefully selecting a spread of positive and negative things to say about the event, I’m just going to wholeheartedly embrace my bias and go on and on about how great the launch was. As well as being an opportunity to blow my own trumpet, this blog post might provide some useful tips for anyone else looking to launch a Giving What We Can chapter or similar event (though saying that might just be blowing my own trumpet too).

The launch was quite a long time in the making. The first step and the thing that really got the ball rolling was contacting potential speakers to give lectures at the event. I had prepared a sizeable list of potential speakers, and my plan was to start from the top and work my way down. As it turned out I didn’t have to move down the list very much at all and I was pleasantly surprised by how willing and able everybody was to help. The first thing that speakers usually want to know is “when is this going to be?” and so contacting the speakers provided the impetus needed to fix a date for the launch. This, then, was a point to work towards. A 

. And it got things working a lot faster.

After this I spent many evenings staying late (later than usual anyway) in the office making all kinds of arrangements for the event. Professor Larry Temkin, one of our speakers, and I sent each other thousands of emails discussing travel arrangements, hotel ideas, timings and all the rest of it. I think I’m probably now qualified to work as some kind of travel rep, but as I’m committed to not being an evil person, that isn’t the life for me (1). Prof Temkin even asked me about must-see sights in Birmingham. I think my response was something to do with curry.

After months of organising (obviously I was kind of doing other things too) the event finally loomed upon me, and on the day of the launch I was still desperately scrabbling around trying to get Skype installed in the room that we’d be using. Peter Singer was due to give his talk over Skype, and so obviously without Skype, this would have been deeply difficult to achieve. Fortunately some of the excellent people at Global Ethics and Political Science (departments here at the university who also helped greatly by funding the Birmingham leg of Larry Temkin’s European trip) came to the rescue and we got Skype set up, literally just a few minutes before Prof Singer was due to be live on the air, beamed into the room onto the screen of a laptop with a microphone pointed at the tinny laptop speakers. It was clumsy and lo-fi, and those with questions at the end had to come up to the front of the room and talk into the laptop like some bizarre audition, but it all worked pretty well.

But that is a digression into a technical headache. Let me put some meat on the bones by telling you about the talks themselves. The content was of course of a very high quality, as to be expected with three eminent profs in the room (three if you count Peter Singer’s virtual presence anyway). Larry Temkin (Rutgers University) gave a talk that was enviable in the elegance of both its construction and delivery, talking directly to the “but I’m a poor student and can’t do anything” response to charitable giving by illuminating the multitudinous ways in which people waste their money, but with a warmth and honesty (admitting that he himself has two fondue pots) that stopped it from being preachy. Peter Singer (Princeton University) used his time-honoured thought experiment of the child drowning in a shallow pond, and the idea that some in the room might have been hearing this example for the first time was as inspiring as watching a child being born. Finally, Alan Fenwick (director of Schistosomiasis Control Initiative, one of Giving What We Can’s top recommended charities) gave a talk that was both moving and visceral, not shying away from graphic detail (and video clips) of the horrors of the neglected tropical diseases that his organisation fights.

So there we have it. Giving What We Can: Birmingham has launched, and you can even watch a video of the launch event 

. Thanks to everyone for their help (there isn’t room but you know who you are) and I’ll see you at the philanthropy games next week, which I may as well shamelessly plug here: it’s a debate about where to allocate a real sum of money. Room G41, 90 Vincent Drive, University of Birmingham (Edgbaston Campus), Wednesday November 7th, 5.30-7.30pm. No prior skills/knowledge required, and for every person that comes, an extra £10 will be donated!

(1) DISCLAIMER: I DO NOT THINK ALL TRAVEL REPS ARE EVIL

GWWC: Birmingham Launch

When searching for high-effectiveness charities, there are a few resources that are incredibly useful. One of those is the output of the 

 (DCP), which provides an analysis of the cost-effectiveness of a great number of health interventions. Given the value of this data for us as donors, we might wonder just how valuable this information is, and whether it might be worth donating to DCP to produce more of it.

In this post I will present a statistical model (designed by myself and Nic Dunkley) for estimating the effectiveness of donating to DCP. Although this will inevitably over-simplify the situation, it will be a first step that we can build from. Although the model may be useful for looking at other similar organizations, the estimates that I will derive rely on data about DCP itself.

Here's the idea. Firstly, we assume that DCP has some number of followers who make their donation decisions based on DCP's recommendations; that is, they put their donations towards whatever intervention DCP recommends most highly. If DCP says the best thing is supporting homeless shelters in the UK, then that's where the followers will send their money. So the current situation would have all that money going towards the intervention that the DCP2 report claims is most effective. We'll refer to the money donated as DCP's "money moved". Furthermore, we'll assume that this money is all translated into DALYs at the rate that DCP claims.

Now, suppose that there's a big bag of interventions that DCP assesses. Then we assume that for some amount of money, DCP can pick a random intervention from the bag and accurately assess its effectiveness. If it comes out better than their current best recommendation, then when they report on this fact, their followers will shift their donations accordingly.

This may not be the best way to model the DCP’s impact. Governments which make use of the DCP may not use it to identify the one ‘best’ treatment in order to fund that, but rather just to work out which treatments pass a minimum cost effectiveness threshold and which don’t. The higher above this threshold a project is, the more likely it is to get done. This would require a more detailed model, which we may produce at a later time.

However, If the original description were a good model of how DCP works, then we now have enough information to work out how much good we can do by donating to DCP! I'm going to be using 

, a statistics package, to do the heavy lifting.

Firstly, we need to know something about the distribution of the interventions in the bag. A quick reminder about probability distributions: the probability distribution of some random variable usually looks like a humped graph. Along the bottom are all the options we could have. In this case, that's the possible cost-effectiveness of a random health intervention, i.e. pretty much any real number. Along the side it shows how likely that possibility is. One very common type of distribution is a “normal” distribution. This just looks like a big hump in the middle that spreads out to the sides (often called a “bell-curve”). Intuitively, that means that most of the time you get one of the possibilities in the middle, and the other possibilities get less likely as they get further away from the middle. Normal distributions are quite nice to work with, and they're also pretty common. Importantly, they can be totally specified by just two parameters: the mean of the distribution (the average outcome you get), and the standard deviation (which tells you how far away from the mean things tend to be). Between the two of these you know where the middle of the hump is (the mean), and how wide it is (the standard deviation), and that's all you need to know.

Back to our health interventions. Fortunately, we have a lot of data from DCP about the cost-effectiveness of interventions: if we plot the effectiveness of the interventions they assessed in the DCP2 report, we get a distribution that looks a bit like this: That's definitely not a normal distribution, but it is a “log-normal” distribution. A log-normal distribution is fairly straightforward: if you take the logarithms of all the possibilities along the bottom, and then plot the distribution, you get a normal distribution. And if we do that for the DCP data, then we do indeed get a very plausible-looking bell-curve. Now, this makes it looks pretty plausible that the DCP data really does follow a (log-)normal distribution, and it'll be helpful if we can work out the two parameters for it that I mentioned above. For the moment, let's assume that taking the mean and standard deviation of the logarithms of the data gives us the right parameters – we can be more sophisticated (in particular we're going to want to think about the possibility of error), but I'll leave that for a later post.

Having the parameters for the distribution means that we can get R to make up entirely new data points as if they came from the original data set. And if the actual data really is distributed in the way we think it is, then this is a lot like DCP finding and assessing a new random intervention. We can then do the rest of our model process – check whether it's a better charity than we had before, and if so work out what the difference is – and get an answer for how much difference that one attempt would make. We can then use R to simulate doing this a large number of times, and then simply average the results! Thus: we generate a large number of samples, to represent a large number of investigations by DCP. We can then work out the difference in effectiveness between each “new” intervention and the current best option. The current best option is better than average, so we'll expect the difference in general to be negative. We can regard all those results as 0, however, as in that case there would (in our model) be no change in where the money goes. Finally, we can then average these results. Call that average A – that is, the average improvement in the effectiveness of the most recommended charity due to DCP doing one investigation.

At this point, we need to plug in two final, very important parameters. These are figures for the money moved (M), and for the cost of DCP doing an investigation (C). These are particularly important as the our final estimate is going to be the average good done by an investigation (A*M), divided by the cost of doing it (C). Our final estimate will therefore be directly proportional to M, and inversely proportional to C. We'll discuss these a little more in a future post. For now, I'm just going to give you some figures that are hopefully about right: namely $530 million for M, and about $100,000 for C.

Putting all this together, we get an estimate for the effectiveness of DCP of around 89 DALYs/$. For comparison, the best intervention that DCP claims to have found so far clocks in at 0.33 DALYs/$! As mentioned earlier, of course, this estimate is only as good as our model, but it certainly looks promising. In the next post I'll talk about how we can improve the model in a couple of ways, most importantly including the possibility of error. This will lead to a significant reduction in our estimate of DCP's effectiveness, but it will still appear to be extremely competitive.

This kind of research will be a high-variance strategy: most of the time nothing useful will happen, but occasionally there will be really, really good outcomes.

The hidden parameter in this model is the effectiveness of DCP's current strongest recommendation. In particular, if new interventions are discovered that have much higher effectiveness, then the need to do further research drops.

The model assumes that the distribution of interventions DCP would evaluate with more funding looks the same as the distribution of interventions they have already identified. In practice they may have looked at the most promising options first.

These results are not immediately applicable to charity-evaluators such as GiveWell. GiveWell looks at charities, not interventions, and we don't have a data-set for that in the way that we have for DCP. It may be that if we make some assumptions we might be able to adapt the data, but this is an open question.

Estimating the Effectiveness of DCP2

Peter Hurford

, Odysseus’s crew sailed past the island of the Sirens. In Greek lore, the Sirens were notorious for singing a song so enchanting that those who listened would sail towards the Sirens and end up crashing into the rocks, sealing their own fate.

The common solution of the era was to avoid listening to the song altogether – plugging one’s ears with beeswax. But Odysseus wanted to listen to the song… though, of course, he didn’t want to end up crashing his ship and killing his crew.

Thus, not trusting his future willpower, Odysseus had his crew tie him to the mast of the ship and precommitted to not be let go. Predictably, when he heard the song he begged to be released, but his crew followed his earlier orders and didn’t let him go. Odysseus survived on the basis of precommitment.(1)

The motivation for precommitment stems from a problem in human nature: there are some goals we find desirable (to know how to dance competently, to have good grades, to be fit and healthy, etc.) but still struggle to do certain activities that would accomplish these goals (take dance lessons, study hard for the upcoming test, exercise and eat right, etc.).

People are frequently averse to taking on short-term costs even if they would achieve long-term gains. (2) People value their current time much more than the future (called 

),(3,4) struggle with low expectancy of success, struggle with the drudgery of actually performing the task, and are occasionally impulsive and forgetful with their tasks.(5,6)

Such failures to do what we want and act against our considered judgments are commonly called akrasia, or “weakness of will”, a term first coined by Aristotle more than two thousand years ago. Odysseus recognized that he wouldn’t have enough willpower to resist the Siren’s song, so he took steps to ensure he would end up acting the way he wanted. He used precommitment to increase his willpower.

In our own lives, we face problems where we really want to do something, but are undermined by our lack of willpower and procrastination. In looking for a solution, we can be like Odysseus… except without our lives having to be on the line. Precommitment is helpful at reducing the impulsiveness which drives procrastination and other forms of akrasia.(7) While we don’t actually have to get tied to a mast, the idea is the same – one takes an earlier and easier action now (paying for a gym membership, promising a friend, or getting tied to a pole) that makes it much easier to perform the harder yet still desirable action later on.