Max Dalton

This blog post summarises my research on the cost-effectiveness of medical research. The full report can be found 

. I’d greatly appreciate comments on the full report!

At the moment, one of Giving What We Can’s top recommended charities is the 

 which distributes treatments for neglected tropical diseases to endemic regions. This has been demonstrated to be a highly cost-effective intervention, with a seemingly huge impact on people’s lives. It is great to do something that seems so obviously good, and effective.

However, it may be that there are even cheaper, more effective interventions out there. Perhaps we can discover a vaccine for schistosomiasis, or some insecticide that can disrupt their life cycle once and for all. Such interventions could be cheaper, and more effective, than the existing drugs for schistosomiasis.

If so, it might be worth funding research in order to develop these interventions. This one-off cost would open up such great new funding opportunities that it could be even more efficient than giving to SCI, in the long-run. 

It might be worth spending money on research now, that would pay off by having an even bigger impact on people’s lives later on.

So to find out how to do the most good, we should consider this other option: 

When selecting a cause you want to find something which is a significant problem (important), which you can have a big impact upon (tractable), and which others have ignored (neglected). This helps to ensure that you have a big impact, and are not duplicating work done by others.

There is an incredible amount of money spent on medical research - billions of dollars per year.[1] So how could medical research be neglected?

Well, when we look at the breakdown of that spending, we find that it is incredibly uneven. Diseases that cause 90% of the disease burden receive less than 10% of the medical research funding, an absurdly low figure.[2] And of 1393 drugs developed between 1975 and 1999 only 16 - just over 1% - were targeted at tropical diseases or tuberculosis, despite their significant disease burden. 

The tropical diseases that affect the poorest people in the world receive little funding.

Why? Most of the funding for medical research comes from governments in high-income countries, or from pharmaceutical countries focused on lucrative high-income markets. High-income countries have the money to spend on research, and they spend it on their own problems, whilst low-income countries can’t afford research, despite the massive impact it could have.[3] This is just what we see in health more generally: much more is spent per capita in higher-income countries, although a great amount of the disease burden is in lower-income countries.[4]

It therefore seems that tropical diseases are relatively neglected, and for the rest of this post I will focus on these diseases.

At first pass, it seems clear that this health research could be very important. It could find vaccines, drugs, diagnostics, and vector controls that could help to bring about the (near-) eradication of diseases that adversely affect millions of lives. But things aren’t quite so simple.

It may be that, for some diseases, no more health research is needed. For instance, it may be that, since the interventions against schistosomiasis are so cost-effective, there is no need for a vaccine, and if this is the case, schistosomiasis research is less useful.[5] We need to 

. It requires specific scientific knowledge to ascertain which diseases are most promising in this respect (something I discuss later).

One example of a disease for which a vaccine is needed is HIV. HIV not only has a massive disease burden, but is also a relatively expensive disease to treat, sucking up healthcare funding. Whilst there are some current control methods, like male circumcision and condom use, they are difficult to popularise. Having another tool to block transmission could greatly reduce the disease burden, and save healthcare spending that could be used on other diseases.[6]

Having established that some tropical diseases appear to be both neglected and important, we turn now to find out how tractable research is.

How much will it cost to produce a new drug or vaccine for this disease?

 This is a difficult question to answer. By its nature, research is the exploration of the unknown, and so we don’t know very much about the new drug or vaccine that will come out at the end of it (if we did, we wouldn’t have to do the research!). This means that we don’t know very much about the type and amount of research needed to get us to that finished product.

Nevertheless, we can get a reasonable idea by examining past drugs, and seeing how much it cost to develop them. One highly cited study finds that the average cost of developing a new drug is $800m.[7] There are some reasons to think that this might underestimate current costs somewhat: for instance, the figure is over 10 years old, and drug development costs appear to rise above inflation.[8] The true figure is therefore probably a little higher than this.

But it isn’t the average figure we interested in. We know that some diseases are likely to be much more simple than others. 

What is the development cost likely to be for the types of tropical disease that we are interested in?

Tropical disease research tends to be carried out by public-private partnerships, rather than by the private sector alone. It appears that these partnerships face lower costs, since their charitable purpose allows them to gain donations of equipment, or discounted prices, reducing their costs. Moreover, tropical disease research is often carried out in low-income countries, where trials are less expensive. Finally, fewer clinical trials are needed for the development of some of these drugs.

It therefore seems that the out-of-pocket costs of this sort of development may be less than half of the cost of private development, or less than a quarter if you account for the cost of capital, 

perhaps in the region of $150m-$178m.[9] So this sort of research could in general be significantly cheaper than most drug research could be.

However, it is important to note that this will vary by disease, and that some tropical diseases look particularly difficult to produce drugs for. For instance, HIV is a particularly complex disease, making vaccine development tricky.[10] This means that it is expected to cost much more to develop a vaccine for HIV - perhaps more like $15bn-$20bn than $800m.[11]

In order to truly assess the tractability of research into a specific disease, we need to investigate the particular scientific challenges relating to that disease. Though tropical diseases may be cheap to develop drugs for on average, 

we need to know the scientific challenges that a particular disease poses in order to fully assess tractability.

Estimating the cost-effectiveness of medical research

In the last section, we saw how much variation there was in neglectedness, importance, and tractability between diseases. We narrowed our focus onto a particularly promising set of diseases: tropical diseases (which are neglected relative to other diseases), that need a new type of treatment (and so are important), and that don’t seem to present particular scientific challenges (so are tractable). Homing in on the right disease is really important.

But it is also useful to come up with a rough analysis of the cost-effectiveness of interventions, to see if it is plausible that research into any of these diseases could be better than Giving What We Can’s current top recommended charities.

Giving What We Can’s top charities tend to be able to avert a DALY (a disability-adjusted life-year, which is a measure of health) for around $50. When I searched for estimates of the cost-effectiveness of research into tropical diseases, I found estimates ranging from $8-$235/DALY, with most estimates grouped around $10-$100/DALY. Inspecting the biases of these papers, it seems that these papers somewhat overestimate the effectiveness of research. 

The true figure for tropical diseases as a whole could be more like $50-$300/DALY.

there is likely to be a great variation in effectiveness between diseases, so these estimates indicate that the very best research could well do better than $50/DALY.

 The right type of research could be the most effective intervention we’ve yet found.

Unfortunately, we can’t yet be sure what these best opportunities are. But it is worth doing more research to find out what the best interventions are. We need to answer three questions.

First, we need to look at the current treatments available for tropical diseases, and the scientific challenges to creating new treatments. This will allow us to 

identify for which diseases research is both important and tractable

. Detailed scientific knowledge will be needed to answer this question.

gain a deeper understanding of the current funding situation, and which types of funding are most promising

. There are different ways to fund disease research: we can pay people to conduct studies (‘push funding’) or offer them prizes when they discover new treatments (‘pull funding’). Each of these types of funding has advantages, but each is likely to be especially suited to particular situations. A detailed analysis of how to use these funding mechanisms could help us to maximise impact.

develop better models for cost-effectiveness

. Many of the cost-effectiveness studies cited above failed to account for certain epidemiological considerations, or considered the average rather than the marginal impact of donations. Fixing these problems would allow us to build more reliable models of cost-effectiveness, which would help us to compare research to other types of intervention.

That is a research agenda. But we still need to ask: 

should we fund research into schistosomiasis, rather than give to SCI?

 We shouldn’t fund research just yet. While it seems likely that research overall is relatively effective, and that some research can be more effective than the current top recommended charities, we just can’t be certain enough about how effective it is, or which diseases need research the most. I expect that giving to tropical disease research will have a lower impact than giving to SCI.

in a few years, as we understand research better, I expect that to change.

In the meantime, it may be a good idea to put more time and money into answering the three questions I set out above.

Applied Clinical Trials Online: Sizing Clinical Research Market

COHRED: Assessing the Efficacy of Health Research as aDevelopment Strategy in Poverty Reduction - Strategy Papers, p. 6

 notes a strong correlation between Norwegian research spending and Norwegian disease burden, but a much weaker correlation with the global disease burden. It appears that Norway focuses on Norway’s problems.

The picture actually seems to be more nuanced than this: see Appendix 1 of my full report.

However, there are also concerns that an HIV vaccine would have low take-up due to social stigma, or that a vaccine might increase risky behaviour. ART (a drug that slow HIV/AIDS development) costs $500/year per patient. See 

Copenhagen Consensus: Vaccine Research and Development

for further discussion of these issues, and a cost-benefit analysis of further research.

The price of innovation: new estimatesof drug development costs

Disease Control Priorities in Developing Countries. 2nd edition., Ch. 6

. Note that a HIV vaccine seems highly important but relatively intractable. How cost-effective research is will depend on the tradeoff between those two factors.

Should we give to SCI or fund research into schistosomiasis?

How valuable is medical research? Finding the solution to a problem in medical research is not only difficult, but it’s difficult to know just how difficult. Given that, how could we know the return we can expect on investing in medical research? Owen Cotton-Barratt is trying to find a framework for tackling precisely these kinds of questions. This post gives a sketch of his current model, and what it might be able to teach us about the value of medical research.

Owen Cotton-Barratt is Giving What We Can’s director of research, and also works on the Global Priorities Project (GPP) [1] within the Centre for Effective Altruism. The GPP works to try to find out how we should compare the different causes that we could be working on. This is closely tied to Giving What We Can’s aim of finding out how individuals can help people the most with their donations.

Owen’s current project is developing a framework to treat problems of unknown difficulty. It’s a work in progress, but he wanted to see whether the kinds of results the model produced would be useful by working through a specific example – the value of medical research. In this post I present a brief sketch of both his model and his results. Note that this work is very preliminary, and my statement of it is rough!

How might you go about finding out how much benefit you can get for an investment in medical research?

You need to build a simplified model that can start to give us a handle on what’s going on, but that gets the major effects approximately correct. This model aims to find a best guess for the average benefit (measured in Quality Adjusted Life Years (QALYs)) produced by an extra unit of resources (measured in dollars) invested in medical research, as well as some idea of how robust this estimate is. It starts by finding the general shape that returns to medical research would have, and then uses rough numbers about the world’s actual research spending and life expectancy increases to find out how many QALYs we might expect to generate per million dollars we put into medical research.

How do the outputs of medical research vary as resources go in?

In research in preparation, Owen argues that we should think the beneficial knowledge from a research area will scale approximately logarithmically, in expectation, with the resources invested. If the research area has many small discoveries rather than a few large discoveries, its development should also exhibit roughly logarithmic behaviour. In the case of medical research, this appears to be a reasonable assumption.

This means that a doubling of resources is equally good regardless of starting point. So money would have diminishing marginal value when put into research, as we might expect. Of course, this model does not hold for arbitrarily large values, but it is a fair approximation over the range we’re interested in.

How many QALYs are produced per $ invested in medical research?

The very rough model above seems to indicate that the knowledge generated by medical research varies logarithmically with money put in. That’s an interesting result, but what we really want to know is how many QALYs we could generate by donating a certain amount to medical research. To assess that, we need to ask how useful extra knowledge is. At first glance, it seems like it generates health benefits in a stream of constant size into the future, so even a small amount of extra research today will produce very large numbers of QALYs.

However, we should assess the value of extra research by the counterfactual where we don’t do this research. Whereas the obvious counterfactual considers the research never happening, the correct counterfactual is generally that the research will happen, but later. Of course the resources that would have been used to undertake this research later will now be freed up, often for further research. But we get diminishing marginal returns from research as the low-hanging fruit are taken, so although there will be a stream of counterfactual benefits going into the future, it will be decaying and its total size may well be finite.

To figure out the actual value, we need some numbers about current medical research. The numbers below are rough, as this is a proof-of-concept calculation. Where they aren’t sourced, they’re Owen’s best guesses. Putting more work into getting good data to use in the model could be a good way to produce more accurate outputs.

How much money is spent on medical research in total in a year? Around $270billion, of which around $100billion is publicly funded.[2]

What about other resources? If we doubled the funding, we wouldn’t double the number of talented people willing to work in the field. And some progress in other areas, such as hardware manufacture, may help medical research. Properly we could consider the elasticity of medical research with respect to these different factors (that is, the responsiveness of the field to change in these factors), with the observation that the elasticities should sum to 1. Alternatively we could estimate the dollar-equivalent value (to progressing medical research) of all of the resources going into the field. We might guess this is between $400 billion and $900 billion, with a median guess of $600 billion.

How fast is the investment growing? Probably between 2% and 5% per year. [3] The median guess is 3%.

How much has life expectancy increased per decade? 2 years. [4]

Not all of that life expectancy increase is due to medical research. On the other hand, medical research also improves quality of life. Given that, how many QALYs per person does medical progress create each decade? 0.15 to 1.5, median guess 0.5. Hence 0.05 QALYs are added to an expected life per year due to medical progress.

It’s possible that not everyone benefits from medical research, because its results don’t reach those in the developing world. How many people are benefited by medical research? Somewhere between 1.5 billion and 7 billion. Median guess: 3 billion.

How much total benefit is produced in one year from the resources we currently put into medical research? The added QALYs are to a person’s total expected QALYs, not the number they get each year. So we model the benefit as just accruing to people dying in a particular year: 1 in 70 people. What we get is 0.05 * 3,000,000,000 / 70, which is approximately 2 million QALYs.

What are the long term benefits and effectiveness of medical research?

You might think that the benefit that medical research produces in a year will continue every year from then on, because once you’ve found a vaccine or cure you can continue to use it. However, we shouldn’t be comparing the case of us finding the vaccine with the case in which it was never found – chances are it would have been found, just later on. On the other hand, the money that would have been used to find that vaccine later on, can now be used to fund other medical research. What we have effectively done is moved medical progress as a whole forward. Therefore, the benefit from funding medical research does last into the future.

In order to find the total value of funding medical research, we need to know how long into the future our funding continues to be a benefit. Since medical funding is increasing each year, the benefit from a particular donation decays over time. How quickly this happens depends on how the benefits from medical research vary with resources put in. Using the result that this is log-linear gives that the benefit of research today decays at a discount rate equal to the rate of increase of medical funding. With a 3% annual increase, the sum of the infinite series generated is equal to 33 times the benefit generated in one year without accounting for this.

The total benefit produced by the $600billion put yearly into medical research is the number of QALYs benefit in a year (around 2 million), times the effective number of years the benefit continues for (33.3). Therefore, the total benefit is around 70 million QALYs. That means per million dollars put in, 110 QALYs are produced. That is $8,500 per QALY. Assuming that the error in our estimates by component are independent, Owen also worked out a rough 90% confidence interval, of between $2,000 and $36,000 per QALY. Accounting for model uncertainty might make it a little wider again.

The result we got – that the return of medical research is around $8,000 per QALY - is very rough, and has a high variance. However, it does give us an idea of its value, which is very useful. There are two particularly interesting things it tells us: This estimate indicates that we are under-investing in medical research. The NHS is typically happy to fund treatments up to around $50,000 per QALY. If medical research on average is more cost-effective than that, rich countries should be investing more in it.

The estimate indicates that medical research as a whole is not typically as effective as the very best direct interventions (like providing people with bed nets). Despite the initial appeal of research as something which will have far ranging ramifications, on average deworming children is more effective.

However, our estimate has been for the average effectiveness of medical research. It is quite plausible that particular research efforts are highly cost-effective, particularly if they are in fields which are likely to be neglected because the beneficiaries would be very poor (for example, developing a vaccine for malaria or schistosomiasis). With further work we could begin to make comparisons of these areas.

How valuable is medical research?

Max studied Philosophy, Politics, and Economics at Balliol College, Oxford. He currently works at the Centre for Effective Altruism, where his roles have included research, writing, and management.

Max Dalton

About

Articles

Should we give to SCI or fund research into schistosomiasis?

How valuable is medical research?