How valuable is medical research? Finding the solution to a problem in medical research is not only difficult, but it’s difficult to know just how difficult. Given that, how could we know the return we can expect on investing in medical research? Owen Cotton-Barratt is trying to find a framework for tackling precisely these kinds of questions. This post gives a sketch of his current model, and what it might be able to teach us about the value of medical research.
Owen Cotton-Barratt is Giving What We Can’s director of research, and also works on the Global Priorities Project (GPP) [1] within the Centre for Effective Altruism. The GPP works to try to find out how we should compare the different causes that we could be working on. This is closely tied to Giving What We Can’s aim of finding out how individuals can help people the most with their donations.
Owen’s current project is developing a framework to treat problems of unknown difficulty. It’s a work in progress, but he wanted to see whether the kinds of results the model produced would be useful by working through a specific example – the value of medical research. In this post I present a brief sketch of both his model and his results. Note that this work is very preliminary, and my statement of it is rough!
How might you go about finding out how much benefit you can get for an investment in medical research?
You need to build a simplified model that can start to give us a handle on what’s going on, but that gets the major effects approximately correct. This model aims to find a best guess for the average benefit (measured in Quality Adjusted Life Years (QALYs)) produced by an extra unit of resources (measured in dollars) invested in medical research, as well as some idea of how robust this estimate is. It starts by finding the general shape that returns to medical research would have, and then uses rough numbers about the world’s actual research spending and life expectancy increases to find out how many QALYs we might expect to generate per million dollars we put into medical research.
How do the outputs of medical research vary as resources go in?
In research in preparation, Owen argues that we should think the beneficial knowledge from a research area will scale approximately logarithmically, in expectation, with the resources invested. If the research area has many small discoveries rather than a few large discoveries, its development should also exhibit roughly logarithmic behaviour. In the case of medical research, this appears to be a reasonable assumption.
This means that a doubling of resources is equally good regardless of starting point. So money would have diminishing marginal value when put into research, as we might expect. Of course, this model does not hold for arbitrarily large values, but it is a fair approximation over the range we’re interested in.
How many QALYs are produced per $ invested in medical research?
The very rough model above seems to indicate that the knowledge generated by medical research varies logarithmically with money put in. That’s an interesting result, but what we really want to know is how many QALYs we could generate by donating a certain amount to medical research. To assess that, we need to ask how useful extra knowledge is. At first glance, it seems like it generates health benefits in a stream of constant size into the future, so even a small amount of extra research today will produce very large numbers of QALYs.
However, we should assess the value of extra research by the counterfactual where we don’t do this research. Whereas the obvious counterfactual considers the research never happening, the correct counterfactual is generally that the research will happen, but later. Of course the resources that would have been used to undertake this research later will now be freed up, often for further research. But we get diminishing marginal returns from research as the low-hanging fruit are taken, so although there will be a stream of counterfactual benefits going into the future, it will be decaying and its total size may well be finite.
To figure out the actual value, we need some numbers about current medical research. The numbers below are rough, as this is a proof-of-concept calculation. Where they aren’t sourced, they’re Owen’s best guesses. Putting more work into getting good data to use in the model could be a good way to produce more accurate outputs.
Resources put into research
- How much money is spent on medical research in total in a year? Around $270billion, of which around $100billion is publicly funded.[2]
- What about other resources? If we doubled the funding, we wouldn’t double the number of talented people willing to work in the field. And some progress in other areas, such as hardware manufacture, may help medical research. Properly we could consider the elasticity of medical research with respect to these different factors (that is, the responsiveness of the field to change in these factors), with the observation that the elasticities should sum to 1. Alternatively we could estimate the dollar-equivalent value (to progressing medical research) of all of the resources going into the field. We might guess this is between $400 billion and $900 billion, with a median guess of $600 billion.
- How fast is the investment growing? Probably between 2% and 5% per year. [3] The median guess is 3%.
Benefits we get from research:
- How much has life expectancy increased per decade? 2 years. [4]
- Not all of that life expectancy increase is due to medical research. On the other hand, medical research also improves quality of life. Given that, how many QALYs per person does medical progress create each decade? 0.15 to 1.5, median guess 0.5. Hence 0.05 QALYs are added to an expected life per year due to medical progress.
- It’s possible that not everyone benefits from medical research, because its results don’t reach those in the developing world. How many people are benefited by medical research? Somewhere between 1.5 billion and 7 billion. Median guess: 3 billion.
- How much total benefit is produced in one year from the resources we currently put into medical research? The added QALYs are to a person’s total expected QALYs, not the number they get each year. So we model the benefit as just accruing to people dying in a particular year: 1 in 70 people. What we get is 0.05 * 3,000,000,000 / 70, which is approximately 2 million QALYs.
What are the long term benefits and effectiveness of medical research?
You might think that the benefit that medical research produces in a year will continue every year from then on, because once you’ve found a vaccine or cure you can continue to use it. However, we shouldn’t be comparing the case of us finding the vaccine with the case in which it was never found – chances are it would have been found, just later on. On the other hand, the money that would have been used to find that vaccine later on, can now be used to fund other medical research. What we have effectively done is moved medical progress as a whole forward. Therefore, the benefit from funding medical research does last into the future.
In order to find the total value of funding medical research, we need to know how long into the future our funding continues to be a benefit. Since medical funding is increasing each year, the benefit from a particular donation decays over time. How quickly this happens depends on how the benefits from medical research vary with resources put in. Using the result that this is log-linear gives that the benefit of research today decays at a discount rate equal to the rate of increase of medical funding. With a 3% annual increase, the sum of the infinite series generated is equal to 33 times the benefit generated in one year without accounting for this.
The total benefit produced by the $600billion put yearly into medical research is the number of QALYs benefit in a year (around 2 million), times the effective number of years the benefit continues for (33.3). Therefore, the total benefit is around 70 million QALYs. That means per million dollars put in, 110 QALYs are produced. That is $8,500 per QALY. Assuming that the error in our estimates by component are independent, Owen also worked out a rough 90% confidence interval, of between $2,000 and $36,000 per QALY. Accounting for model uncertainty might make it a little wider again.
The result we got – that the return of medical research is around $8,000 per QALY - is very rough, and has a high variance. However, it does give us an idea of its value, which is very useful. There are two particularly interesting things it tells us: This estimate indicates that we are under-investing in medical research. The NHS is typically happy to fund treatments up to around $50,000 per QALY. If medical research on average is more cost-effective than that, rich countries should be investing more in it.
The estimate indicates that medical research as a whole is not typically as effective as the very best direct interventions (like providing people with bed nets). Despite the initial appeal of research as something which will have far ranging ramifications, on average deworming children is more effective.
However, our estimate has been for the average effectiveness of medical research. It is quite plausible that particular research efforts are highly cost-effective, particularly if they are in fields which are likely to be neglected because the beneficiaries would be very poor (for example, developing a vaccine for malaria or schistosomiasis). With further work we could begin to make comparisons of these areas.