Giving What We Can no longer conducts our own research into charities and cause areas. Instead, we're relying on the work of organisations including J-PAL, GiveWell, and the Open Philanthropy Project, which are in a better position to provide more comprehensive research coverage.
These research reports represent our thinking as of late 2016, and much of the information will be relevant for making decisions about how to donate as effectively as possible. However we are not updating them and the information may therefore be out of date.
Cause Area: Medical Research
The aim of this report is to provide estimates of the cost-effectiveness of medical research into diseases prevalent in low-income countries. This should help to assess whether it is a promising area for further research or even recommendations from Giving What We Can. I am attempting to assess this primarily by assessing the current literature on the topic, complemented by some simple modelling.
My focus is on tropical diseases, by which I mean Neglected Tropical Diseases (NTDs), plus Malaria, Tuberculosis (TB), and HIV/AIDS. I consider all types of research that aim at controlling these diseases, including basic, applied, and social science research.
I set out some reasons for thinking that this tropical disease research is neglected (underfunded relative to other disease groups), important (with the potential to avert a high economic- and disease-burden, due to the lack of current products), and tractable (with progress having been made in the past), and therefore a promising cause area. I then carry out a literature review, and an in depth analysis of the five policy papers that I found that estimate the cost-effectiveness of such research. Additionally, I apply a new cost-effectiveness calculation, using a method found in my literature review.
I then draw the evidence together, finding the literature to be biased, unreliable and uncertain in a number of ways. However, I find that variation between diseases and funding mechanisms could cause a great variation in cost-effectiveness between different programs. Therefore, whilst many types of research funding will not be cost-effective, it seems likely that some types of funding are more cost-effective than Giving What We Can’s top recommended charities, which lie in the region of $100/DALY (though these estimates are uncertain).
This shows the area is promising, but not that it merits further investigation. To be sure that it merits further investigation, we need to weigh the benefits of finding these more effective interventions against the cost of finding them. In rough calculations, I show that it is not clear whether further investigation should be carried out.
However, if further research is carried out, the focus of research should shift to finding the most cost-effective types of research in this area, which is a task that requires significant scientific expertise. The appendices investigate the most promising funding types and disease areas, and attempt to give a very rough analysis that can be the starting point for deeper investigations of these questions. I also identify a more promising type of model in the cost-effectiveness literature, which may resolve some of the problems that I identify in existing models, and which may complement the disease-specific research.
~~
Before I attempt to estimate the ‘marginal, ex ante’ ‘cost-effectiveness’ of ‘medical research’ into ‘tropical diseases’, it is worth briefly setting out precisely what I mean by those terms.
I am attempting to find the marginal impact of tropical disease research. That is, I’m trying to assess the impact of an additional dollar given to a research programme, rather than the average impact of all of the dollars that fund a research programme.
I am also attempting to find an estimate of ex ante effectiveness. That is, I’m trying to estimate the effectiveness of research looking forwards to studies that haven’t yet been completed, rather than looking backwards to assess the efficacy of studies that have already been completed (ex post).
However, average and ex post analysis is sometimes a useful stepping stone to marginal ex ante analysis, and is much more common, so I occasionally also examine average or ex post research.
In this paper, my primary purpose is to attempt to estimate the cost effectiveness of funding research into tropical diseases. This means that my primary focus is on assessing how much it costs to achieve a desired outcome. In this case, my particular interest is in estimating how many dollars it takes to avert one Disability Adjusted Life Year (DALY), a measure of health.1 It is good to avert DALYs - they represent a combination of years of life lived with ill health (morbidity) and years of life lost due to early death (mortality).
I have chosen this measure because it is widely respected, and the dominant measure in health policy analysis. This means that there is data on the DALY ‘disease burden’: the costs of various diseases in terms of DALYs. It is therefore a highly tractable measure, and one that is a useful proxy for health improvements, and likely to be linked to other valuable outcomes.2
Secondary goals in this study are to identify particularly promising diseases, means of funding, and organisations. These investigations, placed in the first three appendices, are shallower, but should provide an introduction to the key considerations.
By ‘Tropical diseases’ I mean to include two established sets of diseases: neglected tropical diseases (NTDs), and the ‘Big Three’.
NTDs are a set of diseases that are endemic in tropical, low-income countries. They are not united by any common medical features, but they tend to be:
The exact list of NTDs is contentious, but includes, for instance, Schistosomiasis, Chagas disease, and Dengue fever.4
NTDs are thought of as distinct from the ‘Big Three’ diseases: the high burden-of-disease, relatively-well funded tropical diseases of Malaria, HIV/AIDS, and Tuberculosis.5
I have included the latter alongside NTDs because they fulfill all of the criteria for NTDs set out above, with two caveats. First, they are less underfunded than NTDs, though they are still likely to be underfunded compared to other diseases. Second, they have a higher disease burden than most NTDs. Since the relative-overfunding is not too significant, and the burden of disease is high, I thought it worth including them in the analysis.6
There are a variety of control methods available for dealing with diseases. First, drugs, such as antibiotics, can treat a diseased person once they are infected, thus causing the person to be burdened with the disease for less time and, in some cases, save their life. Second, vaccines can be administered, which help to prevent cases of the disease. Third, diagnostic tools can help to identify diseases, and target treatment. Finally, ‘vector controls’ can be used to intercept the transmission of the disease between individuals. An example of a vector control intervention is the distribution of long-lasting insecticide treated bed nets (LLIN), which prevent humans being bitten by infected mosquitos, and so prevents the disease being transmitted to them.
Roughly, there are three types of research which can aid the successful deployment of drugs, vaccines, diagnostic tools, and vector controls. I mean ‘medical research’ to include any of the following types of research:
Basic research, most commonly carried out by publicly-funded laboratories and often in universities, investigates the underlying characteristics of the disease and its transmission. This understanding can then be used by more applied researchers to develop drugs, vaccines, and control measures that target the characteristics of the disease. For instance, Gardner et al (2002) sequenced the genome of one of the malaria parasites, which gave other researchers information about how to build drugs to target the organism.
Applied research uses the basic research to try to develop drugs, vaccines, and vector controls that can be deployed to reduce the disease burden. For instance the RTS,S clinical trials partnership (2015) developed the RTS,S malaria vaccine, and tested its efficacy in a clinical setting. Applied research will generally be focused on developing a drug, a vaccine, or a control product for a particular disease, whereas basic research will be focused on the characteristics of the disease, rather than a particular product type. Applied research is more likely to be carried out by the private sector. Clinical trials, which are required to test safety and efficacy before drugs can be approved for use, are predominantly funded by the private sector, since they are very costly.7
Finally, social science research studies the effects of interventions on the ground, to attempt to assess which of the existing control methods is most effective. Social science research is likely to be focused on a particular intervention: a particular means of deploying some vaccine, drug, or control method.8 For instance, Lengeler (2004) examined randomised control trials of insecticide-treated bednet distribution, to analyse what impact these distributions had.
By ‘medical research’ I mean any research aimed at combatting disease. I therefore intend to encompass basic, applied, and social science research, and research into drugs, vaccines, and vector controls, though in some cases, I focus on a particular one of those types of research, as do the studies that I cite. (In particular, I focus less on evaluating more basic research, because its impacts are remote, and so harder to evaluate.) The reason for this broad focus is that, without knowing more about the specifics of a disease, it is difficult to tell which of these types of research is likely to be most promising. What kind of research is most promising is disease specific, and I discuss later on which types of research seem most promising for a variety of diseases.9
Finally, it is worth noting two broad types of funding for research. Push funding pays for the costs of research projects. In contrast, pull funding provides prizes for research projects that successfully develop a technology. See Appendix 2 for more details.
GiveWell provide an excellent overview of how new medical technologies are developed (that is, how basic and applied research relate). Their analysis can be thought of as a more nuanced version of my basic/applied distinction, and is a useful way of understanding the stages of technology development, and how they relate.10 Please see their discussion for more information.
This section sets out the reasons for thinking that research into tropical diseases is highly cost-effective, which will complement the more empirical and literature-driven work that follows.
Before justifying my focus, I should note that it is often difficult to reliably fund only a narrow research area. Research into one area often has unexpected implications for another field. For instance, basic material science research may provide the foundation for applied medical technologies. Similarly, funding research into tropical diseases may lead to research applications in another area. It is therefore difficult, when funding research, to ensure that all of the implications of the research are focused on, say, tropical diseases, and this is especially the case with more basic research.11
Research and development is in general a promising cause because it is likely to be under-supplied by current institutions, yet have widespread benefits.
If the air is clean, anyone can breath it without depriving someone else of oxygen, and, moreover, it is difficult to stop someone else from breathing the air. These facts make clean air a public good: a good that isn’t depleted if an additional person uses it, and that one can’t effectively stop people from using.
If one could make a machine that improved the air quality, and tried to charge people for using the air, it would be impossible: since there’s no way to stop people who haven’t paid from using the air, people know that they can use the air without paying for it, so no-one pays you, and you make no money. This is an example of a feature common to public goods: because firms cannot charge for public goods, they don’t supply (as much of) them (as they otherwise would), and so society can not benefit from these public goods, and is worse off. So public goods are undersupplied by the market. The standard solution to this is for the government, or some other altruistic agent, to supply the good, thus making society better off.
Research is generally thought to be a public good: once a piece of knowledge is produced, anyone can use it without depriving someone else of the ability to use the idea; and it is difficult to stop someone else from using the idea.12 (Notable caveats to this include the existence of patents, which legally prevent the use of an idea by other parties for a certain number of years, and costly journal articles, which exclude access to knowledge based on ability to pay. Since knowledge is diffused from journal articles into lower-cost mediums, and patents expire after a certain period, it is fair to say that in the long run, research is a public good. Another caveat worth noting is that some education will likely be needed to access the idea.) Therefore, a one-off effort, in a small part of the world, can have a global, long-lasting impact. This helps to explain why the effect of research can be so huge, and why researchers are so often high up on lists of highly effective individuals: their impact can quickly, and costlessly be global.13 We would therefore expect research to be under-supplied, and this is why most governments fund research projects.
Research is also a public good on a global scale: it can easily spread around the world at no extra cost. This means that nations don’t capture the full benefit of the research they work on, since much of that benefit is experienced in other countries. Therefore, national governments will also under-supply research relative to what would be efficient globally. Global or altruistic funding is needed to ensure that research is funded to an effective level.14
Medical research gives us new treatments, vaccines, or control methods, and so (on average, at least) improves health. There have been massive increases in life expectancy since 1900 (life expectancy roughly doubled in developed countries, and tripled in low-income countries), and much of this advance can be attributed to medical research.15 Valuing the total increased quality and length of life leads to a figure of $60 trillion, and has led some to conclude that, even though it is difficult to tell precisely how much impact health research had on this figure, it must be hugely valuable.16
It is, however, possible that there are diminishing marginal returns to medical research: many of the diseases with the highest disease burden already have effective treatment methods, so research resources have increasingly been focused on diseases with lower disease-burdens, and this is likely to have led to diminishing returns. This has been exacerbated by the focus of drug companies on high-income country markets, where many high-burden diseases have been eliminated, focusing efforts on treatments for low-burden diseases, or extremely expensive treatments for the remaining high-burden diseases (like cancer). We can’t simply expect returns as high in the next century as we saw in the last.
However, many diseases still lack effective treatments, and technological progress can lead to further medical advances. More worryingly, increased resistance to drugs and vector controls (there is growing resistance to current anti-malarials, and insecticides, and there are now infections that are resistant to antibiotics) means that health research is needed in some areas just to retain the capabilities that we currently have.17 Indeed, the WHO notes drug resistance across the world, and argues that drug resistance is one of the greatest health problems we face today.18 There remains a need for research on many diseases.
A more mundane reason for focusing on health research is its compatibility with existing work by Giving What We Can and GiveWell, which allows us to prioritise health research relative to the interventions that these organisations have already investigated.[a][b]
Some of the analysis carried out by these organisations is an attempt to estimate how costly it is to avert a DALY (Disability Adjusted Life Year) with various interventions, and there are rough $/DALY estimates for most of these organisations’ recommended charities. Health research is also easily assessed in terms of $/DALY. (Though I should note, as both Giving What We Can and GiveWell do, that we are not only interested in $/DALY: we may also be interested in questions of equality, and other goods that make up a flourishing life: for instance, education, income, family, and security).19
Why does this comparability matter? Because one of the main aims of this report is to assess the cost-effectiveness of research relative to the current top global health charities. This is best done by considering a type of research that can be assessed with the same measure ($/DALY) as top global health charities.
Tropical disease research seems especially promising relative to most medical research, because research into these diseases is relatively neglected, and research seems needed, both for new treatments, and to boost economic growth.
Tropical diseases are underfunded relative to other diseases.20 This is one aspect of the ‘10/90 problem’: diseases that cause 90% of the global disease burden typically receive less than 10% of medical research funding.21 One study found that only 16 of 1,393 drugs released between 1975 and 1999 were for tropical diseases and tuberculosis.22 It is worth noting that these figures have since sparked a renewed interest in funding research into tropical diseases, with funding increasing from $2.8bn to $3.2bn between 2007 and 2014, and 26 drugs were approved in this area between 2000 and 2009, but ‘output remains low... relative to disease burden and relative to [global] diseases’.23
It appears that this is because high-income national governments, although they fund research in proportion to their national disease burden, tend to neglect tropical diseases, which carry a significant proportion of the disease burden, while governments in low-income countries do not devote much funding to health research, largely because they do not have enough revenue to devote to such projects. Therefore, research funding is skewed towards the disease burden in higher-income countries.24
It seems that the inequality in health spending is not primarily due to a difference between diseases in the difficulty of research, but rather to the national focus of spending, and global income inequality. This therefore indicates that we might have higher average returns to medical research if a higher proportion of current medical research funding were spent on tropical diseases.
However, it is also worth noting that cost-effective treatments already exist for some tropical diseases, such as Soil-Transmitted Helminths, and Schistosomiasis, and that cost-effective vector controls exist for malaria. It therefore appears that addressing these diseases is primarily an issue of access to existing medicines, rather than of developing new medicines. This indicates that some of the relative underfunding of tropical diseases may be justified, since no more research is needed, and therefore the relative neglect may have been somewhat overestimated by simply comparing disease burdens to funding levels or number of citations.
Though this under-funding may be justified in the case of diseases for which there are good existing technologies, many tropical diseases do lack adequate technologies, and thus seem plausibly genuinely underfunded.
As mentioned above, there are problems with drug resistance in malaria, vaccines are not fully efficacious against TB, there is no vaccine for HIV, and many NTDs lack proper treatments. Specific problems for a range of tropical diseases are discussed below, but it is clear that there remains a lack of technology for dealing with many aspects of tropical diseases. Therefore, it appears that new technologies are likely to have an impact, and, due to problems like drug resistance, research may even be needed to ensure continued access to types of technology that currently exist.
Limiting tropical diseases may improve economic growth in low-income countries.
It seems plausible that improvements in health would increase the rate of growth in developing countries, since it would seem to allow for more, and more efficient, hours of working and learning in each human life, and therefore makes human labour more valuable and productive.
However, in the empirical literature there is contention over the idea that health causes economic growth. Some of the literature challenges the direction of causality, and the magnitude of impacts, with some authors even claiming that improved health harms economic growth.25 The effect on economic growth is therefore very contentious and tentative, but is a positive possibility that we should account for.
Another reason to prefer tropical disease research to general medical research is that implementing any new treatments is likely to be cheaper.
Because purchasing power tends to be higher in low-income countries (that is, in general you can buy more with a dollar in a low-income country than in a high income country), the costs of distribution and treatment for any new technologies are likely to be lower in those countries. Since tropical diseases exist primarily in low-income countries, we would expect that it will be cheaper to implement new technologies that target tropical diseases.
There has been some previous research on this question from within the effective altruism community, though not much. What research there has been seems to indicate that more research is worthwhile.
GiveWell’s Open Philanthropy Project (OpenPhil) has a project assessing the impact of scientific research funding, and have argued that research focused on global poverty is plausibly a ‘neglected goal’ that they would like to investigate further (for reasons similar to those above, where I discuss how tropical diseases are underfunded). They are currently researching this area further, with the aid of scientific advisors, however their research seems to be constrained by difficulties with recruiting scientific advisors.26
I have previously investigated the same issue as part of the Global Priorities Project. This work was largely theoretical, but found some highly promising results, with returns to research in several diseases estimated as more cost-effective than current recommended charities. However, the estimate was highly uncertain, and I expressed concerns that the model may have excluded some crucial considerations. I concluded that further research would be valuable.27
The previous work on this problem from the effective altruist community therefore encourages future research. This research, empirical, but focusing on a literature review rather than conversations with scientific advisors, may be a useful complement both to the Global Priorities Project’s theoretical work, and to OpenPhil’s more applied work.
In this section, I give a brief overview of the state of research as I found it on this topic. Although I did not carry out a full systematic literature review, I found several hundred publications of some relevance to this project. In the initial part of this section, I give a brief overview of the more general literature: the papers that are most useful to my central purpose are discussed in the next section. On request, I can share all of the papers that I found with interested parties.
I did not carry out the literature review in a standard systematic manner. A systematic review would increase the likelihood that I had not missed relevant papers, and allow others to replicate my research more easily. However, I believe that my research was thorough enough to have found most of the relevant literature.
Instead my procedure was to make an initial search on Google Scholar, around key words such as ‘social returns’, ‘research’, ‘medical research’, and ‘cost-effectiveness’. I also searched on the archives and websites of relevant organisations such as the World Health Organisation, and well-known funders of research into tropical diseases (such as the International AIDS Vaccine Initiative).
I then looked in the bibliographies of papers to search for relevant papers that they cited, and checked on Google Scholar to see which papers cited them. For the most promising papers, I also checked their source websites for other relevant papers (for instance, there was a useful paper in the journal Health Research Policy and Systems, so I checked their archives for other relevant papers).
I then categorised papers according to whether they were focused on low-income disease research, medical research more generally, research more generally, or research evaluation methodology. I focused my reading efforts on tropical diseases, since that was my focus, but skimmed papers in the other sections to get a sense of the broader literature.
There are three main challenges to assessing the returns to any form of research.
One is that research produces unexpected results, since it is by nature the exploration of the unknown. Therefore it is difficult to predict how helpful the results will be, or even what field they will be in (as mentioned above). This makes research especially difficult to evaluate before the fact.
Second, even assessing research after the fact can be difficult, because it is often unclear which studies were the most important in the development of a particular idea. Studies influence each other, so that the most-cited paper may be no more crucial than the less-developed paper that preceded it, and even studies that investigate ultimately unfruitful options may be highly valuable in showing the options to be unfruitful.
Third, and related to this last point, there is a risk that qualitative studies, which take certain scientific papers as case studies, are biased towards studying apparently high-impact papers, and so may have sample selection bias.28
I found many studies that attempted to estimate the returns to research and development in the private sector in developed countries. Rather than focusing on cost-effectiveness, these studies tend to examine the private returns (the financial returns to an individual company of carrying out research), but some also estimate the social returns (by which they mean here the impact that one company’s research has on other companies).29 The Copenhagen Consensus Center has carried out a cost-benefit analysis of general research and development, and found relatively high returns, with a benefit:cost ratio of 3:1.30 There are also many government or NGO studies, as well as academic work, on how to assess and increase the returns to non-profit research funding.31
There are a number of cost-benefit analyses into health research in particular.32 There are also non-quantitative methods, such as the ‘payback framework’, which allow studies to be evaluated ex post. Another literature is on the cost of producing a new drug, whilst other studies focus on optimum funding methods.33 I couldn’t find any cost-effectiveness analyses of medical research in high income countries, nor could I find ex ante or marginal estimates of impact.34
The one exception to this is a blogpost by GiveWell, which attempts an (average, ex ante) cost-effectiveness analysis of the returns to cancer research, estimating $2,800/DALY.This estimate is significantly worse than their cost-effectiveness analysis for their current top recommended charities (which stands at around $80/DALY), but they recognise that it is likely to be an underestimate, because it fails to account for morbidity, and that better-directed research is likely to have a higher impact than the average value that they have attempted to calculate. (The estimate is also highly uncertain.)35 I draw on this analysis to carry out a similar calculation for tropical diseases later in this report.
There are many policy documents, largely from NGOs, advocating for increased investment in research & development (R&D) funding.
Lots of this work is non-quantitative, and it is therefore difficult to use it to make comparisons with other possible causes (for instance, the lack of quantification makes it difficult to compare the impact of research to the impact of the Against Malaria Foundation). However, these policy documents often include substantial detail on the appropriate funding methods, or the highest priority diseases for research (see the appendices for more detail).36
However, I also found a number of policy documents that attempted to find a quantitative cost-effectiveness estimate of research (normally a specific funding mechanism for a specific disease). These policy documents are the key studies that I use, and are discussed in the next section.
I found few peer reviewed documents that focused on research for tropical diseases, and none that were quantitative: most of the documents in this area were policy documents rather than academic papers.
Many of the existing studies, in all areas, are non-quantitative, and/or focus on issues of implementation for research funding, rather than assessing the returns to research funding. When an attempt is made to assess the returns to research funding,it generally takes the form of a cost-benefit analysis, rather than cost-effectiveness analysis (that is, they measure the benefits in some monetary form, rather than in DALYs).37 Some notable exceptions to this are discussed below. Existing studies also tend to focus on past research (ex post evaluation), rather than attempt to assess the returns to future research (ex ante evaluation). Finally, most studies attempt to estimate the average, rather than the marginal, impact of funding. 38 I could find no studies which were marginal, ex ante cost-effectiveness estimates (excepting the Global Priorities Project model, discussed below). This indicates that current research is not wholly suited to the purpose of this report.