Measuring benefits

By investing in the most effective charities, we can extend people's lives. We can improve the quality of their lives. Sometimes we can do both. But how do we measure and compare the potential benefits that charities provide, to ensure we are indeed investing in the most effective ones? We use metrics known as Health-Adjusted Life Years (HALYs), which involve the trade-off between duration and quality of health in a single numerical unit. [1] There are two types: Quality-Adjusted Life Years (QALYs) and Disability-Adjusted Life Years (DALYs). These measure health gain and loss, respectively,[2] and as such, may very broadly be considered inverse metrics.[3] Quality-adjusted life years (QALYs) have been used in the assessment of health interventions for three decades[4] as an aid to resource allocation.[5] Although the measure is well-established, the assumptions underlying it are ever debated.[6]


What is a QALY and how is it calculated?


Individuals move through health states over time; these consist of varying levels of functioning within a set of health domains, which may include mobility, cognition, pain, emotional functioning, self-care, and so on.[7] The domains considered are contingent on the instrument used to measure the health state, such as the EuroQol five dimensions (EQ5D) or the Health Utilities Index.[8] Each health state is assigned a certain value (or utility) which is determined by assessing the preferences of a group of individuals for being in any one state (using methods like the standard gamble or time trade-off). A value of zero represents death, while a value of one represents full health. It follows that any state in between (of less than perfect health, e.g. living in a wheelchair) is also assigned a value in between.[4]


A QALY score is obtained by multiplying (or weighting) the amount of time spent in a health state by the utility score assigned to that health state.[4] For example, it takes one additional year of perfect health (utility score of one) to obtain one QALY. It also takes four additional years in a health state valued at 0.25 to obtain one QALY. Again, if the quality of an individual's life is improved by 0.25 over four years, a QALY is gained.


Now, what is a DALY?


The QALY framework provided a basis for the development of the disability-adjusted life year (DALY) in the early 1990s.[4] The DALY tends to be used in the developing world, and has been widely adopted by the World Health Organization. Members of Giving What We Can also use DALYs, and have been involved in designing recent adjustments to the metric.[9]
The DALY is primarily a measure of disease burden, and thus relates to an entire disease (rather than a health state, as with the QALY). This burden may be a reduction in full health (such as suffering from malaria, for instance), or even premature death. The DALY thus represents a loss of functioning, and can be thought of as one life year in full health lost.[2]

To determine the number of years lost due to a disease, a weight factor that reflects the severity of the disease on a scale from zero (perfect health) to one (death) is assigned. This is the inverse to the QALY.

  • When calculating the number of years lost due to premature death, the life expectancy of the afflicted individual would need to be known. As this is not possible, a typical life expectancy is predetermined. Giving What We Can uses a life expectancy of 60 years, which is the typical life-expectancy at birth for the developing countries in which our charities operate.
  • When calculating the number of years lost due to disability, the severity of the disease state is presented as a percentage of the burden of premature death. For instance, a year of blindness would be considered 60% as burdensome as a year lost.[10]


As with the valuation of health states for QALYs, these weightings may be determined from preference-based studies, or expert valuations. However, controversy exists around the use of these methods to decide that, for instance, a year of blindness should be counted as 60% as burdensome as a year lost, whereas a year of being severely underweight due to malnutrition would be considered 5.3% as burdensome. The DALY metric also allows for optional age weighting and discounting features, which are based on moral assumptions, and as such, are also liable to being debated. The Disability-Adjusted Life Years series by Holden from GiveWell describes these controversies in detail.[10]


QALYs and DALYs face criticisms, which stem from the theoretical assumptions and subjectivity underlying them. Albeit used in different settings, both are measures that combine mortality with morbidity in single numerical units. Some argue that mortality and morbidity are different dimensions that cannot be combined, whereas others advocate the necessity of such criteria to enable resource allocation in a consistent manner, across health states and diseases.[1] As such, the use of QALYs and DALYs, although relatively widespread, remains controversial, and alternative measures continue to be developed.




References


[1] Robberstad B. QALYs vs DALYs vs LYs gained: What are the differences, and what difference do they make for health care priority setting? Norsk Epidemiologi 2005; 15 (2): 183-191
[2] World Health Organization. The Global Burden of Disease concept. http://www.who.int/quantifying_ehimpacts/publications/en/9241546204chap3.pdf [Accessed December 2013]
[3] Gold MR. et al. HALYS AND QALYS AND DALYS, OH MY: Similarities and Differences in Summary Measures of Population Health. Annu Rev Public Health 2002; 23: 115-134
[4] Sassi F. Calculating QALYs, comparing QALY and DALY calculations. Health Policy Plan. 2006 Sep;21(5):402-8. Epub 2006 Jul 28.
[5] National Institute for Health and Care Excellence. Measuring effectiveness and cost effectiveness: the QALY. http://www.nice.org.uk/newsroom/features/measuringeffectivenessandcosteffectivenesstheqaly.jsp [Accessed December 2013]
[6] Seventh Framework Programme. European Assessment Of The Validity Of The Qaly Outcome Measure: The Qaly Controversy: Key Results From The Experiment Conducted By The Echoutcome Project, January 2013 http://www.reglinkassociates.com/Articles/echoutcome0113.pdf [Accessed December 2013]
[7] Chatterji S. et al. Global Programme on Evidence for Health Policy Discussion Paper No. 45: The conceptual basis for measuring and reporting on health. 2002. http://www.who.int/healthinfo/paper45.pdf [Accessed December 2013]
[8] Whitehead SJ. et al. Health outcomes in economic evaluation: the QALY and utilities. Br Med Bull 2010; 96 (1): 5-21. doi: 10.1093/bmb/ldq033
[9] GWWC members Nick Beckstead, Nir Eyal, Toby Ord, and Thomas Pogge were amongst philosophers asked to advise on ethical aspects of the DALY measure at the recent Critical Ethical Choices for DALYs meeting, convened by the World Health Organisation and World Bank in preparation for publication of the 2010 Global Burden of Disease Report.}
[10] Holden. The GiveWell Blog. Series: Disability-Adjusted Life Years. http://blog.givewell.org/2008/08/07/disability-adjusted-life-years-introduction/ [Accessed December 2013]