Our impact
We think it is important that people choose where to donate based on how they can help others the most. Therefore, in order to know whether Giving What We Can is a good target for donations, we need to know the impact of our activities.
To help people to donate to the charities they believe are the most effective, we’ve described several ways that we can quantify our impact.
About our impact calculations
(and some caveats)
These calculations examine the impact of Giving What We Can’s work up to the end of 2014, and compare this to our costs. This yields several estimates of our average impact per dollar spent.
However, while we monitor our impact relative to our spending as a broad sanity check of our effectiveness, these estimates should not be overemphasized, or taken as projections of our future leveraging ratio on donations.
Our realistic scenario impact calculation suggests that we have created value equivalent to moving $104 to top charities which otherwise would not have been donated, per dollar of spending up to the end of 2014; in effect giving us a leveraging ratio of 104:1 for those years.
We expect this ratio to decrease from 2015 onwards, as our costs grow. We do not see this as a problem, since we predict that increasing our costs is necessary for increasing our long-run, absolute impact.
We would therefore encourage potential donors to look at our growth (in terms of both members and money moved so far), and our plans for how we intend to use future donations, and weigh these more heavily than these calculations when making a judgement about whether Giving What We Can is a worthwhile cause to fund.
These calculations build upon the impact evaluation we released earlier in 2015. They update the calculations with previously-unavailable data from 2014, and modify some of the calculations’ assumptions based on this new information.
Lower bound (6:1)
The lower bound calculation uses conservative or pessimistic assumptions, and considers only past donations (in effect assuming that members all stopped giving after 2014). This calculation shows that even if all our members stopped giving now — rather than fulfilling their lifelong pledge — we move around $6 to top charities for every $1 in costs. Therefore, we think that this represents an absolute worst-case scenario for our impact.
We can be confident that for every $1 spent by Giving What We Can, at least $6 will be moved to top charities.
Realistic (104:1)
The realistic scenario uses detailed information about what we expect members to pledge into the future, and provides what we consider to be the most realistic estimate of our effectiveness.
Information for 2009-2014 indicates we should expect that, for every $1 spent by Giving What We Can, around $104 (counterfactually adjusted and time-discounted) will be moved to top charities.
Methodology
Lower bound calculation (6:1)
This calculation is a conservative estimate of Giving What We Can’s impact. It only takes into account impact that has already happened and was concretely measured. In this calculation, we only count money that has already been moved to effective charities, and would not have been moved without us. It takes into account the years 2009-2014, which are years for which we have the most complete data. It considers only money that was actually given up to the end of 2014, rather than using donations made in 2009-2014 to make assumptions about future donations, and only includes money which went to the most effective charities and would otherwise not have been donated at all. It then compares that to our costs up to the end of 2014.
Summary of the calculation
Giving What We Can’s costs up to the end of 2014 were $515,000.
The total for donations to top charities by members and try givers which would otherwise not have been donated which we know of was $1,693,000
From information provided to us by one large donor, we learned about an additional $1,346,000 which was donated to top charities, and would not have been donated if not for Giving What We Can.
Adding these numbers gives the total donations to top charities attributable Giving What We Can in 2009-2014: $3,039,000.
This means that for every $1 donated to Giving What We Can over 2009-2014, $6 was donated to top charities. This excludes future donations from members who joined over the period, donations to other charities and impact from causing people who already planned to donate money to donate it to more effective charities.
Costs
Giving What We Can’s total monetary costs up to the end of 2014 were £208,000. Up to the middle of 2012, Giving What We Can was run entirely by volunteers. For the previous impact evaluation (up to 2012), we worked out the number of hours spent by the volunteers on running it, which gave 11,865. Costing volunteer hours at £10 per hour (indicative of salary costs were these people paid staff)1 , this volunteer time was worth £118,650. We estimate we have benefitted from an additional 572 hours of volunteer time between 2013-2014, which was worth £5,720. Therefore the total costs up to the end of 2014 were £332,000, or $515,000.
Total donations from our community
Until 2014 we used yearly surveys to ascertain how much members donated, and their incomes. For those members who joined before the end of 2013, 67% told us their income and donations for one or more years in which they were members. Member donations recorded by this survey up to the end of 2013 had totalled $3,994,0002. In 2014 we introduced the ‘My Giving’ section to our website, allowing members to report their donations directly. We also moved across data from previous My Giving surveys to member dashboards. The donations recorded in My Giving up to the end of 2014 totalled $7,064,000.
Try Givers (non-members who commit to donating regularly for a specified time) reported donating $11,000 over 2013 and $31,000 over 2014, giving a total of $42,000. We didn’t advertise this option widely until 2014, hence the small number of participants.
We set up a charitable trust which launched at the start of 2014. The trust collects and regrants donations to top charities. Of the $681,000 which moved through it in 2014, $67,000 was donated by members who did not record any donations to these charities in their My Giving that year.
Adding these together, the total donated by the end of 2014 was $7,173,000.
Donations to top charities only
While we think that increasing overall giving is in itself positive, we’re predominantly interested in money we move to top charities. Of the total $7,173,000 reported up to the end of 2014, $3,853,000 was donated to non-top charities. Subtracting this money from the total leaves $3,320,000 donated to top charities.
Donations that would have been made anyway
When they join, we ask all our members what percentage of their income they think they would have donated if they hadn’t joined Giving What We Can3 . This allows us to estimate how much of the total donated would have been donated anyway.
To work out how much money Giving What We Can caused to be donated, we divided the difference between the pledge percentage and the percentage members would have given anyway by the pledge percentage for each member, and then averaged over these:4
(pledge percentage — counterfactual percentage) ÷ pledge percentage
This tells us that, on average, 51 per cent of members’ donations would not have been made were it not for Giving What We Can5.
Multiplying the $3,320,000 donated to top charities by 51 per cent means that the amount donated to top charities which would not otherwise have been donated is $1,693,000.
Additional tracked donations
A large donor made several significant donations which were influenced by Giving What We Can. Of their combined donations, $1,346,000 was donated to top charities6. Adding this to the previous figure gives $3,039,200 counterfactually donated to top charities.
Putting it all together
Given our costs for the period ($515,000), this means the lower bound for our leverage ratio (the number of dollars which we caused to go to top charities for every dollar donated to us) over the period 2009 - 2014 was approximately 6:1.
Flowchart
Limitations of this calculation
This calculation does not include:
- any impact we had on non-members apart from the one large donor mentioned who joined after 2013
- any impact from causing people to give to more effective charities rather than the ones they otherwise would have
- members who did not tell us about their giving
- any impact by non-top charities
- any donations made in the future
Consequently, it is likely to underestimate of Giving What We Can’s actual impact by a considerable margin.
To try to get a more realistic estimate of our effectiveness, the next section outlines a more sophisticated calculation which takes some of these factors into account.
Realistic impact calculation (104:1)
This calculation aims to provide a realistic estimate of our actual impact. The previous calculation gave a very conservative lower bound for our effectiveness — it simply asked how much our members had already reported they had donated to our recommended charities which they wouldn’t have otherwise donated, compared to how much our costs had been.
By contrast, this calculation aims to provide our best guess for how much money will be donated to top charities from our members for each dollar spent by Giving What We Can. The assumptions are therefore intended to be realistic, not pessimistic, although we believe that they are still based on relatively conservative assumptions.
Summary
In this calculation, we start with the total amount of money members who joined between 2009 and 2014 pledged to donate to the most effective charities.
We then adjust to account for people leaving or ceasing giving, donating a different amount than they pledged and a discount rate.
We then calculate our impact from a number of different sources, in each case calculated in terms of ‘top-charity equivalent’ dollars.
The donations we would expect to go to top charities which would not be donated if it hadn’t been for Giving What We Can total $26 million.
Donations which we would expect to go to non-top charities, which would otherwise not be donated total $15 million, when translated to top-charity equivalent dollars. The value of donations by members donating to more effective charities than they would have otherwise totals $18 million.
This means that the total value donated to top charities by the members we recruited between 2009 and 2014 is $59 million, time discounted and adjusted for donations that would have been made anyway.
This equates to $73,000 per member, and suggests that our leverage ratio is around 104:1 – that is, for every $1 spent by Giving What We Can, around $103 will be donated to top charities.
Costs
Total costs for Giving What We Can over the 2009-2014 period were $515,000. This includes actual expenses, and volunteer time costed as if it were paid as salary.
It is plausible that, in order for the rate of membership attrition to remain at its current level (see below), we need to invest time in the future into activities promoting member retention.
It is likely that some of Giving What We Can’s general activities will contribute towards member retention (Giving What We Can continuing to grow might keep members feeling enthusiastic about the organisation for example). However, it is also likely that other activities would be explicitly required to retain these members (e.g. events, newsletters, support etc).
We estimate that for the 802 members in these cohorts, the work involved would be the equivalent of around one full week of work per year for a member of staff (which costs us in the region of $1,200) over the next 40 years. This puts our total costs for these cohorts at $563,000.
Donations pledged by members
The amount of money pledged by members who joined up to the end of 2014 was $344,310,000. Members pledge to donate a proportion of their income (at least 10 per cent) for the rest of their working lives. They provide us with information about how much they expect their average future salary to be, and their current age. From the latter we work out how many working years they are likely to have left, and therefore their total expected income over their working lives. We then multiply this by the proportion of their income they have pledged to donate (in most cases, 10 per cent) to derive the total donations we expect them to make in the future.
This methodology relies on the accuracy of members’ predictions about their future income. In general we have found that these predictions seem conservative, as most people underestimate their future earning potential7 . If members do not estimate their future salary, we use the median salary for their country. We think that is a fairly pessimistic assumption, as our median member has an expected earning potential higher than the median wage8 .
Accounting for membership attrition
Some members leave Giving What We Can, and therefore can be assumed not to actually donate the money they pledged. Others we lose contact with, so that we don’t know whether they donate the money they pledged. The rate of people leaving has so far been 1.7% of members per year.9
Other people lose contact with Giving What We Can. The rate of people going silent has been 4.7% per year (we have counted people as silent if we haven’t had any contact with them for over 2 years) . It seems likely that members who go silent still donate some amount, but it is likely to be less than the amount they pledged. We have assumed that this will be around one-third of their original pledge (for example, if a person pledging the standard 10% of their income has gone silent, we’ve only counted 3.33% of their pledge in this calculation).
Given these numbers, the total of those ceasing donations per year is 4.8%.10 We’ve assumed that this percentage will remain constant over time. This means that after, say, 30 years, each member has a 23% chance of still donating, which we believe is a plausible estimate.
We have made the simplifying assumption that the money pledged will be donated in equal parts over the next 40 years (the median expected working life of members as at the end of 2014). If no members ceased donating, that assumption would mean that around $3,652,997 was donated each year. The assumption that the money will be donated in equal parts seems optimistic, because people’s incomes typically increase with age. The assumption that the percentage of people ceasing donations is a constant percentage, on the other hand, seems pessimistic, as it seems likely that members would either drop off within the first few years, or remain engaged long-term.
Making these assumptions, the total donations we would expect to be made would be $146,855,273.
Discounting future donations
We discount the effective value of donations over time, to account for the diminished value of a donation made in the future, versus a donation made right now. This assumes that there is some loss of value caused by delaying an intervention; in essence, if people donate now, the people affected by the interventions they fund will be alive and making ongoing and valuable contributions to their community, and this is likely to have a compounding positive effect into the future. By contrast, these effects will not be compounding during the period between the present and the date of a future donation, and therefore the total future positive effects will be lower.
The UK Treasury Green Book, used for appraisal and evaluation in the central UK government, recommends using a discount rate of 3.5% per annum to find the present values of benefits and harms accrued in the future.
Using the Green Book rate to discount future donations, we expect future donations to have a present value of $93,973,503.
See below for an explanation of our choice of discount rate
Accounting for members donating a different amount than they pledged
The average pledge percentage for our members is 11.03%. The average percentage of incomes people actually donate is 12.92%. Using the ratio between these, given the amount of money pledged, we would expect to be donated is $110,075,944.
The percentage of income that people actually donated comes from self-reported data contained in our membership records. We considered that the actual average might be lower; however, we cross-checked figures attached to members whose average donation percentage was more than 15% of their income up to the end of 2013 by contacting these members directly, and we revised any for which received updates. By only revising figures for higher-donating members, the lower-donating members are effectively weighted higher, and therefore bring the average down. Additionally, the people who give a greater percentage than their pledge percentage are disproportionately people with larger-than-average salaries (as you might expect), therefore the actual weighted average could be a bit higher than the one we use in this calculation. We therefore expect this number to be fairly accurate.
How much of this would have been given anyway?
Some of the money given by our members would have been given to charity anyway. To work out this figure, we ask people when they become members what percentage of their income they would have given to charity if it had not been for Giving What We Can.11
To work out how much money Giving What We Can caused to be donated, we divided the difference between the pledge percentage and the percentage members would have given anyway by the pledge percentage for each member, and then averaged over these:12
pledge percentage — counterfactual percentage ÷ pledge percentage ( )
This tells us that, on average, 51 per cent of members’ donations would not have been made were it not for Giving What We Can. Multiplying through by that figure means that the amount we would expect to be given by members which would not otherwise be donated is $56,138,731 (meaning $53,937,213 would have been donated to charity anyway).
What charities do the donations go to?
The purpose of this calculation is to find our impact in terms of the dollars we cause to be donated to top charities. This impact could come from two main sources. Firstly, from members donating to top charities when they would otherwise not have donated at all. Secondly, from Giving What We Can causing members to choose to donate to more effective charities than they would otherwise have donated to.
To take this into account, we have estimated the amount of money given to top charities which would otherwise not have been given to charity at all. We have also estimated the amount of money which will be donated to more effective charities (weighting to account for the fact that the money would have been donated anyway, but to a less effective charity). Combining these figures gives the amount Giving What We Can caused to be moved to top charities, in 2014 top-charity equivalent dollars.
Value of donations that would not have been made otherwise
The fraction of money donated by members to top charities (versus all other charities) for the years 2009-2014 was 46%13.
46% of the $56,138,731 in expected donations that otherwise would not have been donated to charity is $25,823,816.14
The remaining 54% of our expected donations ($30,314,915) is going to charities which we consider to be less effective than a ‘top charity’. We still expect them to be reasonably effective – they tend to be well-respected developing-world charities. We have multiplied these donations by a factor which reflects how much less effective we think those charities are than our recommended ones.
We have assumed that a non-top charity is around 50% as effective as a top charity. Using that factor to weight the value of donations expected to go to non-recommended charities, gives $15,157,458 in top-charity-equivalent dollars.
Adding these figures, the total future value of donations that would not have been donated otherwise is $40,981,274.
Value of donations that would have been made anyway
We also need to know the impact we could expect in terms of affecting where people donate in cases where they would have given to charity anyway, but would have given to less effective charities than top charities.
As above, $53,937,213 of expected donations would have been made anyway.
To determine that, we need to know the fraction of people for whom we affected where they gave. Information is limited, but in 2014 we put a question on our pledge form asking whether we influenced where people gave. The percentage of people who said that Giving What We Can affected where they give is 63%. Assuming this information is broadly representative of all our members, the amount of money for which we expect to affect the destination charity is $35,598,560.
As this money would have been given to charity anyway, Giving What We Can’s impact lies in the difference in effectiveness between where the money would have been donated, and where it was actually donated. As above, we have assumed that the value of the charities people would have donated to is about 50% as effective as the ones we influence them to actually donate to. Therefore the expected value of changing the locations of people’s donations, in top-charity donation equivalent dollars, is $17,799,280.
Adding it all together
Overall, the total value we might expect the members in the 2009-2014 cohorts to have, in counter-factually adjusted time-discounted dollars to top charities equivalent is $58,780,554.
This is approximately equivalent to $73,000 per member who joined over that period (there are 802 members who joined by the end of 2014).
Our impact
We can use the total amount we expect our 2009-2014 members to donate to find our total impact – that is, how much we might expect our members to donate to top charities for each dollar we receive.
As above, costs up to the end of 2014 were $563,000.
Dividing the total amount we expect to be donated ($58,780,554) by our costs produces a leverage ratio of 104:1. That is, for every dollar we spend, we expect approximately $104 extra to be donated to effective charities (in present-day, top-charity adjusted dollars).
Flowchart
Spreadsheet
We've provided a Google Docs spreadsheet which performs the calculation above, which you can use to test different assumptions.
To edit the spreadsheet, click the Google Docs 'File' menu and choose 'Save A Copy' (you will need to be signed into your Google account). You can also choose to download the spreadsheet in Excel (.xls) format, which does not require you to be signed in.
Open spreadsheet (Google Docs)
Limitations of this calculation
This analysis does not take into account any impact we have aside from the money we move to our recommended charities by getting people to join Giving What We Can (for example, people who are persuaded by the case we make for effective giving and who donate to our recommended charities, but have not chosen to become members). We believe that this means our estimate is likely to be lower than reality.
Choice of discount rate
The Green Book discount rate incorporates a number of factors.15 One of these is a pure time discount, which we consider to be a dubious method of discounting future value; however, this is only a small component of the rate (less than 0.5 percentage points). The rate primarily tries to capture the effects of potential future catastrophes which would prevent the benefits of a policy (or in our case, a poverty-reducing intervention) from accruing, and the effects of economic growth combined with diminishing marginal utility of resources.
There are many considerations bearing on what the correct discount rate to use is. You might think that interventions such as deworming have strongly compounding benefits to a society, and therefore that the discount rate should be very high. However, as compounding benefits do not get reinvested into the same interventions, we cannot assume that social benefits necessarily lead us to conclude we should apply a very high discount rate.
Another way we might choose a discount rate is by looking at the savings rate as a proxy for the social return that we would lose out on by donating later. In other words, if people pledge to give $100 in the future, what present value of money would be required to generate that future $100 if invested now. The rate of return you could get on investments depends on the risk you’re willing to bear, but a 3.5% discount rate is higher than the average UK savings rate of the last decade, and around what you could expect to achieve at the moment with a balanced investment portfolio.
Previous impact assessments
Our Summer 2015 Fundraising Prospectus contains a previous version of these impact assessments, which used a similar methodology, and covered the period 2009-2013.
The estimates it yielded were:
- Lower bound: 6:1
- Realistic: 60:1
- Marginal/optimistic: 90-300:1
We have since removed the marginal calculation, which uses assumptions that we now believe need further testing.