Giving What We Can's blog, featuring the latest news from our community.

Stay up to date with Giving What We Can's blog. Featuring the latest news about our community, how to donate effectively, charity evaluation, community events and more!

Blog

footerNavigation

About us

Contact us

Our team

Our members

Our events

Our mistakes

Donate to Giving What We Can

Transparency

About

Our best charities

Take a giving pledge

Get involved

Make a bequest

Videos, books, podcasts

FAQs

Sitemap

Learn more

Press kit

Media coverage

Case studies: People who pledge to give

Media

Footer Navigation

topNavigation

Recommended charities

High-impact causes

Why give to funds

How much to donate

Why donate publicly

Recommendations

Comparing charities

Choosing a cause

Inclusion criteria

Trusted evaluators

Our approach

Videos, books, podcasts and essays

Myths and misconceptions

How rich am I?

Can money buy happiness?

Resources

Research

Team

The Pledge

Trial Pledge

Further Pledge

Bequests

Why pledge

Pledges

Newsletter

Podcast

Videos

Sharing our ideas

Events

Giving games

Charity elections

Pledge drive

Careers

Community

Donate now

Best charities

Manage my donations

Tax deductibility

Donate

My pledge

My donations

My income

My impact

Pledge

My giving

Top Navigation

Carolina Flores

Based on the progress on maintaining political momentum and increasing funding for this sector, the rating attributed to this area is good. In 2011, the G8 contributed $4.7 billion to water and sanitation programmes and increased the number of partnerships with developing countries. This represents a 16% increase in aid compared to 2010, and ¾ of the total OECD donations for this type on intervention.

But big problems remain to be solved, and much needs to be done for the 2.5 billion people-nearly half the population in developing regions, and over a third of the world's population - who do not have access to improved sanitation facilities, and in particular for the 1.1 billion (15% of the global population) who have no sanitation facilities at all.

The Overseas Development Institute claims that the poor progress towards the Millennium Development Goal shows that the G8 have paid little attention to this area. DevEx stresses that the majority of the money in this area seems to go into building large infrastructure, and not into basic systems; and on countries which are political allies and strategic partners of the G8, instead of on the countries in the most need.

G8: KEEPING PROMISES?-Water and Sanitation 

Stephanie Crampin

On Tuesday 22nd October 2012, Giving What We can researchers Rob Wiblin (CEO of the Centre for Effective Altruism) and Chloe Kane (Giving What We Can researcher) took part in a Skype conversation with Alix Zwane (Executive Director of Evidence Action: an organisation pioneering 

, as well as Deworm the World, another of our recommended charities), and Karen Levy (Evidence Action’s Regional Director). You can see the highlights of the conversation below.

Dispensers for Safe Water is a charity that Chloe and the research team investigated as part of a larger project, the report of which can be read 

. They found that the new programme looked very promising, and of a comparable level of effectiveness to Against Malaria Foundation, and our other top charities. We took part in this Skype call to find out more about what Dispensers for Safe Water do and to answer questions such as how much funding they need, what they measure and why when recording their success, and more general information about why the dispensers programme is favourable to other WASH-related interventions such as promoting handwashing.

We were impressed by what we heard, and learned a lot, including the following key facts: Dispensers for Safe Water has a goal of "reaching twenty five million people by the end of 2018. The intention of the programme is to become sustainable from carbon revenue by this time.” This is because they are "taking advantage of the increased cost-effectiveness that comes from operating at scale. So every additional resource has more and more leverage, towards that goal." They track that the dispensers working, and full of chlorine. While lacking specific “health evidence on dispensers, they have health evidence on chlorine, and are currently measuring adoption. The two of those things together give meaningful information” which is used to calculate the programme’s effectiveness, and which has so far provided very promising results.

Evidence Action is a new organisation, and expects to have an 

 and donations system by early November 2013. Giving What We Can will report further on their work, and expects GiveWell to do so as well, so look out for more details!

You can download a full transcript of the interview 

Video interview with Evidence-Action

Christopher Hogg

Giving What We Can’s mission statement, staring at me from the wall of our office as I write this, is to “move money to the world’s most effective charities”. Working out what these are though can be tricky, as us at the research team have found out over the past week in delving into the literature of schemes involving 

Handwashing promotion programs (teaching people the importance of using soap properly) compare very favourably to our recommended interventions (in terms of $/

There is not much data relating to the success of WASH schemes, and the methodology is weak, hence at present we will not recommend charities to the same extent as our 

, soap manufacturers are in a position, unique to this sector, to fund handwashing campaigns in a manner that benefits both themselves (in profits) and society (in increasing health).

Washing your hands can make you both happy and healthy.

In most instances diarrhoea is caused by consuming bacterial contaminants, often through fingers or food. Washing hands 

 at important periods of the day (such as after defecation and before eating) removes these contaminants from the surface of the skin, cutting the risk of diarrhoeal disease by 40-50% (Curtis and Cairncross, 2003).

However, rates of handwashing with soap (HWWS) are low. This is not caused by a lack of (clean) water, or indeed a lack of soap. Rather, the soap available is used to wash clothes and not hands (Aunger et al, 2010). Therefore what is required prevent the disease are education schemes that encourage behavioural and attitude changes towards soap.

Tip-toeing into the Pool of Private Public Partnerships...

Traditionally donor funded NGO programmes, such as Charity Water and Water Aid, provide these schemes, often in combination with infrastructural improvements. The (too) few studies that exist on standalone schemes have been shown in 

I wish to stress that private companies which have both a financial and PR incentive, can implement schemes which have the same, if not better results. Unilever is one of these companies, whose Lifebuoy soap brand has the potential to be well publicised and used across the world.

It worked in the past. Could similar adverts eradicate diarrhea in the developing world?

The Lifebuoy soap brand began in Victorian England and is known in North America for its adverts which 

. Its focus is now on the developing world where it is one of Unilever’s top selling products, bought from markets in Ghana to 

, the brand states its main aim: “to change the hygiene behaviour of one billion consumers across Asia, Africa and Latin America, by promoting the benefits of handwashing with soap at key occasions”. At present, no NGO lists this as a single aim. Though we are cautious, given that too often big businesses say one thing, but do another (for example 

Coca Cola’ climate change campaign in contradiction to its relation

), there are two reasons to be optimistic about their strategies in relation to the wider WASH sector.

First, their focus on cost-effectiveness. Giving What We Can’s take on the charity sector is to treat donations in a similar way to business investment, a 

 but which we wholeheartedly defend. Just like its advertising campaigns, the promotion schemes are designed to get the largest impact for the money the company invests in. It makes sense in a competitive market. When Unilever saw that by working with governments, local NGOs and schools to “train the trainers”, it saved the company funds but delivered equal results in Indonesia, it sought to extend the scheme to other countries.

Second, Lifebuoy appears to collect good evidence on the impact of its schemes. This kind of self evaluation is what we want to see 

. The Lifebuoy brand has conducted its own Randomised Control Trial in Mumbai, keeps sales data on its soaps through consumer surveys, and are even developing three dimensional sensors in some of their soaps to monitor their use, a method which, due to its unobtrusive nature, is perhaps more accurate than the current norm of participant observation (Vindigni et al, 2011).

It will be good to see more transparency in Unilever’s data. Their website does not show, or may not have, these evidence reports mentioned above. If Giving What We Can are going to support schemes such as this, being able to see the data and analysing it for ourselves is crucial. Perhaps this is where the private sector collides with the public sector, as companies are notoriously secretive about releasing knowledge which may provide a competitor a route into the market.

We would also like to see the company investing more money on these schemes. Unilever funds are diverted away from the Lifebuoy schemes to other brands, which add less health benefits to the world. There is a growing market for soap in Africa and Asia, and potential for lots of money to be made, although the risks at present seem to be too great for the soap companies to invest.

Money for (new) soap? Large companies must be persuaded that there is large enough demand for soap in developing countries such as India to make investment worthwhile.

An advocacy organisation could therefore be needed who can show and persuade private companies to divert funds to the developing world. Currently I do not know of any group which does this, but please 

 or add a comment below. There is an urgent need for more hygiene promotion schemes, and with the funds, knowledge and incentive of private companies, these can be successfully run without donations. Hopefully I have shown you that this change can be realistically achieved surprisingly soon. Part of the solution of moving money to the world’s most effective charities may well be putting more power to the world’s most effective companies.

Aunger et al, 2010: “Three kinds of psychological determinants for hand-washing behaviour in Kenya” Social Science & Medicine 70 (pp 383-391).

Curtis and Cairncross, 2003: Effect of washing hands with soap on diarrhea risk in the community: a systematic review. Lancet Infect Dis 3(5): 275–81.

DCP2 Report. Cairncross, Valdmanis (2006) “Water supply, sanitation, and hygiene promotion” In: Disease Control Priorities in Developing Countries (2nd Edition). New York: Oxford University Press. 771-792

DfID, 2013: “Water, Sanitation and Hygiene Evidence paper” Department for International Development May 2013.

Vindigni et al, 2011: “Systematic review: handwashing behaviour in low- to middle-income countries: outcome measures and behaviour maintenance” Tropical Medicine and International Health 16(4) pp 466-477

Is private soap squeaky clean? A look at the potential effectiveness of corporate handwashing promotion

Michelle Hutchinson

Giving What We Can aims to review its progress and plans every six months, and to publish the conclusions on the blog. In addition to the usual review of the previous 6 months of and plan for the following 6 months, in this case one of the trustees of the Centre for Effective Altruism (Nick Beckstead) produced a review of Giving What We Can as a project. This blog post and the following one contain the headline conclusions of these and links to them.

The first six months of the year have been an important period of learning for Giving What We Can, and have indicated that we should shift our focus more towards individual discussions of charity effectiveness and donation, away from mass methods of communication. GWWC has continued to grow steadily, and expected money moved to effective charities seems to be well worth the money invested.

Giving What We Can’s mission is moving money to the world’s most effective charities. Each of our members commits to donating at least 10% of their income to the most effective poverty relief charities. Therefore, the key metric we use to measure our progress is the number of members we have. In the first half of this year we gained slightly fewer new members than we did in the preceding 6 months, but about the same number as we had in the 6 months before that; this may indicate that the December giving season has a noticeable effect on our membership numbers. This was steady growth for the period, but meant that we ended it on 329 members, which was slightly below our goal.

We set an ambitious target for this period because it included the second half of a press push we had organised in December, which we were hoping would increase member growth. Having updated the website with an improved look and a blog with meant readers could be kept up to date with new research developments, our aim was to publicise the organisation and website as widely as possible to get members. The media campaign was a big success in terms of coverage, with articles on Giving What We Can in the Sunday Times and the New Statesman and on Sky, BBC Breakfast and NPR, among others. This coverage did not lead to as many new members as we had hoped.

Our best guess for why membership growth following the media push wasn’t higher is that taking a pledge to donate 10% of your income is a big step for someone to take simply from reading a media article. This would indicate that we ought to take a more personal approach, and that it would be good to have a more gradual way for people to get involved. The most important lesson we drew from this was that we needed to learn more about the process which leads to people joining Giving What We Can.

In order to learn more about that process we interviewed members, and did a survey with interview follow-up of people who weren’t previously involved with Giving What We Can. Two findings were particularly interesting. One was that people who heard about Giving What We Can through the media had typically not needed much persuading to join, indicating that media pieces (at least of the fairly brief form we got) didn’t tend to have much persuasive force. The other was that people who weren’t interested in joining Giving What We Can became so after having an in-depth conversation about it. These indicate that in order to make more impact on people’s behaviour, we should focus more on having in-depth and ongoing conversations with people. For more details, our full six month review is 

Over this period, we worked both on first order research into finding effective charities, and on more methodological questions, such as what the best ways are to find great giving opportunities, whether it’s better for people to donate now or invest to give later and what the appropriate age-weighting for DALYs is.

With regard to methodological questions, we produced research papers on the topics we set out to, but they generated less interest than we had expected. Investigating individual charities was reasonably successful. Our individual charity research resulted in finding a charity which we consider to be an opportunity for very high leverage, Project Healthy Children. We also wrote reports on a few other charities which seemed promising, though we did not end up recommending them. However, this research required a large time input, and in order to find a charity which is in expectation more effective than AMF (for example) and for which our estimate is equally reliable and exact would take far more resources, since charities in areas which are effective and have low uncertainty are more likely already to have been found. Overall, we think that at the moment it would be more effective for Giving What We Can to focus on driving donations to charities we know to be cost-effective than on trying to find better charities. Therefore, over the next six months we will be concentrating on outreach. In order to achieve that, we will produce blog content which aims to interest people in the idea of cost-effective giving and to drive traffic to the website.

performance review of Giving What We Can for 2011 and 2012

 estimated the return from gaining one member as being $17,000 to the most effective charities, counter-factually adjusted and time discounted. There was a very large range on this figure. Over 2012, the year in which Giving What We Can took on staff, the cost per member was calculated to be $260. The review contains the conclusions from the 2011 giving survey (the results from the 2012 survey weren’t all in at the time the review was written). Of the over 60% of members who had filled in the survey, 90% had kept their pledge. Approximately 70% of members donated to recommended charities. One question we were particularly interested in was whether members who had joined as students continued to give when they entered employment, since many of our original members were students who now have jobs. Of people in that category, just over 70% responded to the survey, and of those nearly all had kept their pledge. This is a very good sign. Although this obviously does not mean they will definitely continue giving for their whole career, it seems less likely that a person will cease giving once they have begun than that they would fail to start.

The review contained various useful lessons. One was the importance of increased data centralisation – of having one system which captures all our relevant data. We are putting this into practice by making increased use of our CRM. With the arrival of a full-time Director of Community, Ben Clifford, we are able to have one person collecting and analysing all our member data. Another recommendation made by the review was to have a real-time donation mechanism, rather than an annual survey. This would make it easier for people to let us know about their donations, and thus for us to track our impact. We aim to do this in our next iteration of the website, in which we plan to introduce ‘personal dashboards’, which will allow people to keep a record of their giving, and tell them how much they have achieved with that giving.

The first 6 months of 2013 were an important learning time for Giving What We Can. The most important lesson was that we should try out a way of encouraging people to gradually increase their level of giving, and provide as much support and personal discussion as possible – it is not enough to simply present the information to as many people as possible. The next blog post will describe how we plan to put it into practice. The conclusion of the performance review was that the expected amount raised for extremely cost-effective charities by Giving What We Can appears to be well worth the resources invested in it.

Six Month Retrospective Progress Review

Health has been the focus of a large number of G8 commitments, and the achievement is rated as good. In fact, there has been a lot of progress over the past decades. From 1990 to 2011, there have been:

a 30% fall in malaria deaths, with 1.1 million deaths averted,

24% fewer people became infected with HIV/AIDS in 2011 than in 2001. In low and middle income countries, the figure is as high as 50%,

since 1988 the number of cases of polio has decreased by over 99%, and polio-endemic countries have declined from 125 to 3,

between 2000 and 2011, the percentage of households in sub-Saharan Africa owning at least one bednet rose from 3 to 53%,

in the same period, global coverage in measles vaccination rose from 72% to 84% and estimated measles deaths decreased by 71%.

How do these advances connect to G8 intervention?

The G8 supports health system strengthening through a variety of programmes. The biggest commitment - to provide 60 billion in funding to fight infectious diseases and improve health systems between 2008 and 2012 - was 80% delivered by 2011, so it is on track to be fulfilled. In 2005, the G8 pledged to mobilise support for the 

Global Fund to fight AIDS, Tuberculosis, and Malaria

. So far, these countries have provided almost ¾ of its funding - $13.5 billion. The Fund has provided:

anti-Tuberculosis treatment for 9.7 million people,

G8 countries have also provided support for:

Global Alliance for Vaccines and Immunisation

 (GAVI), with a pledge of $4.3 billion. Since 2000, GAVI has helped 72 low-income countries immunise 370 million children and avert more than 5.5 million deaths,

. In April 2013, $4 billion was pledged to the Polio Eradication and Endgame Strategic Plan 2013-2014.

the development of new vaccines, microbicides and drugs for AIDS, malaria, tuberculosis and virus research. G8 funding has increased by 22% since 2005, support for Product Development Partnerships has grown 77%.

maternal and child health, with $5 billion additional funding pledged in 2010 and support for initiatives such as the 

the development of treatment for neglected tropical diseases, with an increase in funding of 44% between 2007 and 2011, and a commitment to the control and elimination of ten priority neglected tropical diseases was also made last year.

But many important challenges remain. The central problem seems to be the lack of a qualified health workforce in developing countries. The improvement in this area has been negligible. Of the 57 countries below the World Health Organisation threshold of 2.3 health workers per 1000 people, 19 countries have seen an improvement, and 18 countries have had a decrease in density of health workers (there isn't data from the remaining 20). This is a critical situation: without health workers in place, new vaccines or treatments cannot reach many of the people who need them.

Also, we must not forget that AIDS and tuberculosis continue to be big problems. 7 million people in need of anti-retrovirals still lack access to these; and there were still 8.7 million cases of tuberculosis in 2011, with 1.4 million deaths. With the latter disease, there is the added risk of multi-drug-resistance TB.

For these reasons, Oxfam considers the report to be overly generous in this area.

G8: KEEPING PROMISES? Health

The G8 rate their work in economic development as good, claiming that there has been positive progress in supporting trade and development, including infrastructure and the investment climate, particularly in Africa. Most of the support has been in the form of:

facilitating dialogue with regional economic communities and governments.

As a highlight, over $20 billion have been donated to 

, a world programme aimed at fostering the economic growth of developing countries through increasing their participation in international trade.

On the other hand, progress towards the target of halving remittance costs (from 10% to 5%)- thereby increasing the amount of money entering developing countries in relation to the money sent - by 2014 has been slow, and it seems unlikely that it will be reached.

G8: KEEPING PROMISES? Economic Development 

The G8 rated their intervention in Aid and Aid Effectiveness to be satisfactory. Official Development Aid (ODA) from the G8 to low income countries has increased 30% from 2004 to 2012. But,perhaps due to the recession, ODA in 2012 was lower than in 2011. Another negative point is that the commitment to focus aid on low income countries does not seem to have been achieved, as only roughly 1/3 of aid has gone to these countries.

As a strong positive, the G8 are carrying through funding the Highly Indebted Poor Countries Initiative. Their pledge of $1 billion to the debt relief trust fund has been far surpassed: the total donated is now $1.9 billion. In addition, in 2005, the G8 agreed a proposal for debt relief that would cancel these countries' remaining debts, with an agreement to offset their debt and interest on it dollar for dollar up to their burden shares. As of the end of 2012, all had paid the agreed amounts.

Aid effectiveness has become progressively important in recent years. In particular, the G8 are taking part in the Global Partnership for Aid Effectiveness and Development Cooperation, set in 2011. This partnership prioritises the use of monitoring frameworks with the aim of achieving higher levels of transparency in interventions. In conclusion, though good work has been done on debt relief, it seems that the G8 should direct their aid more towards low income countries, and that more work needs to be done on aid effectiveness.

G8: KEEPING PROMISES?-Aid and Aid Effectiveness

_ This is the first part of a week long series looking at this summer's G8 summit. The G8 Summit was hosted by the UK government on 17-18 June at Lough Erne. One of the important products of this summit was the Lough Erne Accountability Report, a document detailing the progress the G8 have made towards their goals in nine key areas for world development, with a focus on the poorest countries: aid and aid effectiveness, economic development, health, water and sanitation, food security, education, governance,peace and security, and environment and energy._

The heads of government of the countries which form the G8 - the USA, the UK, Japan,Canada, France, Germany, Italy, and Russia - meet every year to discuss an agendarelevant to global development. All eight are in the top twelve countries with the highest gross domestic product, and the sum of their GDPs is about 60% of the world's total. For many, this makes them a symbol of concentration of capital and global injustice, and they have often been accused of not doing enough for international development. Because of their power and economic strength, these countries have huge potential to act upon the world's biggest problems - if they do choose to act, and do so effectively. In this context, accountability mechanisms are most welcome. They allow us to check the progress the G8 are making towards their own compromises, and to know the size of their contribution to important humanitarian goals. In short, they give us material on which to answer the question 'Are the G8 keeping their promises?'.

This is the second comprehensive accountability report on development goal G8 have produced since 2010. It covers the 56 development commitments present in that report, plus those agreed for inclusion in subsequent summits. In the majority of cases, countries are rated collectively, as most commitments have been made by the G8 as a group. Both quantitative and qualitative measures are employed. Whenever possible, the report uses the same data sources as previously (for example, the OECD or the United Nations), to maintain consistency.

In the past two decades, there has been great progress in eliminating global poverty: between 2000 and 2010, the percentage of the world's population living in global poverty fell from 34.1% to 20.6%. This doesn't mean that the G8 are doing enough. Massive problems remain - after all, over one in five people still live in extreme poverty. It is extremely unlikely that the first Millennium Development Goal - the eradication of global poverty by 2015 - will be reached. The progress in all the areas covered in the report is rated as satisfactory or good. In general, there has been an increase in funding, with more money pledged and delivered. Highlights include a 30% increase in official development aid between 2004 and 2012, the inclusion of considerations on aid efficiency, the commitment to donate $60 billion to fighting infection diseases and improving health systems between 2008 and 2012 (which was, in 2011, on track).

However, it is clear that progress has been insufficient in many areas. Some of the funding appears to have been directed into interventions of reduced effect, and some key areas lack funding. For example, only about a third of official development aid goes into low income countries, there are big funding shortfalls in education, and 1.1 billion people still lack access to basic sanitation. There clearly is scope for improvement in aid efficiency.

As an additional issue, the accuracy of the report has been questioned. The Overseas Development Issue (ODI) claims that the report:

lacks benchmarks for individual countries, hiding the significant differences in performance,

uses divergent reporting standards by different donors,

fails to recognise incompatibilities between different G8 policies.

We will look at each area of the report in detail, starting with 'Aid and Effectiveness'.

G8: Keeping promises? A look at the Lough Erne Accountability Report

Clara Marquardt

Giving What We Can promotes effective altruism - giving informed by rigorous evidence of programmes' and projects' effectiveness. An increasing demand for evidence necessitates a corresponding growth in the supply of such evidence by NGOs & social enterprises. Clara Marquardt from the Cambridge GWWC chapter interned over the summer with 

, an organization aiming to conduct "the most robust evaluation that has ever been conducted of a mass media intervention in a developing country". Clara took the opportunity to discuss the movement towards impact measurement and the challenges associated with real world randomized controlled trials with Will Snell, DMI's Director of Public Engagement & Development.

Clara: Could you give us a brief overview of DMI's objectives and approach?

Will: I would describe DMI as an organisation that uses radio and television to improve health outcomes, especially maternal and child health outcomes, in developing countries by changing behaviours. We are bringing together two different disciplines. One is the creative and logistical expertise in managing large-scale multi-issue media campaigns focused on for example the prevention and treatment of diarrhoea or safe child birth; the second is the scientific rigour of measuring the health impact of those campaigns.

We have developed a statistical model with the 

London School of Hygiene and Tropical Medicine (LSHTM)

 that allows us to predict and then measure not just behaviour change outcomes of a campaign but also health impacts in terms of numbers of lives saved. Historically there has never been any such attempt to measure or model the impact of media campaigns on mortality. This is because the impact on all-cause mortality is very hard to identify statistically given the many possible causes of deaths and the limited impact of any campaign.

The model predicts that DMI's campaigns will reduce under-five mortality by 16-23% within each African country at a cost that is as low as that of any other available intervention…between 2 and 10 USD per DALY.

Clara: You have already mentioned a defining characteristic of DMI - a strong focus on 'science' and impact measurement. How did this come about?

Will: This is a story about Roy Heads, DMI's founder and chief executive who had previously set up and run the BBC World Service Trust's health division and in doing so had worked on a large number of extremely successful mass media health campaigns.He set up DMI with the objective of injecting a more scientific approach because he felt that the impact of these campaigns while significant was not being measured sufficiently robustly.

Amongst other things this was leading to a lack of prioritization of such campaigns by funders and governments…the aim was to prove their impact and thereby persuade donors to make this a public health priority alongside complementary supply side initiatives. The scientific evidence base could also be used to maximize a campaign's mortality impact by informing the choice of health issues to campaign on.

Clara: Is DMI then starting to see this strategy pay off? Are donors and funders responsive to the provision of rigorous evidence? Which type of funder is proving the most receptive?

Will: DMI is starting to see the benefits and an increasing interest among funders. At the moment we have a track record, an evidence base based on behaviour change outcomes. The first time that we are directly measuring mortality outcomes is in the current Randomised Controlled Trial. While that is on going (endline results will be published in 2015) we don't have hard evidence on health outcomes other than a couple of campaigns where have been able to measure trachoma prevalence - in Ethiopia for instance. A lot of funders are responding to what we are doing. By definition the most science driven funders are waiting for the results of the Randomised Controlled Trial - this is gold standard evidence for them.

I would say that the major push for impact measurement is coming from certain bilateral donors who are extremely evidence and research focused, and also from some more public health focused organisations. Both groups of donors are driven partly by value for money considerations… I agree that the development of new financing mechanisms such as Development Impact Bonds (DIB) and the movement towards impact measurement are mutually reinforcing. The development of a DIB market is dependent on a certain amount of accurate data and evidence being available but once DIBs become more widespread this will motivate NGOs and other organisations to become more robust in terms of impact measurement in order to secure continued funding.

Clara: You have already mentioned DMI's on-going Randomized Controlled Trial (RCT) - an evaluation quite unique in its scope and scale. How would you describe the project?

Will: We were in discussions with the Welcome Trust when developing the DMI-LHSTM model to predict how many lives we save and they wanted us to provide some real world evidence that this could work. They funded us to set up a RCT to provide gold standard evidence that observed mortality reductions are attributable to our campaigns rather than other factors.

This is an evaluation design that has rarely if ever been used in the field of media - it is very difficult to randomize media interventions. We looked at nearly every country in Africa and Burkina Faso was the only country where this design could work because radio is very fragmented and localized. While there is a national network very few people listen to it, partly because it is in French, partly because it is under resourced. Everyone listens to local FM community radio stations and TV is more or less non-existent in rural areas. Given the presence of 'media dark areas' we were able to select 14 zones across the country each measuring roughly 50 km radius around a small town and randomly selected 7 to be intervention zones and 7 to be control zones.

Map of Burkina Faso showing intervention areas in blue and control areas in red

In the intervention zones we are broadcasting a 2.5 years child health campaign - a mixture of radio spots (one minute adverts broadcast 10 times a day) and dramas (interactive 5-10 minute health dramas broadcast every evening) (see 

Clara: From a technical perspective, you have already explained how you managed to implement treatment randomisation on a national scale. Are there any other areas where the Burkina RCT did diverge from the idealized 'textbook' RCT methodology? In other words, what are the challenges of actually realizing this rigorous evaluation design?

Will: Sure, the most obvious way in which we have diverged from an idealized design is in the number of clusters. Having 7 intervention and control clusters reduces our statistical power to attribute impact.There are obvious practical reasons why we couldn't go beyond that: There are a limited number of radio stations in Burkina Faso with a sufficient listenership. Also, this is a 12 million USD trial - would we have doubled in size we would have more or less doubled in costs. That notwithstanding we have a 80% statistical power (a one in five chance that we will have a significant impact on mortality but won't be able to prove it statistically).

The other area in which we have faced scientific challenges has been around controlling for other health interventions. There are a number of nationwide interventions including vaccination campaigns that should theoretically affect control and intervention clusters equally. But there have been some health campaigns where there has been a possibility of them broadcasting or operating in some areas but not in others. We have been working closely with the Ministry of Health to try and avoid that happening during the broadcasting period of our trial. That has been possible in most cases so we think that contamination from other projects is minimal… but will never be completely eliminated.