What to do about What Works?

 

When I was appointed as CEO of Campbell I said that the most important challenge of all is to get the evidence from reviews into policy. So, over the last few months, I have been learning about how this problem

One approach is the Nordic model. In Denmark, Norway and Sweden there are government-funded research institutions employing researchers whose full-time job is to produce systematic reviews.  Which reviews are undertaken each year are agreed through a consultation exercise with  government agencies. So the model has demand-led approach to evidence generation, but not on a  contract-by-contract basis. These reviews inform policy discussions, guidelines development and so on.

The advantages of the Nordic model include that experienced researchers are on hand to produce  the required reviews. One reason for the development of the model was experience that academics  were not interested in writing reviews. They were more interested in researching say the ideological  basis for the primary curriculum in Sweden in the early 1800s than issues of more immediate policy relevance.

It’s politically correct these days to say evidence-informed policy rather than evidence-based policy.  The Norwegian Knowledge Centre for Education provides a good example of how this works in  practice. The Ministry and the Centre agree a priority topic for a review, say school dropouts. On the basis of the review, two or three promising interventions are then tested in Norway using randomized controlled trials (RCTs) at the county level. Evidence-informed policy is not a blueprint  approach. It is using evidence to guide policy choices in the local context. 

The UK What Works Centres (WWCs) are a different model. There are now seven centres dedicated  to the collating and promoting the use of evidence across health, education, early years, crime  reduction, local growth, aging and well-being, as well as centres for Scotland and Wales. These  WWCs are more arms’ length from government than in Nordic countries, receiving funding from Big Lottery and research councils as well as central government. This independence means that each centre needs to identify its policy and practice partners and build the relationships which are usually important for successful policy influence.

The largest, and clearest example of influence, is the Education Endowment Foundation who have  produced a Teacher and Learning Toolkit, and Early Years Toolkit, based on evidence from reviews.  The evidence is presented in a very user-friendly format showing the size of the effect as extra  months’ learning, cost and strength of the evidence. A review by the UK National Audit office found  that use of the Teacher and Learning Toolkit by school administrators rose from around 1/3 in 2013  to 2/3 in 2015. An important factor behind this increase has been the Pupil Premium, a payment  made directly to schools who are told to use it an evidence-based way to improve learning outcomes.

Across the ocean Results for America, with support from the Arnold Foundation, is promoting  Moneyball for Government and the Cities Initiative, to encourage adoption of programs which have  been shown to work by RCTs. So, unlike the European initiatives, although the policy targets are  government agencies, non-profits are the driving force. And the evidence is from single studies rather than reviews

So there is a variety of international experiences. All of these will be presented and discussed at the What Works Global Summit. 

What may have struck you is that there are a growing number of national agencies supporting  production of reviews of the global evidence. And you’d be right. One hoped for outcome of the  What Works Global Summit is a consensus that this demand should be coordinated. And what agency can facilitate that demand? Who are you going to call? Call Campbell.

Contact us

  • P.O. Box 222, Skøyen,
    N-0213 Oslo, Norway
  • (+47) 21 07 81 00
  • info@campbellcollaboration.org