The effects of school-based decision-making on educational outcomes in low- and middle-income contexts
- Authors: Roy Carr-Hill, Caine Rolleston, Rebecca Schendel
- Current phase: Review
- Published date: 2016-12-07
- Coordinating group(s): International Development
- Type of document: Title, Protocol, Review, Plain language summary
- Volume: 12
- Category Image:
- PLS Title: School-based decision-making has positive effects on education outcomes but less so in low-income countries
- PLS Logo:
- PLS Description: Decentralizing decision-making to schools has mainly small to moderate positive effects in reducing repetition, dropouts and increasing test scores. These effects are restricted to middle-income countries, with fewer and smaller positive effects found in low-income countries or in disadvantaged communities.
- Title: The effects of school-based decision-making on educational outcomes in low- and middle-income contexts
Although there have been significant improvements in recent decades, access to education remains limited, particularly for girls, poor children and children in conflict-affected areas. There is also worrying evidence that many children who are enrolled in school are not learning. Recent estimates suggest that around 130 million children who have completed at least four years of school still cannot read, write or perform basic calculations (UNESCO, 2014, p. 191).
Many governments have attempted to address this situation, while also improving efficiency and reducing costs, by devolving decision-making authority to schools, as it is assumed that locating decision-making authority within schools will increase accountability, efficiency and responsiveness to local needs (Gertler et al., 2008). This devolution includes a wide variety of models and mechanisms, differing in terms of which decisions are devolved (and how many), to whom decision-making authority is given, and how the decentralisation process is implemented (i.e., through ‘top-down’ or ‘bottom-up’ processes). All models and mechanisms are presumed to increase responsiveness to local needs and accountability by bringing community members into direct contact with schools, and to increase efficiency by making financial decisions more transparent to communities, reducing corruption and incentivising investment in high quality teachers and materials.
Although the rhetoric around decentralisation suggests that school-based management has a positive effect on educational outcomes, there is limited evidence from low-income countries of this general relationship. Existing reviews on school-based decision-making have tended to focus on proximal outcomes, while the more comprehensive reviews that do exist are not formal systematic reviews, according to the criteria set by the Campbell Collaboration. They also need updating, as they (a) rely on literature that is now nearly ten years out of date and (b) focus almost exclusively on Central America, referencing almost no evidence from other low- and middle-income countries (L&MICs). Existing reviews on this topic also tell us very little about why school-based decision-making has positive or negative effects in different circumstances.
This review aims to address these gaps by answering the following questions: (1) What is the impact of school-based decision-making on educational outcomes in low- and middle-income countries (L&MICs) (Review Question 1)? (2) What are the barriers to (and enablers of) effective models of school-based decision-making (Review Question 2)?
For the purposes of the review, ‘school-based decision making’ was defined as any reform in which decision-making authority has been devolved to the level of the school. Within this broad definition, there are three main mechanisms discussed in the literature: (1) reforms that devolve decision-making around management to the school level; (2) reforms that devolve decision-making around funding to the school level; and (3) reforms that devolve decision-making around curriculum, pedagogy and other aspects of the classroom environment to the school level.
This review followed an explicit protocol following methodological guidance provided by the Campbell Collaboration and the EPPI-Centre at the UCL Institute of Education (Becker et al., undated; Gough et al., 2012; Hammerstrom, 2009; Shadish & Myers, 2004).
To be included in the review, all studies had to: 1) be empirical in nature and focused on primary and secondary schools within L&MICs; 2) investigate a change in decision-making authority from a higher level of decision-making authority to the level of the school (excluding studies where the intervention was conceptualised, managed and implemented by an external decision-making agency, or aimed exclusively at improving the functioning of existing devolved decision-making structures); 3) provide data on the relationship between school-based decision-making and at least one educational outcome (either proximal, e.g. attrition, equality of access, increased enrolment; or final, e.g. student learning, as captured by test scores, psychosocial and non-cognitive skills, etc.); and 4) rely on data collected since 1990.
To be included in reference to Review Question 1, studies needed to be causal in nature, meaning we included: (1) Experimental designs using randomised or quasi- randomised assignment; (2) Quasi-experimental designs; and (3) comparison group designs using before-and-after data at baseline and endline, as well as those using cross-sectional endline data only, where analysis was used to control for confounding. For Review Question 2, we included studies of any empirical design, so long as they provided additional data relating to those interventions featuring in the impact component of the synthesis.
Potentially relevant literature was identified through a five-stage search strategy, which comprised: 1) Identification of existing systematic reviews in related areas; 2) Targeted searches in a wide range of bibliographic databases and websites; 3) Hand searches of the eight most relevant journals relating to the topic; 4) Citation chasing; and 5) Contacting experts involved in the research area. A comprehensive list of search terms was developed in collaboration with information scientists at the EPPI-Centre. Search terms were also translated into French, Spanish and Portuguese for use in regionally specific databases. All identified literature was subjected to a two-stage screening process. Relevant studies were then appraised for robustness of evidence and methodological rigour prior to synthesis.
In order to answer Review Question 1, we conducted meta-analysis, relying on the use of ‘standardised mean difference’ (SMD) calculations to compare effects across studies. In our meta-analysis, we were able to report on the impact of any school-based decision-making reform on six educational outcomes: 1) student drop-out; 2) student repetition; 3) teacher attendance; and 4) student learning, as assessed via i) language test scores, ii) math test scores, iii) aggregate test scores (i.e. tests of more than one subject). We also examined heterogeneity by investigating differences in impacts based on three moderating variables – level of decentralisation, income level, and type of evaluation design. Further, we discuss and synthesise sub-group effects discussed in the included studies themselves. Analysis in reference to Review Question 2 followed the principles of framework synthesis (Thomas et al., 2012), in order to identify the main barriers and enablers that appear to have influenced the impact of the interventions under review.
We identified 2,821 titles through our five-stage search. Of these, 100 met our eligibility criteria. Thirty of the 100 met the design criteria required for RQ1, but three were removed from the RQ1 synthesis, due to high risk of bias. A fourth study had to be excluded due to missing data. Twenty-six impact studies were thus included in the meta-analysis. These 26 studies investigate the impact of 17 individual interventions. Of the 73 non-causal studies subjected to quality appraisal, nine were identified to be of sufficient quality to provide additional data on the included interventions.
Devolving decision-making to the level of the school is found to have a somewhat beneficial effect on drop-out; a pooled effect of reducing drop-out by 0.07 standard deviations (SDs). For repetition, the equivalent pooled effect is a reduction of 0.09 SDs. Effects on test-scores are larger and more robust. We find a positive and significant improvement of 0.21 SDs in aggregate test scores on average, and positive and significant improvements of around 0.07 SDs in scores on language and 0.08 on math tests. Further analysis of test score results suggests that these results pertain to middle income countries, while we did not find statistically significant improvements in test scores in low-income country settings, with the exception of one study in Kenya (now a middle income country). Evidence does not show that effects on teacher attendance are significant overall, but there is evidence that effects are stronger in contexts of high decentralisation.
In common with other comparative studies of the impacts of educational initiatives (Kremer et al., 2013; Snilstveit et al., 2015), these effects of decentralised school-based decision-making are relatively small in magnitude. For example, Snilstveit et al. (2015) conducted a recent and broad-ranging review of interventions to improve learning outcomes in L&MICs and report that the most substantial effects on test-scores are for ‘structured pedagogy programmes’, which found a pooled effect on math scores of 0.14 SDs, while a large number of education intervention types showed no overall effects. Accordingly, while educational effects appear small in comparison to those in some other fields, effects of school-based decision-making may be considered similar to interventions that demonstrate medium-sized effects on education outcomes.
Most of the included studies do not conduct any sub-group analysis relating to individual characteristics, such as gender and student background; those that do differ in their findings. However, there is some evidence to suggest that school-based decision-making reforms have a stronger impact on wealthier students with more educated parents. It appears that school-management reforms may be particularly impactful on children in younger grade levels.
School-based decision-making reforms appear to be less effective in disadvantaged communities, particularly if parents and community members have low levels of education and low status relative to school personnel. Devolution also appears to be ineffective when communities choose not to actively participate in decision-making processes. Small schools, however, may find school-based decision-making to be effective, particularly if community members establish a collaborative, rather than an adversarial, relationship with teachers.
Conclusions and implications for policy, practice and research
Overall, we can conclude that devolving decision-making authority to the school level can have a positive impact on educational outcomes, with magnitudes of effect in the median range for education programmes, but that this is only likely in more advantaged contexts in which community members are largely literate and have sufficient status to participate as equals in the decision-making process.
Our findings carry a number of implications for policy and practice. First, it appears that school-based decision-making reforms in highly disadvantaged communities are less likely to be successful. Parental participation seems to be the key to the success of such reforms and this is linked to the real authority or status and cultural capital of community members. Second, the involvement of school management committees in personnel decisions appears to play a role in improving proximal outcomes, such as teacher attendance, but success is also likely to be linked to the overall teacher job market and the prospects of long-term employment. Third, the specifics of programme design appear to be crucial. Given the limited evidence available in this review, and the contextualised nature of that evidence, we cannot conclude with certainty that incorporating certain elements into school-based management reforms are generally beneficial. However, it does appear that the details of such supplementary elements may be important. The evidence also suggests that, at least in some contexts, impact on student learning may take longer than is often allowed within evaluation timelines. Where donors are involved, this also means that decentralisation reforms may require sustained donor commitment over the long term.
The review also suggests a number of fruitful directions for future research. Although a large number of titles were identified during our initial search, the small number of impact studies included in the meta-analysis represent a limited geographic diversity and a small number of discrete interventions. There needs to be further robust analysis of the impact(s) of large- scale school-based decision-making reforms that have recently been implemented, as well as further analysis of the conditions that mitigate their impact. There is also a clear need to examine the potentially negative impacts of these reforms, given widespread adoption of such policies. Although this review has highlighted a number of potential enablers and barriers of effects, the limited evidence base has prevented us from drawing any robust conclusions on the conditions necessary for positive impact. A future review of the same topic, drawing on broader qualitative evidence, would complement the findings of this study.