Impact of financial inclusion in low- and middle-income countries

Additional Info

  • Authors: Maren Duvendack, Philip Mader
  • Published date: 2019-01-07
  • Coordinating group(s): International Development, Methods
  • Type of document: Title, Protocol, Review, Plain language summary
  • Volume: 15
  • PLS Title: Financial inclusion interventions have very small and inconsistent impacts
  • PLS Description: Financial inclusion programmes seek to increase access to financial services such as credit, savings, insurance and money transfers and so allow poor and low-income households in low- and middle-income countries to enhance their welfare, grasp opportunities, mitigate shocks, and ultimately escape poverty. This systematic review of reviews assesses the evidence on economic, social, behavioural and gender-related outcomes from financial inclusion.
  • Title: Impact of financial inclusion in low- and middle-income countries
Download files:

About this systematic review

Financial inclusion programmes seek to increase access to financial services such as credit, savings, insurance and money transfers and so allow poor and low-income households in low- and middle-income countries to enhance their welfare, grasp opportunities, mitigate shocks, and ultimately escape poverty. This systematic review of reviews assesses the evidence on economic, social, behavioural and gender-related outcomes from financial inclusion.

What are the main results?

This review includes studies that synthesise the findings of other studies (meta-studies) regarding the impacts of a range of financial inclusion interventions on economic, social, gender and behavioural outcomes. A total of 32 such meta-studies were identified, of which 11 were of sufficient methodological quality to be included in the final analysis. The review examined meta-studies from 2010 onwards that spanned the globe in terms of geographical coverage.

Impacts are more likely to be positive than negative, but the effects vary, are often mixed, and appear not to be transformative in scope or scale, as they largely occur in the early stages of the causal chain of effects. Overall, the effects of financial services on core economic poverty indicators such as incomes, assets or spending, and on health status and other social outcomes, are small and inconsistent. Moreover, there is no evidence for meaningful behaviour-change outcomes leading to further positive effects.

The effects of financial services on women’s empowerment appear to be generally positive, but they depend upon programme features which are often only peripheral or unrelated to the financial service itself (such as education about rights), cultural and geographical context, and what aspects of empowerment are considered.

Accessing savings opportunities appears to have small but much more consistently positive effects for poor people, and bears fewer downside risks for clients than credit. A large number of the meta-studies included in the final analysis voiced concerns about the low quality of the primary evidence base that formed the basis of their syntheses. This raises concerns about the reliability of the overall findings of meta-studies.

Background

Financial inclusion is presently one of the most widely recognised areas of activity in international development. Financial inclusion initiatives have built upon donors’ experience with microfinance, but have displaced and superseded microfinance interventions in recent years with a more encompassing agenda of financial services for poverty alleviation and development. With financial inclusion, policymakers and donors hope that access to financial services (including credit, savings, insurance and money transfers) provided by a variety of financial service providers, of which microfinance institutions (MFIs) are a subset, will allow poor and low-income households in low- and middle income countries to enhance their welfare, grasp opportunities, mitigate shocks, and ultimately escape poverty. Another hope is that increased access to financial services will advance macroeconomic development, which is also expected to benefit poor/low-income households. More recently, some donors have suggested behavioural changes (such as household spending decisions) to be desired outcomes of access to financial services, as well. Unlike most previous systematic reviews, which focused on microfinance interventions (or sub-sets thereof), we explicitly adopt a broader scope to review any available systematic review and or meta-analysis evidence on financial inclusion as a whole field.

Systematic reviews and meta-analyses (in short: meta-studies) have sought to clarify the impacts from financial inclusion on poor people in low- and middle-income countries, based on an array of different underlying studies which include quantitative and qualitative work based on long-term and short-term data. The bulk of these meta-studies have focused on microfinance, and many specifically on microcredit. The very different quality and approaches of these meta-studies, and of the studies underlying them, however, pose a major challenge for policymakers, programme managers and practitioners in assessing the benefits and drawbacks of finance-based approaches to poverty alleviation. Increasingly there is confusion about the impacts, and a risk of “cherry picking” among different findings. Further, many meta-studies are not taking into account what is missing from their primary studies, which would affect an understanding of the evidence, for example when not analysing or reporting gendered impacts. More recently, primary studies have also sought to understand the impacts of financial inclusion initiatives more broadly, but the systematic review evidence has not yet progressed as far as for microfinance.

Objectives

The objective of this systematic review of reviews is to systematically collect and appraise the existing meta-studies of financial inclusion impacts, analyse the strength of the methods used, synthesise the findings from those meta-studies, and report implications for policy, programming, practice and further research.

Systematic reviews of reviews are undertaken in other sectors for which evidence is widely available, but they are non-existent in international development. This systematic review of reviews thus provides the opportunity to develop and pilot an evidence synthesis approach in a sector where there is a large body of evidence of variable quality, but a systematic appraisal and synthesis of the body of systematic reviews and meta-analyses is lacking.

This study critically engages with approaches to systematic reviews of reviews with a view to further developing systematic review of review methods, and it aims to answer the following questions to gain better clarity about financial inclusion impacts:

Impacts:

  • What is known from existing meta-studies about the (social, economic, and behavioural) poverty impacts of different types of inclusive financial services (e.g. credit, savings, insurance, money transfers), regardless of provider, on poor and low-income people in low- and middle income countries? This includes poverty impacts through macroeconomic development, to the extent that it results from financial inclusion.
  • What is known from existing meta-studies about the gendered impacts of different types of financial inclusion activity (e.g. credit, savings, insurance, money transfers) – in other words, what does the evidence tell us about how gendered participation affects interventions’ effects, and about whether or not (and in what ways) financial services empower women in low- and middle income countries? o What is known from existing meta-studies about the reasons for financial services uptake, or other participant views about the financial services on offer?

Methodology:

  • Including using a gender and equity lens, what methods and standards have meta-studies used to draw conclusions from the studies they reviewed?
  • What difference does the choice of methods and standards make to the results? o How could the methods and standards be improved in order to draw more robust and reliable conclusions via meta-studies?

Search methods

We adopt a multi-pronged search strategy that explored 7 bibliographic databases to identify published literature, plus a wide range of institutional websites for published and unpublished literature, and back-referencing from recent systematic reviews to ensure additional sources were identified. In addition, a snowballing approach was adopted and an advisory board plus leading authors working on financial inclusion topics were consulted to ensure that no key studies were missed. We also ran citation searches on included systematic reviews and meta-analyses in Google Scholar, Scopus and Web of Science to identify more recent systematic reviews or meta-analyses not retrieved in the database searches. No restrictions were placed on the language of papers but all searches were limited to 2010 onwards.

Selection criteria

We adopted the following selection criteria to establish study inclusion or exclusion:

Types of reviews

We include studies that self-identify as systematic reviews and or meta-analyses of the impacts of financial inclusion (including, but not limited to, microfinance). These, in turn, focused on synthesising quantitative, qualitative and or mixed methods evidence.

Types of participants

Our population is the population of participants in inclusive finance activities in low- and middle-income countries.

Types of interventions

We include meta-studies that address at least one or more types of intervention for financial inclusion. The key is that the intervention must be fundamentally a financial service directed at poor and low-income people. In most cases, we find the interventions are one or more sub-categories of microfinance: credit, savings, insurance, leasing, and/or money transfers. However, our search strategy explicitly targets the broader range of inclusive finance activities, including mobile monies, mobile payments systems, index insurance, or savings promotion.

Types of outcome measures

Meta-studies capturing a wide range of poverty indicators (including income, assets, expenditure, personal networks, gender/empowerment, well-being, health, etc.) are included.

All meta-studies were screened by two research assistants independently, with the two review authors independently reviewing each meta-study marked for inclusion. Full texts were obtained and screened when a decision could not be made; an arbitration procedure was in place in case of disagreements.

Data collection and analysis

A total of 32 meta-studies were identified after completing the screening process. However, only 11 of these were assessed to be of sufficient methodological quality to be included in the final analysis. We note that a large number of these meta-studies voiced concerns about the low quality of the primary evidence base that formed the basis of their syntheses, which in turn raises concerns about the reliability of the overall findings presented at the review level. Combining a wide range of low quality studies into systematic reviews to aggregate their findings is risky.

A coding tool was developed to extract data from the included meta-studies on the following areas of interest:

  1. Context
  2. Type of intervention
  3. Type of review, design and methods used
  4. Outcome measures
  5. Quality assessment
  6. Study results and findings

Data were extracted at the meta-study level. However, for meta-studies classified as high- and medium-confidence, when necessary, we also extracted information at the primary study level.

The synthesis of results was guided by a theory-based mixed methods synthesis approach with a focus on a narrative synthesis that incorporates quantitative elements as appropriate.

Results

Five out of the 11 (medium- and high-confidence) meta-studies that we reviewed drew largely positive conclusions about the relationship between financial services access and changes for poor people, and the other six drew largely mixed, neutral, or unclear conclusions. The detailed review of the evidence base uncovered a nuanced picture, reflecting large variations across the effects of different interventions and for different people in different contexts. Findings across the reviews were heterogeneous and often inconsistent, both within and across reviews, and many reviews did not find evidence of expected or presumed impacts.

The present high-level evidence does not suggest that financial inclusion initiatives have transformative effects. On average, financial services may not even have a meaningful net positive effect on poor or low-income users, although some services have some positive effects for some people. Overall, we find:

  • The impacts are more likely to be positive than negative, but the effects vary, are often mixed, and appear not to be transformative in scope or scale, as they largely occur in the early stages of the causal chain.
  • The effects of financial services on core economic poverty indicators such as incomes, assets or spending are small and inconsistent.
  • The effects of financial services on women’s empowerment appear to be generally positive, but they depend upon programme features (which are often only peripheral or unrelated to the financial service itself, for instance exposure to women’s rights), context, and what aspects of empowerment are considered, and their assessment is confounded by a difficulty of consistently conceptualising and measuring empowerment.
  • The effects of credit and other financial services on health status and other social outcomes appear to be small or non-existent.
  • There is no evidence for meaningful behaviour-change outcomes leading to further positive effects.
  • Accessing savings opportunities appears to have small but much more consistently positive effects for poor people, and bears fewer downside risks for clients than credit.

Many of the primary studies that were included in the meta-studies we analysed in depth had medium or even high risk of bias, due to their study design, poor reporting of methodology, and other causes. As some of the meta-studies highlighted, it is mainly the higher risk of bias studies that drive most of the positive impact estimates. Our findings thus broadly confirm the ‘stainless steel’ law of evidence that, the more rigorous and lower risk of bias studies become, the less likely they are to find effects. This applies to both our reviews and to the underlying primary evidence that they have reviewed. Given that the reviews we classified as being of lower methodological quality were more likely to report positive effects, we must treat their positive findings with caution.

In summary, almost all effect sizes we find are quite small and hardly indicative of transformative changes from financial inclusion, and are found dominantly on lower-order or intermediate outcomes. Many effects are strongly heterogeneous, both across studies and over time, places, populations, gender, ethnicity and between interventions; this suggests them to be unreliable and/or context-dependent. Positive findings tend not to repeat from one context, intervention type or study to another, and at least as many findings are mixed or inconclusive as are positive. As a result, the positive results found for financial inclusion are fragile, and need to be treated with caution. An exception appears to be with regard to savings, where both immediate outcomes and wider poverty measures are affected in a positive, but relatively small, way; however, we base this mainly on the findings of one high confidence meta-analysis (Steinert et al. 2018). There is no savings “revolution” going on, but savings at least appear to do some good and no harm.

Authors’ conclusions

We have taken the evolution of the financial inclusion impact literature toward a natural conclusion, with a higher level of evidence systematisation, to provide an overview of what has become an increasingly perplexing array of meta-studies that each offer partial overviews. By reviewing these reviews, we have drawn on what is likely the largest-ever evidence base on financial inclusion impacts, and have uncovered strengths, gaps and weaknesses of the existing high-level evidence. We hope that we have reduced the amount of confusion and uncertainty arising from the many different meta-studies on financial inclusion published in recent years, not least thanks to our systematic assessment of the variations in quality within that field.

The (perhaps boring) truth that seems to emerge about financial inclusion is that it is not changing the world. On average, financial services may not even have a meaningful net positive effect on poor or low-income users, although some services have some positive effects for some people. Considering that for most people financial services (whether they can access them, and how they use them) will be only one among many possible determinants of their life chances and their socio-economic well-being, this finding ought not to be unexpected, and we anticipate that it will be confirmed by future research. The potential and actual impacts of financial inclusion need to be viewed against those of comparable interventions, such as graduation and livelihoods-enhancement programmes.

We note that, fortunately, our findings regarding impact chime in with an emerging realism around microfinance, including in the donor community: recognising that erstwhile claims of transformative impact were unrealistic and that the hype for microfinance, particularly microcredit, was overblown. We welcome this newfound realism and wish to encourage it with the help of this review, in which we provide a systematic overview of the evidence as well as the areas of doubt in the evidence base. At the same time, we wished that going through all stages of the hype cycle – enthusiasm, inflated expectations, and disillusionment – had not been necessary in order to arrive here. And we must warn that we see a similar hype of strong claims emerging around the much more encompassing notion of financial inclusion, with the promise of marrying macro-structural economic improvements with micro-structural poverty relief. We found no evidence for the wider claims made for the beneficence of financial inclusion, as offering poor people a better service, or as having broader macro-structural effects, being any truer than those once made for microfinance, in large part due to a lack of appropriate research at the meta-study level. We strongly caution against repeating the hype cycle, this time around the idea of financial inclusion.

At the same time, we think it crucial to bear in mind that the alternative to financial inclusion is not to do ‘nothing’, but rather it is necessary to uncover what kinds of interventions work best for whom and where, and how best to deliver them. The policy and research space – and ultimately poor and low-income people themselves – would benefit from a more open and clear-sighted discussion on the many valid alternatives to financial inclusion programming and on how best to gain the necessary evidence to inform that discussion. To this end, our review also includes a brief examination of the impact evidence for graduation and livelihoods programmes.

In terms of evidence gaps, it is noteworthy that none of the meta-studies we reviewed (high-, medium- or low-confidence) managed to assess debt levels or indebtedness patterns in depth as an outcome of financial inclusion. While we cannot comment on the reasons for the lack of attention paid to the issue, except that we are aware of it also being a blind spot of the underlying primary studies, we find this to be a glaring omission of the financial inclusion literature as a whole. We believe the political economy of research funding needs to shift such that researchers are enabled and encouraged to more rigorously explore the most important potential downsides and risks of development initiatives like financial inclusion. Furthermore, we found no evidence (among the high-, medium- or low-confidence meta-studies) for the claim that financial inclusion interventions lead to macroeconomic development and subsequent improvements in the lives of the poor; this may be because the argument has only become prominent in recent years. There is also not much attention given (among the high-, medium- or low-confidence meta-studies) to service/amenities-related programmes such as water credit, sanitation loans, or loans for micro solar systems, especially the notion of ‘Green Microfinance’ where microfinance is applied to promote environmental sustainability.

Moreover, given that the majority of financial inclusion effects we found in assessing the high- and medium-confidence studies were at the early stages of the causal chain, there is a need for studies to better capture long-term effects and demonstrate more meaningful impacts, especially at the final stages of the causal chain. The vast majority of the studies that our meta-studies reviewed had a duration of 1 to 3 years. These studies are likelier to find changes in behaviours or attitudes rather than structural changes to people’s poverty status, and it is not safe to assume that the latter will result from the former. The design of most studies underlying the meta-studies that we reviewed has not been conducive to establishing whether short-term or immediate outcomes (such as financial knowledge or entrepreneurial propensity) would translate into intermediate outcomes (such as savings accumulation or microenterprise income) and especially more distal, transformative outcomes (higher net worth or higher incomes). We would suggest that this also reflects a problem of the political economy of development research, with a combination of funder restrictions (favouring shorter timelines over multi-year projects) and difficulty of gaining long-term support from implementer organisations discouraging appropriate designs.

We have also encountered some important limitations of working at this level of systematisation, including: difficulties of assessing the reliability of the levels of evidence underlying ours; analysing effect sizes that are presented in standardised and indexed form, which often reveal little about the underlying measures used; the different ways in which data have been analysed and findings presented across very different types of meta-studies; crude categories for intervention and outcome types, lumping together a highly diverse evidence base that muddies the waters further. Another problem we encountered was that the meta-studies we reviewed, regardless of their own quality, often built on a relatively weak underlying base of underlying studies, making their findings fragile. To put it differently, combining a wide range of low quality studies into systematic reviews to aggregate their findings is risky, and perhaps analogous to the behaviour of financial institutions in the run-up to the 2008 financial crisis, with pooling dubious individual assets (such as sub-prime mortgages and loans) into “triple-A” structured financial products, with only seemingly better aggregate results.

Going forward, we would recommend that authors of primary studies and meta-studies engage more critically with study quality and ensure better, more detailed reporting of the concepts, data and methods they used. At the systematic review of review level, more methods guidance (especially in terms of synthesis approaches) and clearer reporting standards that adapt the Cochrane (health-focused) guidance to the social science and international development context would be helpful.

Contact us

  • P.O. Box 222, Skøyen,
    N-0213 Oslo, Norway
  • (+47) 21 07 81 00
  • info@campbellcollaboration.org