João Pedro Azevedo, Lead Economist, Education Global Practice, World Bank Group and Silvia Montoya, Director, UNESCO Institute for Statistics (UIS)
Our choice of a measure shapes our understanding of the size and nature of a problem. In a recent blog we discuss why the learning poverty measure is well suited to monitor the educational impacts of COVID-19. In this blog, we discuss two complementary concepts: the learning poverty gap and learning poverty severity, to look at distributions among the learning poor and measure how these changes affect learning inequality. The UNESCO Institute for Statistics (UIS) and the World Bank provide data for both of these concepts. Data for many key SDG 4 indicators were updated in the March data refresh. The UIS is also collecting data on the national response to COVID-19 on education, equity and inclusion.
While learning poverty is a simple concept to grasp, this indicator alone does not provide a picture of the learning level and distribution of learning among those below the minimum proficiency level (MPL). Because learning poverty is a headcount ratio, estimates treat all students below the minimum proficiency level as being equally learning deprived. It also does not reflect improvements in learning below the MPL threshold, which can fail to create compatible incentives as it may miss progress in foundational subskills critical for developing reading proficiency, for example, knowledge of spoken words and how to use them, hearing and making the sounds of words, mapping sounds to letters and letters to sounds while learning letter names, among others, as described in the reading rainbow (Figure 1). Understanding the heterogeneity among the learning poor is critical to combat learning poverty as children who do not master these subskills in early primary grades remain unable to read with comprehension.
Figure 1: SDG 4.1 framework and reading rainbow of literacy subskills
In order to capture changes below the MPL, one can extend the learning poverty measure to a more general class of inequality sensitive indicators, as discussed during the seventh GAML and TCG meetings. We examine the desired attributes of this proposed measures and some of their axiomatic properties in a recent paper. In this blog we summarize some of the main motivations for this broader family of learning poverty measures, and illustrate its value based on two specific cases that are sensitive to the distribution of learning and the potential inequity among the kids below the minimum proficiency level, namely, learning poverty gap (α=1) and learning poverty severity (α=2) (Figure 2).
Figure 2: Distributional sensitive measure of learning deprivation
- First, the world faces a learning crisis, with 53% of 10-year-olds unable to read and understand a simple age-appropriate text.[1] In low-income countries, the learning poverty headcount ratio approaches 90%. In this context, it is critical to use a measure that can differentiate the magnitude of the learning crisis, by looking at both the learning poverty rate and learning poverty gap and to be able to recommend sound policy solutions.
- Second, if two educational systems have the same learning poverty rate, the one with the lower level of learning among those below the MPL should be considered worse off, other things being equal.
- Third, learning is a cumulative and progressive process, and a good measure should be able to capture changes in foundational learning, which take place at lower levels of the learning scale. Such sensitivity can help policymakers prioritize foundational learning and help capture the progress that education systems are achieving from investments in foundational skills.
- Fourth, there is overwhelming evidence that teaching at a level too high for students’ proficiency has a detrimental effect on how much they learn, so a good measure should be able to take students’ initial knowledge into consideration.[2]
- Fifth, a distribution-sensitive measure captures an important dimension of the complexity of the problem: unequal distribution of learning among the poor. The greater the inequality of the learning poverty gap, the more flexible the education system must be to both identify students‘ needs and offer appropriate learning opportunities. Understanding such heterogeneity can be of critical importance to plan lessons and/or design Teaching at the Right Level interventions.
Other justifications for using distribution-sensitive measures can be grounded on egalitarian preferences for society. This assumes that given a choice between improving learning poverty of two children by the same amount, society’s preference will be for helping the one who is worse off. Several arguments can support this value judgment, such as the diminishing marginal value of literacy and the importance of social cohesion but also economic growth.
The extent depends on the prevalence of countries with:
- the same learning poverty level but different learning poverty gaps (Figure 2, panel A), or
- the same learning poverty gaps but different learning poverty severity (Figure 2, panel B).
Figure 3 illustrates those points using the latest available data from 99 countries in the learning poverty database where the learning poverty gap and learning poverty severity indicators are available.[3] The figure shows a wide range of learning poverty gaps among the poor in countries with similar levels of learning poverty (panel A). Several countries have around 70% learning poverty, including the Philippines and Nicaragua, but the learning poverty gap among the poor in the Philippines is almost three times the size of the gap in Nicaragua. This suggests that the effort required to tackle learning poverty in the Philippines might be larger than in Nicaragua.
But this is not the whole story. It’s also important to look at inequality or learning poverty severity. For example, learning poverty severity in Nicaragua is almost 10 times what it is in the Philippines, suggesting a far greater level of heterogeneity among the learning poor students in Nicaragua. This finding illustrates the empirical relevance of the distinction between measures of learning poverty level, gap, and severity and the importance of clarity on which of the three measures might be the most relevant when analyzing different situations. As a result, policies to reduce learning poverty could differ considerably if the levels of the learning poverty gap or learning poverty severity are drastically different.
Figure 3. Relationships between learning poverty, the learning poverty gap, learning poverty severity, the learning deprivation gap, and the learning severity gap
| Panel A. Countries where students are at the same level of learning poverty, require very different levels of effort (learning poverty gap). | Panel B. Countries that require the same average effort (learning poverty gap); have very different levels of learning poverty inequality among students below the MPL. |
As school systems reopen after extended closures due to COVID-19, it will be critical to meet students where they are and to monitor changes in the learning distribution among the poor, given that evidence suggests that a significant source of inequality is within groups. To measure and track the problem of within-group inequality, learning poverty severity is the appropriate measure. The relevance and usefulness of these complementary measures are clear from numerical, empirical, and policy perspectives.
[1] World Bank 2019.
[2] Banerjee et al. 2016.
[3] All learning assessments used are anchored in a standard deviation of 100 points, this should be sufficient to have the FGT-class of measures to be minimally comparable. Of course, within country temporal comparisons, assuming temporal comparability of the assessments, are the ideal case. All gap measures are relative to the test specific MPL. One interesting aspect is that once the gap conversion is made, the measure becomes test independent, and can be presented independently of any scale. One important assumption however, which is shared with the global poverty monitoring, when doing cross country comparisons, is that the learning (income) marginal sensitivity of the cardinal variable are the same. Meaning, improving one learning point, is equality hard (or equality well captured) across all assessments (or different measures of income and consumption).