Traditional ways of measuring educational proficiency, based on assessment results and the percentage of children enrolled in schooling, may not be as effective a measurement while COVID-19 impacts upon attendance rates.
Analysts working with SDG indicators on learning proficiency, in particular Indicator 4.1.1, would in many cases be familiar with a formula such as the following:
The percentage of enrolled learners who are proficient, S, is typically known from assessment results. The percentage of children of the age of the grade in question who are not enrolled, N, is typically known from household data. P is what planners are ultimately interested in: the percentage of the relevant age cohort in the population who are proficient. While there is some ambiguity over whether Indicator 4.1.1 refers to S or P, P is clearly important and must be monitored.
The above formula assumes that non-enrolled children do not reach minimum proficiency levels, following the approach taken in UIS (2020) and UIS (2017). The relationships can be illustrated as in the following graph, which is based loosely on levels of learning proficiency and the out-of-school problem in developing countries. Here P is a function of S and N, according to the above formula.
In the context of COVID-19, sudden declines in enrolment, meaning N rises, and declines in proficiency in the population due to disruptions in schools, meaning P drops, are possible. In examining the effects on the three statistics P, N and S, it is important to remember that N and S influence each other. Assuming that children who drop out of the schooling system are the academically weakest learners, an assumption that is likely to hold for a number of reasons, one can expect S either to rise or decline, depending on the magnitudes of the two effects: dropping out of school and learning losses. If one has estimates of P and N, then the new S is calculated according to the following transformation of the previous formula:
To illustrate the ambiguity, we can imagine learning losses resulting in a drop in P from the 35% seen in the graph to 33%. If out-of-school N increases from 5% to 9%, S declines from 37% to 36%. However, if N displays a larger increase, from 5% to 13%, then S displays an increase from 37% to 38%. Even with learning losses in the population of a specific age, if enough learners drop out, and we assume that these learners are those who struggled most academically, then assessment systems may in fact detect an increase in the percentage of proficient learners.
Clearly, planners need to be fully aware of these relationships. Above all, the first formula above should not be used to conclude that more out-of-school children on its own produces a decline in P. If S was 37% (as in the graph) and then out-of-school N increases from 5% to 13%, one cannot conclude that P drops to 32%. This would ignore the fact that if N changes, S automatically changes too.
The UIS will soon be releasing its projections of the impact of COVID-19 on the learning proficiency of children and youth, building upon previous UIS projections published in early 2020, before the pandemic. Projections of this kind are vital and inform the planning conducted by national governments, global bodies, and development assistance organizations.
The data suggest that the impact of the pandemic on learning occurs through two distinct channels:
Disruption in schooling has denied learners their usual classroom experience with teachers and other children. Disruptions have been particularly serious for younger children, who are most in need of contact teaching, and often receive meals through the school. These needs are felt especially in developing countries.
The economic shock of the pandemic has put education budgets under pressure, and worsened household poverty. These factors also impact negatively on learning.
The findings suggest that the effects of school disruptions on learning proficiency could be felt for many years, even after the pandemic is declared over. In future years, however, as children who did not directly experience the disruptions enter the education system, the longer-term budgetary and poverty effects will be of greatest significance. It is important for planners both to mitigate the effects of the disruptions, and to plan for the educational well-being of future age cohorts of children not affected directly by the disruptions.
Current focus
The following diagram outlines the variety of factors influencing any schooling system’s ability to minimize learning losses:
This illustrates how future priorities depend to a large degree on the strengths and weaknesses of the schooling system that existed before the pandemic. Systems with effective support and accountability structures, and which had been experiencing improvements in learning proficiency before 2020, are likely to be the most resilient to the shock of the pandemic.
Data for Sustainable Development 