Quality and learning indicators

Understanding what quality means varies between countries. Different education actors and organizations also have their own definitions. However, most tend to agree on three broad principles: the need for relevance, for equity of access and outcome, and for proper observance of individual rights (UNESCO, 2004).

UNESCO’s framework on the variables of education quality has five dimensions:

1. Learner Characteristics: including learner aptitude, perseverance, readiness for school, prior knowledge, barriers to learning, and demographic variables.
2. Context: including public resources for education, parental support, national standards, labour market demands, socio-cultural and religious factors, peer effects, and time available for schooling and homework.
3. Enabling Inputs: including teaching and learning materials, physical infrastructure and facilities, and human resources.
4. Teaching and Learning: including learning time, teaching methods, assessment, and class size.
5. Outcomes: including skills in literacy and numeracy, values, and life skills.
   (UNESCO, 2004: 36).

The use of indicators

For educational quality and learning outcomes to improve, planners need access to evidence-based analyses of the current situation, trends over time, and information on the strengths and weaknesses of a system, and their causes. A strong monitoring and evaluation system that looks at relevant indicators can provide that evidence. Indicators can help track the progress of strategies and programmes within an education sector plan. Indicators of education quality can have meaningful implications for policy by enabling comparisons to be made across time, within different places or contexts, or against standards or global benchmarks such as Sustainable Development Goal 4 (SDG 4).

More specifically, indicators enable educational planners and decision-makers to:

• Monitor changes in areas such as teaching quality, the curriculum, and student performance, which can alert policy-makers to impending problems.
• Measure the impact of educational reform efforts.
• Encourage an education system to improve by comparing it, or parts of it, with other countries or systems.
• Focus attention on educational subsystems that may require improvement, such as particular districts or levels of education.
• Focus attention on key equity indicators, such as the performance of different subgroups such as girls, students living in poverty, or students with disabilities. (Adapted from Kaagan and Smith, 1985: 24).

Indicators for monitoring education quality

Education systems are typically analysed in terms of context, specific inputs, social or institutional processes, and outputs or outcomes. Indicators can be developed to measure issues that fall under each of these categories.

1. Context indicators: provide information on the contextual factors that affect learning, e.g. student characteristics, socio- economic conditions, cultural aspects, status of the teaching profession, and local community issues. Context indicators are often challenging to develop and measure as they concern qualitative issues. Common data-collection tools include surveys, classroom observations, inspection reports, and self-evaluations.
2. Input indicators: primarily measure the deployment and use of resources to facilitate learning. They reveal whether the planned financial, material, and human resources are being delivered in the planned quantities, at all levels of the system. Information on input indicators is relatively easy to obtain since inputs are often “countable” by nature, and management processes involve keeping records of many inputs automatically. One challenge may be the differences between producing inputs and ensuring that they are available at the endpoint. For example, the textbook/pupil ratio may be measured in terms of the number of textbooks that are delivered, or by the number of textbooks in use in schools. In some cases, there may be a discrepancy between the two figures.
3. Process indicators: measure how educational programme activities were conducted – whether they were carried out to the desired standard of quality. This includes how specific educational processes are conducted in practice, e.g. the application of standards, teaching quality, time on task, school climate, and educational leadership. Like context indicators, process indicators also concern qualitative issues and may be obtained through surveys and pedagogical observations, inspection reports, and self-evaluations.
4. Output indicators: measure the effects of the programme activities to see whether the programme objectives were attained. They reveal how the education system is performing in terms of subject knowledge, competencies, repetition, progression and completion rates, and employer satisfaction. Output indicators may be obtained through national examinations, international assessments, surveys, and systematic field observations. Output indicators typically involve measurement of learning outcomes based on national examinations or international assessments. Output indicators provide the most important data for understanding whether educational quality and learning outcomes are improving as intended.
   (Adapted from: Scheerens, Luyten, and van Ravens, 2011).

Indicators should be based on context and on the specific learning goals of the education system. They should be designed to allow for measurement of change over time and be disaggregated by gender, geography, socio-economic situation, and other equity issues.

The importance of measuring equality

The importance of equality in education is emphasized in the SDGs and Education 2030 Framework, with Target 4.5 aiming to “eliminate gender disparities in education and ensure equal access to all levels of education and vocational training for the vulnerable, including persons with disabilities, indigenous peoples and children in vulnerable situations”. To achieve this, education systems must monitor and address inequalities in access, participation and outcomes for all population groups. In more equitable education systems, learners’ access to education and their learning outcomes are relatively independent of individual socio-economic and cultural circumstances.

To measure equality, planners need to be able to disaggregate assessment data by different population groups in order to track their progress. Learner characteristics that are known to have predictive effects on education outcomes and can serve as key indicators for equality measurement include:

• gender,
• disability,
• forced displacement,
• immigration / migration status,
• cultural, linguistic and ethnic diversity,
• poverty,
• residency.

Collecting data on these indicators enables the aggregation of data by key equity dimensions, such as gender or poverty, and the comparison of the degree of inequality between different subgroups. To measure progress towards equity targets accurately, it is important that learning assessments be administered to disadvantaged children who do not attend school. One means of achieving this is through sample-based household surveys. Household and administrative school data can also be linked to explore the effect of variables such as facilities and teaching methods on disadvantaged learners.