Rosetta Stone Project – Technical Cooperation Group on the Indicators for SDG 4

The objective of the Rosetta Stone to link together assessments, which have been administered in the recent past, to build concordance tables to compare their outcomes to and benchmark national results to those of the regional assessments. This method enables countries to measure SDG 4 indicator 4.1.1.

In a first effort to implement this approach and establish concordance tables, the regional assessments of ERCE and PASEC are linked to two international assessments of IEA, namely TIMSS (for mathematics) and PIRLS (for reading). All countries participating to these regional assessments will subsequently be able to freely use concordance tables and benchmark their national results to those of the regional assessments.

Events

Publications

ERCE

PASEC

On 27 June 2022, policymakers and stakeholders were invited to learn how Rosetta Stone can help compare national assessments to regional assessment programmes, ensuring that countries report on SDG4 global indicator 4.1.1 using adequate data.

It was an opportunity to hear from regional assessment programmes that participated in the study - Programme d’analyse des système éducatif de la CONFEMEN (PASEC) and Estudio Regional Comparativo y Explicativo (ERCE), managed by the Regional Bureau for Education for Latin America and the Caribbean (OREALC/UNESCO Santiago). Countries who used Rosetta Stone shared their experiences, challenges and outcomes.

If you missed the event, see the material below.

Concept note

Presentations

Rosetta Stone: Linking assessment programmes for reporting of SDG 4.1.1
- Silvia Montoya (UNESCO Institute for Statistics)
Establishing a Concordance between Regional Assessments and TIMSS/PIRLS
- Lale Khorramdel and Matthias von Divier (TIMSS & PIRLS International Study Center, Boston College)
IEA's Rosetta Stone: Project and Implementation
- Oliver Neuschmidt (IEA)
ERCE 2019 and Rosetta Stone Project
- Carlos Cofre Cayuman (LLECE)
Rosetta Project: PASEC Experience
- Hilaire Hounkpodote (CONFEMEN - PASEC)

Policy Brief: Establishing a Concordance Between Regional (ERCE/PASEC) and International (TIMSS/PIRLS) Assessments

This document presents an executive summary of the first results of the Rosetta Stone Study, consisting of the establishment of a concordance table that projects the score distributions estimated from two regional assessments to distributions on TIMSS and PIRLS. These assessments are UNESCO’s Regional Comparative and Explanatory Study (ERCE; Estudio Regional Comparativo y Explicativo) in Latin America and Caribbean countries and the Programme for the Analysis of Education Systems (PASEC; Programme d’Analyse des Systèmes Éducatifs de la CONFEMEN) in francophone sub-Saharan African countries.

Analytical Reports

Establishing a Concordance between ERCE and TIMSS/PIRLS

Establishing a Concordance between PASEC and TIMSS/PIRLS

ERCE - Regional Comparative and Explanatory Study

ERCE is a regional assessment conducted in 19 countries of Latin America and the Caribbean. It measures students’ achievement in the sixth grade.¹ The content of ERCE is expected to align well with the TIMSS fourth grade assessments in numeracy and mathematics; and the reading component of ERCE is expected to align with the PIRLS fourth grade assessment in literacy and reading comprehension.

The overarching goal is to construct a concordance table in which the scores from ERCE in mathematics and reading equates the scores from TIMSS and PIRLS, respectively. The concordance tables represent the “Rosetta Stone”, providing a translation between countries’ ERCE results and the TIMSS and PIRLS achievement scales. Countries participating in ERCE will be able to use the tables to determine the percentage of their students expected to reach the TIMSS and PIRLS international benchmarks or any other benchmarks which can be measured on the TIMSS and PIRLS scales.

A implementation of the Rosetta Stone methodology was conducted by specialists from UNESCO Santiago, IEA Hamburg and Boston College in Columbia and Guatemala.

1 ERCE also measures achievement in lower grades. However the assessed grade level is different (3^rd grade for ERCE).

PASEC - Programme d'analyse des systèmes éducatifs de la confemen

PASEC is also a regional assessment conducted in approximately fifteen countries mainly located in Africa. For the purpose of developing concordance tables with PASEC, testing instruments from PASEC 2019 will be administered in three African countries, in Senegal, Guinea and Burundi, together with those used to create the Rosetta Stone concordance tables, TIMSS and PIRLS, in March and April 2020. Based on the results from the preparatory implementation in Senegal, the most appropriate testing instruments related to the IEA tests (TIMSS and PIRLS) will be determined for the main data collection phase. This phase will consist of the re-administration of all the test instruments from PASEC 2019 in conjunction with a separate test session for TIMSS and PIRLS, the Rosetta Stone instruments. It will be implemented in approximately 100 schools per countries, generating data for approximately 2300 to 2500 students per country.

The data collected from the administration of PASEC assessments will be entered, cleaned, scaled and weighted by the PASEC team, using the same procedures as those used for PASEC 2019. IEA and the TIMSS & PIRLS International Study Center at Boston College will then take on the data analysis and will apply necessary procedures for the Item Response Theory (IRT) to establish the links between the evaluations.