UNESCO/PASCAL Observatory Webinar 'Learning Cities’ COVID-19 recovery: from research to practice - The challenge of measurement, planning and evaluation’

On 8 July 2020, the UNESCO Institute for Lifelong Learning (UIL), together with PASCAL International Observatory, hosted the second webinar in the ‘Learning cities’ COVID-19 recovery’ series. This most recent event looked at the challenges of planning learning initiatives, measuring their effectiveness and evaluating learning outcomes – particularly with regard to distance learning, which has overtaken more formal, in-person learning engagement during the current pandemic.

On behalf of UIL, Ms Christina Drews welcomed the participants, drawing their attention to the UNESCO Global Network of Learning Cities (GNLC), for which UIL is the Secretariat. As part of its new strategy, the UNESCO GNLC has identified seven topics that cover those issues that its member cities consider to be the most important. One of these topics is also the subject of the webinar: educational planning, monitoring and evaluation (M&E).

The webinar’s moderator, Dr Catherine Lido, Professor of Psychology and Adult Learning in the School of Education, University of Glasgow, and Associate Director at Urban Big Data Centre (UBDC) and Deputy Director of the PASCAL Observatory, then introduced the aims of the webinar and her work at UBDC, where new indicator frameworks are being developed to evaluate learning city initiatives. Dr Lido leads the Educational Disadvantage and Place team at UBDC, which looks at learning engagement, from primary school through to higher education and employment. One of the main remits of UBDC, and this webinar, she said, was to exchange knowledge on how we can more holistically explore the learning that goes on in cities, particularly as such learning moves increasingly online in a post-COVID world.

Ms Candy Lugaz, Project Coordinator, UNESCO International Institute for Educational Planning (IIEP-UNESCO)

Ms Candy Lugaz, Project Coordinator of IIEP’s research on ‘Cities and Education 2030’, presented ‘Planning for achieving SDG 4 at city level’, in which she shared findings from a recent IIEP-UNESCO research project conducted in four city regions in France. As part of the project, interviews were conducted in schools, and socio-cultural and youth centres in order to develop a city education strategy that addressed the needs of the local community. The strategy called for strong partnerships and a transversal work approach in order to overcome obstacles (indicators, resources and even steering committee issues) so that effective qualitative and quantitative evaluation tools could be implemented.

Mr Lee Jae-Jun, Mayor of the UNESCO learning city of Goyang, Republic of Korea

Mr Lee Jae-Jun, Mayor of UNESCO GNLC member Goyang City, Republic of Korea, which is the coordinating city for the network’s thematic cluster on educational planning, monitoring and evaluation, presented Goyang’s roadmap to becoming a learning society. Through a combination of methods – forums, surveys, workshops and in-depth interviews with learners and experts, the city has begun to rethink conventional approaches to lifelong learning in favour of concepts such as ‘learning for growth’, ‘learning for everyone’, ‘making learning every day’, and ‘learning to live together’. The city aims to develop an accessible, diverse and sustainable lifelong learning strategy with associated indicators and six potential models for learning city success.

Mr Sebutege Ange, Mayor of Huye, Rwanda

Mr Sebutege Ange, Mayor of PASCAL Learning City Network member Huye, Rwanda, opened his presentation with an introduction to the city’s planning frameworks in terms of meeting the UN Sustainable Development Goals (SDGs) and realizing the Africa 2050 vision, including key interventions to date, including the construction and redevelopment of education and research facilities, establishment of an integrated cultural development centre, and the promotion of information and communication technologies (ICTs) in education.

Mayor Ange then introduced Dr Pierre Claver, Professor at the University of Rwanda, and Josephine Malonza, Assistant Lecturer at the Centre for Sustainable, Healthy and Learning Cities and Neighbourhoods (SHLC), who spoke about monitoring and evaluating competence-based curricula (namely, their M&E Framework), integrating monitoring from higher level district and council levels down to individual peer-to-peer reviews. Ms Malonza also introduced the work of the SHLC, whose work in Huye includes developing large-scale surveys at neighbourhood level to better meet the educational needs of the community. These surveys offer a comprehensive audit of the housing and living conditions in the area, public service provision and related sustainability indicators in diverse (planned, unplanned and mixed) contexts. Such work is supplemented with key actor interviews and focus groups with neighbourhood workers (e.g. personnel from neighbourhood committees, social workers, estate managers and homeowners).

Ms Angélica Patricia Alvarado Juárez, Mayor of the UNESCO learning city of Huejotzingo, Mexico 

Mayor Alvarado Juárez of Huejotzingo, Mexico, also a coordinating city of the UNESCO GNLC’s thematic cluster on educational planning, monitoring and evaluation, gave an overview of the educational reforms taking place as part of Mexico’s ‘New School Model’. She emphasized Huejotzingo’s learning city agenda, which calls for a reduction of disperse/isolated efforts conducted by different actors within the city (the goal is to bring together actors in a common institutional effort) in favour of standardizing evaluation, to meet the needs of all learners nationwide. Their vision of monitoring and evaluation is one in which all educational sectors and stakeholders work collaboratively. Key examples of the city’s successes here include the creation of digital platforms for engaging English teachers in pedagogical upskilling. Ms Alvarado Juárez called for more regulatory frameworks and collaborative approaches for monitoring not only formal, but also non-formal and increasingly online learning opportunities in the city and country-wide.

Debate

Engagement of local actors in the different stages of planning

By highlighting that consultation was the most important aspect of IIEP’s work, Ms Lugaz noted that the myriad changes occurring throughout any city at any one time need to be analysed concurrently, therefore necessitating consultation with families and teachers as a crucial feedback mechanism for the development of meaningful policies.

Recommendations to cities recently embarking on a lifelong learning approach in terms of monitoring and evaluation mechanisms

Mayor Jae-Jun of Goyang explained that mechanisms that work in one city may not work in another; what matters instead is an understanding and analysis of the local circumstances. Sustainability and time series analysis (wherein a series of data points is indexed in time order) are crucial for keeping track of the changes over time, he explained, but both quantitative and qualitative data strands are necessary to gain a better understanding of the complexity involved in monitoring and evaluating the impact of lifelong learning policies over time in a given city. The challenge, he said, was in keeping track of the effectiveness or challenges entailed in program implementation. ‘Time series analysis is key if you want to see the changes in your cities across time,’ he told participants, ‘a combination of both qualitative and quantitative approaches is very important for this exercise.’

Media and information literacy (MIL) – a prerequisite for future lifelong learning cities

According to Mayor Jae-Jun, MIL is key for developing robust learning cities and for lifelong learning more generally. Co-creation is also a must – people, he said, must feel included in the implementation of policies to feel they are empowered owners of their city’s policies; they should be thought of as ‘co-creation members’.

Creation of planning and monitoring indicators supporting the effectiveness of education policies; community engagement

For the City of Huye, Mayor Ange stated that the SDGs provide a clear vision of what the city wants to achieve and that it therefore developed a seven-year plan outlining relevant strategies in consultation with different stakeholders such as the City Advisory Committee and the University of Rwanda. Community engagement is both Huye’s goal and its vision, Mayor Ange continued. Ms Malonza added that the university is committed to engaging with all communities throughout the city.

Sharing of best practice across cities

The Mayor of Huye responded that sharing best practice is dependent on the creation of strong networks at local and global levels. ‘Digital mechanisms are also crucial for sharing best practice,’ he continued. ‘For example, webinars or uploaded information for others to access has enabled Huye to share its vision with other cities and the nation.’

Challenges in building meaningful indicators; Indicators to keep track of learning city development

Ms Angélica Alvarado Juárez, Mayor of Huejotzingo, listed several challenges, including municipalities having limited ability to implement education policies. However, through the learning city approach, local governments are able to play a more active role, with a wider array of stakeholders able to provide non-formal education opportunities. The mayor stated that ‘civic participation was a determinant for Huejotzingo’s learning city vision, particularly because the municipality has limited participation in the planning and monitoring processes of educational policies.’

Communication of governments’ objectives to the public and long-term engagement

Mayor Juárez explained that Huejotzingo communicates its goals through social media/networks and also through the creation of neighbourhood committees. Over 2,400 private companies are active in the city, and they are invited to participate in learning city initiatives along with teachers and NGOs. In this way, Huejotzingo has established an agenda aligned to the 2030 Agenda to ensure commitment in the longer term.

Concluding remarks

Michael Osborne, Professor of Adult and Lifelong Learning and Director of Research within the School of Education, University of Glasgow, and Director of PASCAL, provided the concluding remarks, highlighting IIEP-UNESCO’s development of procedures for the systematic planning at urban level. In addition, the cities of Goyang and Huye demonstrated examples of their planning frameworks and strategic visions. Notably in their presentations was a focus on the determinant role played by neighbourhoods in which citizens are active in lifelong learning activities organized by city authorities.

Finally, taking the discussions of the webinar as a whole, Prof. Osborne offered recommendations for the learning cities networks of UIL and PASCAL, calling for them to:

• systematically review the range of indicators being used for assessing learning cities’ success;
• better operationalize indicators using simple, valid and reliable instruments;
• create inexpensive mechanisms to generate data and measure progress.

Reflecting on the webinar, Ms Catherine Lido acknowledged the power of harnessing novel forms of data to address educational inequalities. Webinars are an ideal venue for global knowledge exchange at the national, regional and community levels, she said. The presentations and interactive Q&A allowed for a broader picture of learning cities. Ms Lido concluded the session by thanking all participants, presenters and UIL, and encouraged online participants to view, review and pose further questions on the UIL and UBDC YouTube channels. 

The full webinar series details can be accessed here: https://uil.unesco.org/event/new-webinar-series-learning-cities-covid-19-recovery-research-practice

The summary was prepared by Catherine Lido, University of Glasgow/PASCAL Observatory, with support from Sergio Hernandez Mendoza, University of Glasgow.