External evaluation 2002

2] The CSI endeavour therefore was an opportunity to develop an integrated coastal management approach as well as an avenue for piloting intersectorality within UNESCO. These two interlinked aspects of CSI each posed significant challenges in their own right. This duality of CSI is reflected in the Terms of Reference for the evaluation and in this report.

3] The evaluation was undertaken by a team of three experts, Ms Dominique Benzaken (Australia), Dr Magnus Ngoile (Tanzania) and Professor Arno Schmid (Germany). Collectively, the evaluation team combined expertise in the social sciences, marine sciences and ecology, integrated coastal and natural resource management, and cultural and landscape planning. The evaluation was undertaken in accordance with UNESCO guidelines for programme evaluation. The evaluation consisted of a desk review of CSI documentation, semi-structured discussions with UNESCO staff at Headquarters and in selected Field Offices, visits to selected Field Projects and discussions with local stakeholders and a small pilot survey of recipients of the Internet-based Wise Practices Forum. The information was analysed and interpreted based on the Terms of Reference schedule of questions.

4] Since the inception of CSI, its objectives and activities have closely followed the mandate and programme of action of governing bodies and in particular those of Rio and Barbados and Barbados +5. CSI has taken a thematic approach to address priority issues of coastal regions and small islands focusing on geography and key themes for action. As we approach the key milestones of Rio +10 (2002) and Barbados +10 (2004), CSI will have a critical role to play, both in evaluation and the determination of future directions.

5] Over the last five years, CSI has developed and implemented an integrated interdisciplinary model for developing, elaborating and testing Wise Coastal Practices using three interactive modalities: Field Projects, University Chairs and an Internet-based Wise Coastal Practices Forum (user name csi, password wise).

6] Although the evaluation team visited only a small number of Field Projects, UNESCO University Chairs and UNESCO Field Offices, and participated in two regional fora, evidence shows that there was overall support for the CSI approach. Factors contributing to this support included the provision of alternative livelihoods, training opportunities, technical support through University Chairs, participation of local institutions in project design and implementation and avenues for regional networking. CSI’s facilitation role in bringing partners and accessing resources, providing avenues for inter- and intra-regional fora and building capacity was seen as critical by those beneficiaries. The adoption of wise coastal practices beyond the pilot stage however requires resources, which are beyond the mandate and capacity of UNESCO, to develop the institutional capacity of the Member States.

7] Finally CSI has devoted significant effort to the documenting of its activities through its website and publications and has to be commended for the quality and diversity of its products.

8] The evaluation team found that, based on its review of the CSI platform design, outcomes and impacts on intended beneficiaries, CSI has been remarkably successful towards the achievement of its stated objective. It is the view of the evaluation team that the design of the CSI platform is an elegant and innovative approach, which could be transferred to a range of environments and situations.

9] CSI has put in place effective processes and networks for consultation to support its planning and programming processes. The CSI network of staff at Headquarters and in Field Offices, Project Leaders and University Chairs have been a key group in shaping the CSI Wise Coastal Practices model and the strategic direction of CSI activities.

10] CSI has placed significant effort on developing a flexible and responsive mode of operation, which has been well supported by those involved. In particular CSI has developed strong links with Field Offices and successfully capitalised on the UNESCO decentralised administrative arrangements to support its planning and programme delivery. This approach has allowed CSI to respond effectively to local needs and emerging issues and to maximise opportunities as they presented themselves.

11] CSI has successfully incorporated into its programme of activities key issues identified at local and global levels in particular the role of local indigenous knowledge and the voice of civil society in a global context. Those issues have led to two projects funded through the UNESCO-wide cross-cutting programme to start in 2002-03.

12] CSI contribution to the preparation of the UNESCO Medium-Term Strategy (1996-2001) (C/4) and Programme and Budget Biennia (C/5) during that period has been the main focus and the primary outputs of CSI planning and programming activities. These strategic documents however are not meant to include the level of specificity required for the implementation of individual programmes and should be complemented by detailed programme plans. The availability of a CSI programme plan, compiled from the extensive preparatory work undertaken by CSI for the preparation of the C/4 and C/5 documents, would have facilitated and enhanced the evaluation. The evaluation team found that the CSI work plan, which was prepared in anticipation of the 1998-99 Programming and Budget Biennium, was most useful and could provide, with adjustments, a good template for the development of a CSI programme plan. The evaluation team recommends in paragraph 32 that such a plan is prepared for the Medium term Strategy (2002-2007).

13] Overall, CSI funds have been allocated primarily to activities in Member States (Field Projects and University Chairs) and decentralised to Field Offices, while funding for networking and communication activities, including the Internet-based Wise Coastal Practices Forum, have been mostly managed from Paris Headquarters. From the information provided, however, it was difficult to assess with accuracy patterns of resource allocation to specified activities between 1996 and 2001. A key strategy of the CSI approach has been to actively pursue the engagement of UNESCO Sectors as well as extra-budgetary funds to support and enhance Field Project activities, beyond the initial CSI seed funding.

14] Analysis of the CSI total budget between 1996-2001 showed an overall decline in regular funds accompanied by a reduction in the amount of decentralised funds. The reduction in decentralised funds is as much a result of the decline in regular funds as it is a reflection of CSI changing priorities and role of Headquarters (e.g. the Internet-based Wise Coastal Practices Forum). The extent of decentralization was found comparable to that of the Science Sector Divisions. The reduction in regular funds has not been compensated by an increase in extra-budgetary funds, which has remained more or less constant over the evaluation period. A summary of CSI budget between 1996 and 2003 is given in Table 10 (paragraph 175).

15] Under these circumstances, it will be critical for CSI to increase its effort in attracting extra-budgetary funds to support its in-country activities over the long term. CSI will need to strengthen its strategy over the next 6 years and focus the priority outcomes and actions to ensure its investment is well targeted.

16] A UNESCO-wide strategic approach to activities in coastal regions and small islands is also required to better define UNESCO priority outcomes and to direct sectoral and intersectoral activities (including CSI). Such a process could be initiated by the College of Assistant Director Generals (ADG) in the context of the cross-cutting programme.

17] CSI management systems, in line with UNESCO-wide systems, have been designed to report on input not on specified outcomes. As a result, they could not provide management information (e.g. expenditure information) at the level of detail and form that would allow for a substantiated analysis of the effectiveness and efficiency of resource use for specified outputs, let alone outcomes, over the evaluation period. The UNESCO-wide proposed shift to outcome-based management and the reform on decentralization should provide the framework and impetus to address these issues. Developing accountable and transparent mechanisms will be a challenge, which will require specialist expertise and resources not currently identified.

18] CSI personnel at Headquarters and in Field Offices were found to be very dedicated and hard working; however, the refocusing of CSI activities as well as new activities over the evaluation period has put a strain on human resources. CSI human resources need to be reviewed to ensure CSI programme demands are met and that staff is able to be effective, productive and appropriately rewarded.

19] Over the last 5 years, CSI has significantly contributed to the development and testing of integrated management practices and has provided UNESCO Sectors with a valuable resource and an avenue for collaborative activities that they may not have had otherwise. As to be expected, all Sectors were found to have activities in coastal regions and small islands. However, few were aimed at integrated management and of an interdisciplinary nature. Given the small number of CSI Field Projects and the current engagement of the various Sectors in those projects, the opportunity for overlap and duplication with other Sectors was not found significant. The main features of CSI value adding to UNESCO standard program delivery are described in Table 18 (paragraph 279).

Features	UNESCO Sectors’ Standard practices	CSI platform value added
Governance	Intergovernmental governing body in some case with a statutory basis (eg WHC IOC) with secretariat in UNESCO and representatives in members states (Membership can be different from General Assembly and can lead to conflict in governance)	No governing structures, instead a Informal, networked, interdisciplinary network In HQ and UNESCO field offices (Small size and absence of formal structure is as much a strength as it is a weakness)
Mandate	Global, sector based with multiple themes and foci Knowledge oriented objectives	Spatially targeted, intersectoral, with one major theme (Sustainable development), Management oriented objectives
Priority setting	Sector based planning process, consultative; Governing bodies set priorities and directions for programmes within global framework	Decentralised planning process, participatory, CSI network set priorities and directions for activities within global framework
Mode of operation Implementation strategies	Focus on knowledge building and integration, networks of expertise, incremental, information rich, technology intensive, Formalised, planned, long term Investigation of technologies for the identification, protection and monitoring of global resources and assets (natural and cultural) for informed decision making, using national and international expert networks and partners Weak link to policy	Focus on problem solving at the grass roots level, social technologies, practical integrated outcomes using existing knowledge Experimental, responsive, flexible Development and implementation of tools for diagnosis, integrated management responses and appropriate technologies for sustainable living, using UNESCO sector competencies and in collaboration with local and national partners Strong link to policy
Activities	Large global thematic programs, long term, some intersectoral activities (Science, education), University Chairs, partnerships with research institutions and other international programs (UN and NGOs) Technology transfer and capacity building, Education, knowledge dissemination through international thematic networks, meetings publications and website primarily.	An integrated system of small relatively short term intersectoral pilot projects (all sectors), regional networks, University Chairs, and internet based forum on integrated management solutions Capacity building and institutional development, Education, information dissemination through internet based forum, meetings, publications, website, and regional networks
Resources	Large human resource infrastructure (HQ/field office) Financial resources limiting	Small decentralised network of CSI specialists (HQ/Field Office) and consultants (remote) Human resource limiting

20] There is however an issue as to where the leadership for the coastal regions and small islands agenda should be. The development of a UNESCO strategic approach for coastal regions and small islands, as mentioned previously, should provide a mechanism to address this issue.

21] UNESCO experience with intersectorality has been difficult and has not as yet been able to significantly change the way it does business, despite numerous attempts over the last ten years. The coordination of major events across Sectors has been the most successful modality to date, but these are not truly intersectoral activities. While, to our knowledge, none of the earlier intersectoral projects were formally evaluated, they were an important input in the deliberations and the report of the 2000 Working Group on Intersectorality. The Working Group, which was set up by the Director General as part of his UNESCO reform process, identified a number of impediments to effective intersectoral collaborations and possible strategies to overcome them. The findings of the report have been used to evaluate CSI performance as an intersectoral endeavour.

22] CSI has been successful in developing strategies to promote intersectorality within UNESCO at Headquarters and in Field Offices. These include building effective networks, fostering an enabling environment and establishing sound principles of engagement for Sectors (e.g. brain sharing, cost sharing and credit sharing) as well as a strong focus on joint problem solving. However, significant structural and administrative impediments remained which have limited the extent to which CSI could pursue its intersectoral mandate. The current hierarchical organization of Sectors, supported by inter-governmental bodies, is probably the single most limiting factor for successful intersectoral activities. Another important limiting factor for CSI has been the lack of clear status within the organizational structure.

23] The recently established UNESCO-wide cross-cutting programme is the first serious attempt to address those structural and administrative impediments. It is notable that CSI has performed well under this new programme. The success of the programme in fostering intersectoral collaborations will have to be carefully monitored and should be evaluated at the end of 2003.

Table 19. Intersectorality: impediments and strategies to address them (adapted from the CSI pre- evaluation study)

Obstacle	Remedy
(i) Rigid vertical structures and hierarchy	Building lateral linkages bottom-up and top-down
(ii) Inappropriate programming	Revitalize new thinking and innovation at the pre-programming phase through networks
(iii) Inefficient coordination due to excess of coordinators	Revisit the role of coordination units and re-assign them
(iv) Insufficient, unreliable project evaluation	Revise evaluation procedures and mechanisms
(v) Inadequate performance evaluation	Revise evaluation procedures and mechanisms
(vi) Exaggerated individualism	Develop team spirit through a global vision and common objectives
(vii) Frequent lack of coincidence between sectoral structures and disciplinary expertise of their staff	Benefit from house-wide available expertise, wherever it is
(viii) Lack of information about available expertise	Develop appropriate information tools
(ix) Overload of routine work at the vertical structures	Simplify outdated administrative routines to leave room for creative thinking
(x) No stimuli to innovative approaches	Encourage networking
(xi) Negative perception of staff members involved in intersectoral work	Change managers attitudes through accountability with regard to intersectorality
(xii) Lack of in-house visibility for team members	Ensure credit-sharing and visibility
(xiii) Poor internal communication, no public spaces for informal communication	Develop new tools, create virtual spaces and a forum of ideas
(xiv) New technologies are not fully operational nor fully used house-wide	Urgently remedy the situation and provide training as appropriate

24] The CSI Platform as a result of its mandate, structure and mode of operation has provided a flexible mechanism for integration of Sectors’ activities in coastal regions and small islands. CSI has defined a niche for UNESCO as a provider of expertise in integrated coastal management practices based on an interdisciplinary approach to problem solving and tested strategies for intersectoral collaborations.

25] These strategies can be replicated to a suite of issues, environments and circumstances, as they are essentially process based. An important lesson to draw from the CSI experience is that collaborations are about people working together. However, unless there is a supporting organizational environment, those cannot be sustained in the long term. Organizational change requires strong leadership and incentives to change. The cross-cutting programme is a significant first step in that direction.

26] Through its activities in Member States, CSI has been able to focus government attention on significant coastal issues, facilitate a broader debate, bring together stakeholders and attract funding toward the integration of Field activities and Wise Coastal Practices into mainstream management systems (some Field Projects have entered this stage). This success however builds on and is dependent upon UNESCO regional structures, such as Field Offices and National Commissions. Success has been most evident where UNESCO had receptive and adequately resourced Field Offices and a supportive National Commission. The reform on decentralization should provide the impetus for better integration of activities at regional level.

27] The issue of adoption and implementation of sustainable coastal management strategies remains a vexed issue, which requires attention in its own right and is beyond the scope and capacity of CSI and possibly UNESCO. There is however an opportunity for UNESCO and CSI to facilitate the development of regional strategies for implementation, building on its existing networks and UNESCO Field Offices. Greater collaborations with United Nations (UN) and bilateral funding agencies at a regional level are a critical, yet relatively unexplored, element of implementation of regional strategies.

28] CSI has communicated about its activities widely and extensively through the website, the Internet-based Wise Coastal Practices Forum, its publications and its presence in member countries. The effectiveness of those communication activities however needs to be evaluated in terms of target audiences, media and messages to identify the best mix of products to achieve the required impacts. An evaluation of communication activities will provide the basis for developing a communication strategy.

29] CSI has attracted significant extra-budgetary funds for a number of its projects, however CSI efforts have been primarily directed at drawing in other UNESCO Sectors and it has not focused as strongly as it should have on generating extra-budgetary funds. Extra-budgetary funds and associated funds are critical to the long-term adoption of wise coastal management practices, given that UNESCO can only provide seed funding. The potential to attract extra-budgetary funds is significant at the regional level and should be formalised in regional strategies. This is critical in the context of the overall decline in regular funds.

30] Opportunities (and risks) for CSI and more broadly for UNESCO in the coming 6 years include:

	Introduction	Activities	Publications	Search
	Wise Practices	Regions	Themes