Information about projects to help adapt to climate change is scattered, hard-to-find and incomplete, making keeping track of them impossible
More and more people are recognising that the world needs to adjust to climate change as well as cut emissions.
In the 2015 Paris Agreement, countries agreed to set a global goal on adjusting, which is known as adaptation. But it is still very difficult to track and demonstrate progress towards this target because of a lack of rigour in how these projects are officially reported and evaluated.
Two years ago, we at the Stockholm Environment Institute (SEI) set out to create a synthesis of evidence on the effectiveness of adaptation action and support. But the UK government cut its aid budget and our project was one of those to be scrapped as a result.
Beyond funding, however, our analysis was hindered by a widespread and pervasive lack of rigour in adaptation project reporting and evaluation.
Evaluation of adaptation efforts has long faced difficulties. For six years, climate negotiators tried to assess progress towards the Paris Agreement’s imprecisely worded global goal on adaptation.
But, as governments recognised at Cop26 in 2021, they faced “methodological, empirical, conceptual and political challenges”.
In 2021, the two-year Glasgow–Sharm el-Sheikh work programme on the global goal on adaptation aimed to address these.
Lots of adaptation action takes place in the form of discrete project interventions financed by public sources, including directly from governments and from funds that rich countries put money into.
By assessing these interventions in ways similar to those used in development finance, we hoped to obtain a picture of the effectiveness of adaptation action and support, and thus of progress towards the global goal on adaptation.
But we found that data on project outputs and outcomes is not easily accessible or publicly available.
If information is available, it is scattered across multiple sources and fragmented across databases. Moreover, most of this information is only entered into databases at the time the project is approved.
Information about how the project is going is even harder to find, as it is often not made public or remains scattered across multiple locations, from project websites to academic publications.
The lack of a uniform system of adaptation indicators also leads to inconsistencies in how funders monitor, evaluate and report outputs and outcomes of adaptation interventions.
Moreover, baseline data is commonly missing and outcomes are often confused with outputs. Outputs are the tangible and measure results that can be observed in the short term while outcomes are the longer term effects that are expected to be achieved as a result.
Evaluations of adaptation interventions are rare. A recent systematic review of global adaptation research revealed that only 58 out of 1682 articles reported change in climate risk reduction outcomes after implementation.
This suggests that, for now, the UN’s new global stocktake on adaptation must rely chiefly on fragmented and scattered project documentation.
When evaluations are conducted, they tend to emphasise implementation processes rather than outcomes.
A recent assessment of the Least Developed Countries Fund considers mainly how its projects are aligned to reducing vulnerability and increasing resilience, rather than if these objectives have actually been achieved.
Meanwhile, an evaluation of the adaptation portfolio of the Green Climate Fund counted the number of beneficiaries but was unable to assess impact.
Here are four ways to fix this.
1. Create a global adaptation database: A free, comprehensive and easy-to-use global database of adaptation interventions must be established to analyse and synthesise the effectiveness of these interventions. It should include sources of funding, project duration, alternative project names, intervention design, adaptation outcomes and all evaluations. Such a database would complement existing efforts that include some of these elements, such as IATI’s d-portal, SEI’s AidAtlas and Unep’s Adaptation Gap Report.
2. Standardise and improve reporting of adaptation outcomes: The Sharm-El-Sheikh Adaptation Agenda proposes a set of 30 outcomes to report on adaptation progress. These should be used to develop a set of common indicators to be adopted under the new framework for the global goal on adaptation. These indicators should then be used consistently when evaluating and reporting project-level outcomes. For the global stocktake, these indicators must also allow for global aggregation of evaluation results.
3. Invest in rigorous adaptation evaluations: Funders must require and invest in rigorous monitoring and evaluation of and learning from the interventions they support. Project evaluations should be transparent about whether it is possible to attribute an outcome to a specific intervention. Evaluation guidance must include procedures to inform funders and other stakeholders of both intervention successes and failures. Initiatives in this direction include the new evaluation policy of the Adaptation Fund, which includes long-term evaluation.
4. Learn from the development community: Adaptation evaluators should take advantage of existing knowledge of and experience with impact evaluation in development when assessing progress in adaptation. The development community has put forward tried-and-tested methodologies for impact evaluation, and established organisations such as 3ie and the Campbell Collaboration already support, produce and synthesise rigorous evaluation evidence on development effectiveness.
Richard Klein and Biljana Macura are Senior Research Fellows and Nella Canales is a Research Fellow at the Stockholm Environment Institute. Richard Klein leads the team “International Climate Risk and Adaptation”.
