Tens of thousands of studies evaluating government programmes are collecting dust in institutional vaults

Unearthing ‘hidden’ science would help to tackle the world’s biggest problems Sharing them could benefit everyone

Tens of thousands of studies evaluating government programmes are collecting dust in institutional vaults

On 22 and 23 September, world leaders will meet in New York City at an unusual Summit of the Future. Devised by United Nations secretary-general António Guterres, the meeting aims to bolster global efforts to build a better world. Guterres is visibly frustrated by the glacial progress towards achieving the Sustainable Development Goals (SDGs) — the bold commitments made in 2015 to tackle inequality, poverty, climate change and more by 2030. It is now clear that most of the goals will be missed.

A question on the minds of many delegates, including scientists, will be how the world can do better. One answer is to turn to research — and some of what is needed already exists but lies neglected. As things stand, governments and institutions have tens of thousands of ‘evaluation studies’ that were commissioned to assess whether a policy or programme was completed, whether it worked, how much it cost and what barriers got in the way. Few of the commitments that will emerge from next week’s summit will be more useful than an undertaking by all parties to open their vaults and help researchers to synthesize the best of the available evidence. Governments spend trillions on public services each year. Rigorous, independent evaluations offer a robust way for scientists, policymakers and citizens to learn which policies are worth continuing, which should be modified or expanded and which shelved.

Evaluations come in many flavours. Some are impact evaluations, such as randomized controlled trials conducted to measure the success of a programme as it is rolled out — as was the case for an initiative to see whether booster classes improve learning in refugee camps. Others evaluate processes or programmes by reviewing documents, data and interviews once a project is completed. Some are published in academic journals. Many are not.

The most comprehensive database of development impact evaluations, held by the non-profit International Initiative for Impact Evaluation (3ie), contains some 17,000 of these studies. Most have been published in the past 15 years, reflecting the growing pressure on governments and international funders to show that their money is being spent well and making a difference.

Yet transparency is not the norm. Many evaluations remain buried in institutional vaults, inaccessible or unknown to many. Often, the reports are read by just a handful of people before being filed away.

The UN is helping to change this. The Global SDG Synthesis Coalition, formed in 2022, is a group of some 40 UN organizations that want to make better use of the mountain of studies in their filing cabinets. The UN Development Program (UNDP) alone has upwards of 6,600, and there are tens of thousands more across other UN and national agencies, says Isabelle Mercier, who leads the UNDP’s independent evaluation office.

The aim of this coalition is to synthesize UN evaluations and other research to extract lessons on five themes in relation to the SDGs: partnership, people, planet, prosperity and peace. The first synthesis report, on partnership, was published in September 2023 (see go.nature.com/3xr5srk). It drew on several hundred studies to examine prospects for the SDG 17 — how factors such as trade, technology and cooperation could accelerate all of the targets.

One of its findings is that collaborations between low- or middle-income countries are often more fruitful than those between high- and low-income nations. For instance, after Brazil moved from a paper-based to an electronic census in 2010, officials shared their experience and technology on data collection with countries in Latin America, the Caribbean and Africa.

Tracking what works

National governments are also paying more attention to evaluations. In 2021, the UK government launched an Evaluation Task Force to ensure that more spending decisions are based on evidence as to what works. In April, it launched a registry to improve civil servants’ access to hundreds of evaluation reports, and mandated that government departments register studies there. In the United States, the Foundations for Evidence-Based Policymaking Act, which became law in 2019, requires federal agencies to have a designated evaluation officer and associated plans. In 2023, the Australian government set up the Australian Centre for Evaluation (ACE) to inject research into policy.

In the same year, the Inter-American Development Bank and several other major funders signed a Global Evidence Commitment, developed by 3ie, pledging to use and produce more research on which policies work. This momentum is encouraging, but more is needed. At present, countries and institutions tend to conduct and use their own evaluations and evidence syntheses, leading to duplication. Some of those studies lack rigour or fail to capture the lived experience of those affected by policies.

A Global Evidence Report commissioned by the UK Economic and Social Research Council, the country’s main social-sciences research funder, and published on 9 September (see go.nature.com/3twyhsi), called for shared investment into research evidence by the international community. It recommends, for instance, establishing a fund to evaluate programmes and synthesize evidence on pressing topics, such as how to tackle climate change and support ageing populations.

These are good ideas. Nations at next week’s summit should back them and, in so doing, help to reinvigorate global efforts to solve the world’s biggest problems. Better evidence can light the way to a brighter future, faster.

Nature 633, 493 (2024)

doi: https://doi.org/10.1038/d41586-024-02991-5

This story originally appeared on: Nature - Author:furtherReadingSection