Be it for the contribution to global health, climate resilience or education: investing in research and innovation is an appealing option for donors looking to promote sustainable development.
But it can be tricky to assess just how well this investment performed and how successful it was. The full benefits of research and innovation often take a long time to materialise – sometimes long after a project is over – and can be hard to quantify.
At the same time, there is value in the knowledge gained. Just like other public goods, it helps other academics and businesses to find solutions to development challenges in different contexts. And we must learn from past undertakings to inform future programmes. In this blog, Marco Antonielli, Principal in our Monitoring, Evaluation, Research & Learning practice, reflects on the question: how can we learn from research and innovation programming in a development aid context?
In 2020-21, I worked on the endline evaluation of the Newton Fund for international research and innovation partnerships. Over the course of the 7-year programme – which was managed by the Department for Business, Energy and Industrial Strategy (BEIS) and worth over £585 million – the Fund supported 18 countries with 2,000 overseas partners in over 5,400 grant-assisted projects. It generated over 5,700 research publications between 2014-21.
My team and I at Tetra Tech evaluated how – and to what extent – the Fund delivered its goal of developing equitable science and innovation partnerships in the 18 partner countries to promote their economic, sustainable development and social welfare.
This experience showed us that evaluating aid funding in regard to research and innovation requires two important shifts in the evaluation practice and in the way the funding is managed.
1. Evaluation methods need to be combined and flexed under an overarching theory-based approach.
Every investment and research programme requires an explicit theory that outlines how it will deliver on the changes we aim to achieve – not just for evaluation purposes. It also helps stakeholders – funders, delivery partners, researchers – understand and demonstrate the progress made towards final impacts, which usually coincide with sustainable development and people’s welfare. As it turned out, consulting stakeholders to produce a Theory of Change was more important than the final product itself while identifying the nature and sequencing of results set the stage for diversifying our evaluation methods.
The combination of different evaluation methods also allowed us to examine different questions. In addition to more traditional evaluation methods, we also used formative and process evaluation methods, for example, to assess value for money of projects. Exploratory methods, where outcomes were not well defined or understood, helped us evaluate the Fund’s benefits to the UK as the donor country. Through this mixture of methods, we were better able to identify and describe the multi-faceted results of the Fund’s activities and uncover stronger evidence of the varied relationships between research and innovation and sustainable development.
2. The criteria of research excellence need to be re-aligned with the purpose of research and innovation in an aid context.
Where judged on scientific values, research excellence is typically defined through peer reviews, citation counts, number of patents or spin-offs. However, in the world of sustainable development, research excellence is more concerned about the translation of research outputs into real-world applications as well as the interactions between an innovation and its socio-economic environment. Drawing from this insight, we closely examined whether research findings were properly communicated to a relevant audience and whether they were likely to influence policy and practice.
Our own evaluation therefore focussed on newly formed research partnerships. Since they need more time to deliver, we assessed how financially sustainable they were beyond the project lifecycle. We also looked at the Fund’s strategic direction from the perspective of sustainable development impacts to find out how a Fund’s portfolio of projects might achieve more than the sum of the parts.
Sure, research and innovation programmes pose distinct challenges for managers and us evaluators alike. But assessing and learning from performance is – and remains – the most important step to helping reap their full dividends. It only takes a little…well, research and innovation.