Read time: 2 minutes
Date: 02/2026
Steve Hanney (PhD) is Emeritus Professor with the Health Economics Research Group (HERG) at Brunel University of London. For more than three decades, he has studied the best ways to organise health research systems and to assess the impact of their research. He worked with Professor Martin Buxton in the 1990s to develop and apply the well-recognised Payback Framework for assessing health research impact. He has been involved in evaluating research funding programs in the UK, Europe, USA, Canada and Australia.
I currently collaborate with Steve on the evaluation of an Australian research funding program and had this conversation with him about the pathway from research investment to impact. I and asked him to share his knowledge about how research funders can best assess the value of their investment in research.
Top Tips: Professor Steve Hanney – on how funders can effectively evaluate research investment
Mary Haines: What do you think is the key to high-quality, useful evaluations of research funding programs? Can you share a stand-out example that’s made real-world difference?
Steve Hanney: “I think there are two key elements – the first is consulting the research funding organisation about what categories of impact are especially important to them,” Steve says. “Secondly, we would say using a framework to organise the collection, analysis and presentation of the evaluation data is key.”
He gives the example of an evaluation of research funded by the charity Asthma UK that was undertaken by his team. The organisation sought an evaluation using the Payback Framework, which showed that its different types of funding were making impacts at different levels. Research by the organisation’s two professorial chairs was found to have a particularly important impact on improving treatment for asthma, including through the creation of a Centre for Asthma Research jointly funded with the UK Medical Research Council.
“The main societal impacts from projects and fellowships, which we also looked at, came from only a minority of those funded, but they were very important examples,” he says.
Asthma UK was able to use the evaluation findings to inform their future research strategies and promote their research achievements, he says.
Mary Haines: Could you describe your how your evaluation informed the way Asthma UK planned its future research investment?
Steve Hanney: “The Payback Framework basically has two elements. The first is a multi-dimensional categorisation of impacts,” Steve says.
As well as traditional research impact factors such as publications and citations, the framework adds wider societal impacts from research – such as how the research informed health policies, benefits to the healthcare system, increased health equity, increasing satisfaction with the healthcare system and broader economic impacts.
“The other part of the Framework is a model of how to assess the impact, so we looked at various stages in the whole process,” he said.
This includes looking at what happened prior to the research starting – such as how the research agenda was set and whether end-users were involved in discussions – which enables impact to be better attributed to the specific research.
Steve says the Asthma UK evaluation endorsed much of what they were already doing, including with agenda setting, and particularly by showing the jointly-funded Centre for Medical Research had a major impact, which was an incentive to for them to continue on that pathway, with the establishment of a second jointly funded research centre.
“Assessing the impact of research is a really useful way for funders to show that their investments are, indeed, having a beneficial effect.”
– Emeritus Professor Steve Hanney
Mary Haines: So can you tell us about some of the pitfalls that research funders should avoid when assessing the impact of their research investment?
Steve Hanney: “I think the biggest potential problem is to try to conduct the evaluation too soon after the research has been funded – you have to allow a certain length of time for research to make an impact.”
While the process is speeding up and an often-cited figure of 17 years from research to impact isn’t necessarily the case, Steve says anything less than five years is generally too rapid for the impact from most of a research program to have occurred.
He also highlights the pitfall of funders assuming all pieces of research should make an impact.
While funders can expect researchers to consider potential pathways to impact, they shouldn’t go one step further and expect to see impact from all research, he says.
“If you are only funding research where you expect that every piece of research is going to make an impact, you could make a case for saying that you’re not really funding proper research. Because research has to have a degree of uncertainty; you are looking at things where there are questions that need to be answered.”
Mary Haines: So what are your key tips for funders seeking to measure research impact effectively?
Steve Hanney: “So firstly, think about the aims of the research impact assessment,” he says, adding that there are four “As” to consider: Accountability, Advocacy, Allocation and Analysis.
“Then, recognise that not all research should be expected to make a wide societal impact.”
Secondly, he says it is important to consider conducting case studies as part of evaluation, as they can provide in-depth understanding of major examples of impact.
Thirdly, Steve suggests using a framework to organise collection, analysis and consistent presentation of the data. For example, he says the Payback Framework provides the structure to collect and organise data including surveys, interviews and case studies, so that they can clearly illustrate the impact of health research.
Another tip for funders is that rigorous impact assessments, which cover aspects including the monetary value of research, can enable them to make a business case for their funding.
“They think it’s going to be hard to do, but it will benefit them in the long run to sharpen their programs and to galvanise support.”
Mary Haines: So what about the future? Research impact assessment is now seen as a key part of public accountability, so how do you see the field evolving? What about AI?
Steve Hanney: There are lots of new AI tools available to pick up examples of research use in public policies and guidelines. AI will increase the extent to which parts of the process of assessing research impact can be automated.
However, he cautions there may also dangers with using AI.
“It could intensify some of the dangers that always existed in the evaluation field, in that the easiest things to evaluate are simple numbers, which don’t necessarily show what you are interested in.”
It is important to avoid a system where research evaluation simply drives researchers to produce a lot of articles at the expense of producing research that may actually improve lives, he cautions.
“We’ve always used a broad definition of impacts to include the wider societal impact, the impact on policy and on healthcare practice, equity and economic benefits… I think that is something that is now much more widely accepted.”
Interview by Mary Haines, Founder and Director of MH Consulting Group (MHC), a boutique consultancy specialising in research strategy, evaluation and review, strategy and programs, and facilitation. MHC developed the five minutes interview series as a platform for leading professionals to share their know how.
You are welcome to republish this article. Please include the following attribution:
This article was first published by MH Consulting Group: www.mhcgroup.com.au