Peer review is seen as the gold standard; it means the research community can trust it’s the best. However, it also has its limitations, which is why UK Research and Innovation (UKRI) conducted a review to establish an evidence base for interventions that could make processes more efficient, more relevant, better quality and more diverse in their inclusion of types of project and researchers funded.
Read more on our response to the review of peer review.
Current systems are resource heavy
Growing numbers of applications mean increased workload and pressure on the assessor pool. Gone are the days when you could get a 100% hit rate to requests for reviewers meaning many more invitations to secure the required number. And for reviewers completing a review thoroughly and comprehensively takes a significant amount of time.
The system can act as a buffer to innovation and creativity.
It’s not very good at recognising disruptive thinking, such as breakthrough or transformative projects. On top of that, it can have a tendency to produce more conservative decisions, as well as bias in favour of established names and institutions.
It has a narrow scope of excellence.
This is often based around the potential of papers being cited, rather than reflecting the excellence of wider aims, such as public engagement, commercialisation or spin out companies.
Innovative interventions
UKRI is trialling innovative interventions in the peer review process and building the evidence base. Many of our research councils have run programmes to explore alternative methods.
EPSRC
The Engineering and Physical Sciences Research Council’s (EPSRC) New Horizons Programme aimed to stimulate and challenge the community to come forward with high-risk discovery research proposals. It trialled an anonymous applicant approach, a simplified application and peer reviewing process, and strived for a shorter submission to approval decision time frame.
The New Horizons Programme reported a change in applicant behaviour, with an increased number of applications from females, younger researchers, and Asian or mixed ethnicity groups. Also, panel members felt the anonymity helped to remove bias from the assessment process and enabled greater focus on the assessment of the quality of science. Overall, it achieved a reduction in decision or processing time and was well received by the community, where they particularly appreciated the simpler application process.
Innovate UK
Innovate UK ran an experiment to test whether we could better utilise the assessor pool; specifically, in reducing the number of reviewers per application from five to three. It was shown to result in the same decision 89% of the time.
NERC
A key insight from an implementation of partial randomisation by the Natural Environment Research Council (NERC) is that it saves significant time at panel meetings by removing the need to rank applications that have the same overall score, and which are often very different in scope or science remit.
BBSRC
The Biotechnology and Biological Sciences Research Council’s (BBSRC) use of online discussion boards for peer review received positive feedback from both applicants and panel members. A survey conducted to understand the perceptions around discussion boards’ enhanced robustness, transparency, efficiency of the peer review process. Panel members were even more positive, with 61% stating the boards improved transparency and helped manage workloads more flexibly, and 65% agreeing that the virtual setting aided in providing constructive feedback.
Doing things differently
So, how can we do things differently?
We are working with a new metascience unit to better understand what works in the funding processes undertaken by UKRI. We are also running new experiments in real time in conjunction with colleagues running research programmes across UKRI.
Going for gold
It’s important to stress that we are not loosening our commitment to funding excellent research. To truly reflect our thriving and developing research and innovation system, the peer review process needs to evolve. Changes are designed to save time and reduce the burden on everyone involved, increase transparency and reduce bias, and increase the ability to fund those high-risk, high-reward projects. Peer review will still be the gold standard – just more polished.
Top image: Credit: UKRI
