As part of our ongoing transition to the UK Research and Innovation (UKRI) Funding Service, we have updated our approach to review usability. We always take great care to ensure that reviews are appropriate and of value to both applicants and our moderating panels.
In line with UKRI’s guidance on writing good reviews, reviews must be based on evidence, and this should be clearly shown within the comments. Reviewers have sight of and utilise the same published funding opportunity-specific assessment criteria as applicants, which can be found on the funding opportunity page.
Reviewers should:
- familiarise themselves with any specified assessment factors or scoring matrix
- provide comments and recommendations that are consistent with, justify, and explain their scores
- ensure their comments are comprehensive, but concise
- clearly identify strengths and weaknesses of the application in a constructive manner
- raise any concerns in the form of questions for the applicant
- avoid jargon, panellists may not be specialists in reviewers’ fields
- consider other aspects of the proposed research or innovation, like its methodology, experimental design and data management plan, as well as its potential impact (and the pathways to achieving it) including any ethical and social issues
Usable reviews
We are only able to use a review during the assessment process if it meets the following criteria:
- it includes enough information to help panellists and UKRI staff make an informed judgement on the application
- the comments are only based on information that’s included in the application
- the application has not been reviewed negatively because of any equality, diversity and inclusion requirements (for example, decisions to work part-time or past absences for health reasons)
- the comments are not speculative, inflammatory or damaging to applicants
- journal metrics, conference rankings or personal metrics have not been used as a substitute measure for assessing the applicant’s contributions
- the reviewer does not have a conflict of interest with the application and has not revealed their identity (if reviewers are in any doubt as to whether they have a conflict of interest, they are encouraged to contact us via operations@ahrc.ukri.org before accepting and completing a review)
If we feel that there are elements of a review that do not meet our criteria, the review will be marked as ‘unusable’.
Unusable reviews
Reviews may be deemed ‘unusable’ in any of the cases outlined below.
The reviewer explicitly or implicitly identifies themselves within their comments
The applicant and panel are not made aware of the identity of expert reviewers.
Reviewers must not mention their name or any personal information that could identify them within their comments.
Anonymity of expert review is important to ensure that reviewers can express their views freely and openly.
The review lacks sufficient detail
Our decision-making processes rely on expert review of the applications submitted.
To moderate an application effectively, our panels need to have a clear idea of the strengths and weaknesses of an application as identified by reviewers.
Comments should be evidence-based, comprehensive and justified overall, with critical analysis of the application, so that the moderating panel can make an informed decision based solely on the expert reviews and applicant response received.
The reviewer has not addressed the areas expected for effective expert review, and there is little or no critical examination of the application
When writing expert reviews for AHRC, reviewers must utilise the core assessment review criteria which are available within the Funding Service, considering:
- the vision
- the approach
- the applicant and team’s capability to deliver the project
- ethics and responsible research and innovation
- the resources and cost justification
Reviewers should engage with the application in front of them, and not as it could be if changes were to be made.
Reviewers should identify the strengths and weaknesses of an application for the applicant to respond, and for our panels to moderate effectively.
The reviewer’s comments are framed in an offensive manner, or the tone and language used is confrontational or emotive
Reviewers must adhere to the principles and core values within UKRI’s principles of assessment and decision making.
Decisions must be made impartially, fairly and based on merit, using the best evidence and without discrimination.
We want to foster a positive research culture for everyone, by everyone, where everybody is appreciated, respected and valued.
Within this, we require reviews to be balanced and objective.
Therefore, it is important for reviewers to remain constructive, factual and non-confrontational, to provide the applicant with comments that they can respond to effectively.
For most funding opportunities, the project lead will have sight of the expert reviews received as part of the right to reply process.
The commentary on how a project has been conceptualised refers to the protected characteristics of an individual involved in the project
We welcome constructive comments that directly relate to a project, the way in which it has been conceptualised and would be managed.
Comments should focus on whether applicants have appropriate skills, expertise and experience, and whether the project has been designed in an appropriate way to achieve its goals.
Any concerns should be articulated in an objective and evidence-based way.
Our aim is to ensure that the substance of valid academic criticisms remain, while ensuring that applicants are not subject to comment regarding their protected characteristics.
Applicant response to reviews
Applicants will have the opportunity to respond to their reviews at the applicant response stage (if applicable to the funding opportunity), and panellists will read all usable reviews at the moderating panel.
We value academic freedom and want to facilitate productive and constructive assessment.
Therefore, project leads should feel empowered to clarify, rebut and resolve any issues within their applicant response, and we encourage panellists to consider any issues within reviews where necessary at the panel meeting.
If applicants are concerned about reviews that they have received, we encourage them to get in touch with us before they complete their applicant response, so we can investigate.
Please note, we reserve the ability to mark a review as usable or unusable at any time.
Read more about our peer review processes and guidance on writing a good review and scoring.
Last updated: 13 November 2024