Context
Research England’s terms and conditions of funding state that higher education providers are required to supply Research England with data to inform allocations of funding and for other purposes. Research England allocations may be amended in the light of any data assurance work carried out with individual providers or other data amendments that Research England agrees to accept.
The aims of the data amendment process are to:
- Improve data quality, by promoting to providers the concept that their final submitted data is the version used, consistent with published data. Post-submission amendments are exceptional and explicitly justified.
- Ensure the most accurate available data is used across Research England which informs allocations of funding and for other purposes.
- Identify amendments that Research England require and ensure that providers make these amendments.
- Ensure that consistent, defensible decisions are made about the use of amendments across Research England.
- Apply policy that is communicated clearly externally and internally.
- Make decisions once for all purposes for the data, for each error. Decisions or recommendations against the criteria will be made separately for each purpose for which the data is used.
Purpose and scope of panel
The purpose of the data amendment process panel is to ensure that rigorous, consistent decisions are made, in line with published guidance, on whether providers’ data should be amended to correct errors that are identified after final submission of the data and to determine how amended data is used.
Research England require data to be of sufficient quality. This will include for the following uses:
- supporting the distribution of funding by Research England
- monitoring the sector as a whole, to understand trends in sector activity, and emerging risks at a sector level and work with the sector to address them
- operating the Knowledge Exchange Framework
- operating the Research Excellence Framework
- supporting the Department for Science, Innovation and Technology (DSIT) and the Department for Education (DfE) to fulfil their overall responsibility for the policy and funding framework in which the sector operates, and other public bodies such as UK Research and Innovation (UKRI) in the delivery of their prescribed functions
Within scope will be various data sources that are managed by or which Research England relies on to deliver its functions. The panel will also consider overrides when data amendment is not appropriate.
Ways of working
Decisions
Recommendations for decisions as to whether amendments for a data collection should be made will be agreed by assessing whether both of the following criteria are met:
- Amendments relate to clear evidence of data error rather than re-interpretation of data (such as re-categorisation).
- Errors are likely to have a material impact on one or more uses of the data by Research England. Therefore, each process that consumes data that can be amended should have thresholds that Research England believes represent material change, and the actions that will be undertaken if amended data breaches these thresholds.
Thresholds in the second criterion may be linked to:
- the errors being widespread: for data not aggregated for the provider (or a department of the provider), this will normally refer to a change to at least either 5% of or 100 instances of a data item within the data return
- the errors being significant in value: an error will normally be considered significant if the change in the data item results in substantive difference to that previously indicated
A substantive difference includes, but is not limited to:
- a date change of greater than two months, or one that results in movement across reporting periods
- a change to a value of more than 10% if the field is a scalar field
- a change to the categorisation (if the field is categorical)
In some cases, an assessment of materiality for the second criterion may require judgement rather than fixed thresholds (dependent on the use of the data). This may include that the error has a disproportionate effect.
Decisions will be made on the following:
- whether to approve the recommendations presented – a recommendation for amendment due to a Research England audit exercise will normally automatically be approved
- the deadline by which the amendment needs to be submitted
- if the data quality is so low as to recommend to the relevant persons that the data be suppressed until an amendment is supplied
Decisions will be one of the following:
- Accept and recalculate outputs relevant to the data, with required time frame for this to happen.
- Accept, recalculate current outputs, and suppress until this amendment is received, signed off and processed. A required time frame for this to happen will also be necessary.
- Accept and incorporate the amended data within all onward uses.
- Reject.
Final decisions will be clearly recorded and communicated in a timely manner to providers.
Responsibilities
The Senior Leadership Team of Research England has overall responsibility for the data amendment process.
The Senior Management Team has been delegated to act as the panel and has responsibility for decisions on whether amendments should be made to correct specified errors. Decisions will only be deferred back to the Senior Leadership Team in the following cases:
- where a material impact of over £1 million in Research England funding has been identified
- where the case is more complex and would benefit from wider discussion and input from this group
Recommendations, including justifications, will be made by experts assessing all the criteria based on different uses of the data.
Members of the Senior Management Team or Senior Leadership Team will be required to state any conflicts of interest relating to individual data amendment cases.
Meetings
Data amendment process panel undertaking will be a standing item at each Senior Management Team meeting. A summary of decisions will be supplied to the next meeting of the Senior Leadership Team.
The Senior Leadership Team may provide advice to the Senior Management Team about interpretation of the criteria, to ensure all decisions about data amendments are clear and consistent.
A record about the reasons behind decisions made by the Senior Leadership Team for complex cases will be made.
Process
Errors in signed off data, identified by providers or others, should be assessed through this data amendment process to determine whether amendments should be made.
Once an amendment has been accepted, signed off by the provider and made available internally, the amended data will be used for all onward uses. Depending on the nature of the data amendment, it is possible that an amendment will be accepted with no outputs needing recalculation, but with the intention of using the amended data for all onward uses.
Any issues raised (for example, an indication of broader data quality issues) during the data amendment process that might affect judgements in work areas, including within governance and assurance, will be relayed to the relevant teams.
Each process that uses data will ensure:
- it can identify if an amendment is material for its purpose
- a process is in place to allow for data suppression (or other handling of data), including if amended data is not submitted and signed off in the expected timeframes
- a process is in place to re-run analysis and outputs with amended data
- a process to incorporate amendments into future iterations
The panel will support the periodic review and comment on the above to ensure consistency between processes.
For amendments that are required, providers will subsequently submit the amended data.
In some cases, differences occur between the changes described in the data amendment report and the changes arising from the data submitted. These changes may be a trivial consequence of the described corrections, others may be more substantial or affect the extent to which amendments meet the criteria (for different uses of the data).
A member of the Senior Management Team (usually the Associate Director of Analysis) will have authority to determine whether amendments made to submitted data should be reconsidered by the panel. This may be done via correspondence.
Providers will be expected to respond to amendment requirements in a timely manner.
Where we have failed to gain assurance over the quality of data, through a data audit or other assurance exercise instigated by Research England, the provider will be asked to submit to the data amendment process to correct data.
We will not usually reconsider a request for amendment unless there has been a material change to the size or nature of the error, or the uses of the data have changed such that decisions about materiality are likely to change.
Complaints
Providers can raise concerns around agreed outcomes with the Research England Governance and Risk Team at email: REGovernance@re.ukri.org
If they are not able to resolve the issue, you can use the UKRI complaints process (see step 2 under ‘How to make a complaint’).