Methodology for MRC portfolio analysis
The Medical Research Council (MRC) reports widely on the breadth and outcomes of MRC-funded research. Information on the MRC award portfolio and its impact appears in government and other publicly available reports, news articles, our blog and other MRC publications.
In this section, we explain the methodologies and approaches we use to collect, assess and present data about MRC-funded programmes and their outputs. In general, the majority of data analysis methodology follows similar approaches using two main datasets:
- grant management data – all grant and financial data on awards made by MRC is accessed via dedicated grant management systems. All requests for awards made via a specific board, panel, scheme or initiative, including all expenditure and commitment data, is handled via MRC Information and Analysis team
- researchfish® – the majority of data on outputs and impact assessment are obtained via MRC researchfish® database.
In addition, MRC gathers information for analysis or validation from other sources, including citations and bibliometric or citation data.
We use a range of classification systems, such as the Health Research Classification System (HRCS), to define portfolios of awards associated with certain research themes or areas within the larger ‘all MRC’ portfolio.
Portfolio analysis
In general, our reports focus on MRC as a whole: reporting across all MRC-funded research since 2006 or in a given period. However we may also wish to report on specific research topics or dedicated award schemes. In these cases, we provide quantitative data from this sub-set of awards, which is commonly referred to as an award portfolio.
Portfolios are defined by disease area, by academic field or by scheme objective through use of bespoke search criteria. These can be MESH or other text analysis terms or the UK Clinical Research Collaboration (UKCRC) HRCS.
Health Research Classification System
MRC uses the Health Research Classification System (HRCS) to classify the main focus of all awards. The HRCS follows a two dimensional framework, with coding from both HRCS dimensions are applied to each award:
- the research activities classify the type of research activity being undertaken (from basic to applied). There are eight groups: Underpinning, Aetiology, Prevention, Detection and Diagnosis, Treatment Development, Treatment Evaluation, Disease Management and Health Services. Each group has from four to nine subcodes (48 in total) and each award typically has one or two research activity codes applied (up to four for large or complex awards)
- the Health Categories are used to classify the type of health or disease being studied. There are 21 categories encompassing all diseases, conditions and areas of health research and each award can have up to five health categories assigned.
In the total MRC portfolio, the majority (60 to 70%) of the awards are classified as discovery science by their HRCS coding to Aetiology or Underpinning, or both.
Most of the remainder of the awards are focused on research into Treatment Development, Detection and Diagnosis, Treatment Evaluation or Prevention.
Portfolios of MRC awards can be identified by HRCS classification (for example all awards associated with a given ‘research activity’) or by MRC funding initiative (for example Stratified Medicine) or by combining the two grouping systems, for example Prevention and National Prevention Research Initiative (NPRI) awards.
researchfish®
Since 2009 researchers receiving funding from any MRC initiative are required to annually report all research outcomes and outputs to researchfish®.
Analysis of researchfish® outputs
Standard approaches to assessing researchfish® data
The reports from researchfish® are used across a number of different MRC reports and while there will be some differences in how that data are displayed, there are a number of key principles which remain constant across all reports:
- all figures reported are validated (if possible) and, when using aggregated data, we present ‘unique’ numbers of de-duplicated outputs to best reflect the available data
- percentages generated from researchfish® data for MRC reports are rounded up or down to the nearest whole number. Therefore, some may appear as zero if this represents less than half of one per cent, and not all tables may sum to 100% because of rounding
- where instance of further funding are reported in currencies other than Pounds Sterling the values are converted using an average exchange rate for each calendar year as reported on OANDA
- ‘new’ awards refer to new commitments made within the specified period, whereas ‘active’ awards refers to all awards still active (meaning those incurring spend) within the specified period.
Analysis of individual output types
Publications
In the grant level analysis, where more than one grant claims to have contributed to a publication, each is credited equally in the analysis. In this case, several thousand publications are counted multiple times in the All MRC portfolio. In the analysis of the aggregated outputs of the portfolio, the publications have been de-duplicated through the use of unique identifiers (PubMed ID, DOI or other identifier) for each publication.
It is important to note that researchers are asked to indicate only the year (as opposed to the month and the year) of their outputs. As the grant may begin at any point during the year, outcomes identified as occurring within the first year of the grant may have occurred within one month of its start date if the grant begins in December or within almost two years if the grant begins in January and the outcome reported does not occur until December of the following year (for example January 2012 – December 2013).
Open access: MRC has an open access policy that requires electronic copies of any research papers, reviews, or conference proceedings that have been accepted for publication in a peer-reviewed journal to be made freely available from Europe PubMed Central. This policy applies to research that is supported in whole or in part by MRC funding. All deposited publications must be made freely accessible from Europe PMC as soon as possible or at least within six months of the journal publisher’s official date of final publication.
Due to time lags in publishing, ID assignment and Europe PMC processing, one would expect lower absolute numbers of publications and proportional compliance in the most recent year, and that these would increase with the next data gathering period.
We will work with Europe PMC to obtain further information about whether these papers were openly accessible within six months of publication, and to filter our results with respect to publication types that have to comply with the open access policy.
Collaboration
Principal Investigators were asked to provide information about their collaborators. These responses were then coded for the country and sector (public, private, etc.) of the collaborator to allow an analysis of the number of international MRC Collaborations. The frequency is indicative of collaborations, not collaborators. If three different MRC researchers report collaborations that would be counted three times even if the partner were the same in all collaborations.
Further funding
Different areas of science have different costs associated with them and therefore both the scale and diversity of external funding are of interest. To accommodate these two factors, the analysis is broken down into two parts: instances of Further Funding and value of that funding.
If a Principle Investigator reported two instances of Further Funding with the same funder, for the same amount, with the same start and end dates, it was assumed that this was double reporting and only one of those two instances was counted. Many researchers reported receiving further funding from MRC but upon further investigation, the overwhelming number of these reports related to the project’s core funding rather than Further Funding. Therefore, all instances of further funding reported as having been made by MRC were excluded from this analysis, including renewal of MRC grants included in this analysis.
The estimated amount of Further Funding spent is calculated using the total amount of Further Funding reported and pro-rating the funding for the period covered by DGP8 (2006-2016) and when the grant was active. For example, if an investigator reported £100,000 of funding from 1 April 2015 until 1 April 2017, it is estimated that 50% of this grant, or £50,000, will have been spent by January 2016.
The Further Funding by country of funder is represented on the maps by circles and each circle’s size represents the amount for Further Funding reported from each particular country. Global Further Funding sources identified as Global are also listed. The scale is noted at the bottom left of each map.
Next destination
The next destination section requests that the Principle Investigator of a research project report on the next employment role and the sector of the next employer of team members on leaving the research group.
Research materials
Research materials include databases, data analysis techniques, cell lines, in vitro and in vivo models of mechanisms or symptoms, and new equipment created as a result of research. Such materials have considerable potential for reuse by other researchers in future applications and are therefore a highly beneficial output of MRC-funded research.
These outputs were reported as a single output type in researchfish® until 2013. All products were ‘Research Materials’ until 2013 when the new subdivisions of ‘Databases and Models’, ‘Software and Technical Products’, and ‘Tools and Methods’ were created.
Bibliometric and citation data
Publication outputs reported in researchfish® are verified through PubMed Central and the aggregated list of unique publication identification numbers are sent for bibliometric analysis to a commercial provider. Citation impact data, normalised by year and subject area, for all publications associated with MRC funding have historically been provided by either Elsevier or Clarivate Analytics.
Elsevier provided a FWCI (field-weighted citation index) for each publication reported by an MRC researcher. The FWCI divides the number of citations received by a publication by the average number of citations received by publications in the same field, of the same type, and published in the same year. The indicator is always defined with a world average baseline of 1.0. For the calculation of subject area, or field of research, for the FWCI, a scheme encompassing more than 300 subjects based on Scopus journal classification has been used.
Clarivate provided a field-normalised citation impact (NCIf) score for each publication reported by an MRC researcher. The NCIf accounts for both the field of research and the year of publication in the analysis. Therefore, a NCIf score of 1 is considered the global average for publications in a given field and year.
Last updated: 27 August 2024