Independent report

Research culture initiatives in the UK: appendices

• From: UKRI
• Published: 8 May 2024

• Print this document or save as a PDF

Authors: Jane Powell, Emily Britton-Drewry, Yolana Pringle, Janet Metcalfe, Neil Jacobs, Katherine East, Lis Grey and Georgia Woollett.

Please cite as: Shift Insight, UK Reproducibility Network & Vitae. Research Culture Initiatives in the UK, 2024.

This work is licensed under a Creative Commons Attribution 4.0 International License.
This work was commissioned and published by UK Research and Innovation (UKRI). The work was produced by Shift Insight, UK Reproducibility Network & Vitae.

Appendix 1: overall methodology

The project was carried out in several stages, with each stage building on the previous one. This was especially true for the development of the research culture framework, which was an ongoing and iterative process that involved consultation with a variety of stakeholders. This appendix details the overall methodology across all phases of the project.

Project phases

Stage A research method agreement and scoping:

• mapping of existing frameworks
• creation of a new framework
• testing with stakeholders

Stage B literature and desk research to gather information about:

• literature review
• desk research
• revision of framework

Stage C direct engagement with the sector:

• call for evidence
• workshops
• additional desk research

Stage D analysis:

• data from workshops
• call for evidence data

Stage E results and conclusions:

• final reporting

Our methods included

Developing a framework to describe research culture initiatives

A framework was developed to provide a nomenclature for the breadth of activities and understanding encompassed by the term ‘research culture’. This was then used to inform the search terms within the literature review, to structure the call for evidence, to categorise examples of practice to improve the research culture and environment, and subsequently to identify any gaps.

The framework was developed using an iterative approach, evolving in conjunction with the literature review, desk research and call for evidence. It was further developed and revised via feedback from key stakeholders in interviews and co-creation workshops.

Conducting a literature review to gather information about activities, initiatives and networks

The project was grounded in a literature review, which gathered evidence about initiatives addressing research culture, and their evaluations. This work was conducted by Lis Grey (University of Bristol) and Katherine East (King’s College, London) for the UKRN. The strategy for this review was developed by research experts UKRN and informed by UKRI and the project team (colleagues from Shift, CRAC-Vitae and UKRN).

The authors used databases which covered an extensive range of disciplines, sectors, research types, and groups of researchers. Insights from the literature review informed other stages of the research and are also reported here as part of our overall findings where appropriate. During this stage, an initial list of activities was also created.

Full details of the approach and findings are given in a brief literature review report, which is provided in Appendix 8: Research culture: a literature review.

Putting out a call for evidence across the R&I sector

Building on data around initiatives collected in the literature review, the call for evidence was designed as an online survey. This was structured around the research culture initiatives framework. Researchers and others in the R&I domain could submit data around their own initiatives or those of others in their sector. Additional questions were asked about the levels of evaluation, collaboration and sharing of activities.

The call for evidence was distributed widely across a range of channels, paying attention to ways of reaching and engaging underrepresented groups.

Identifying and filling gaps in the collected data

During data collection, we used agile processes to identify and address gaps in responses. Members of the Shift team searched for new initiatives in sectors with fewer responses. Shift entered data about known initiatives from the literature review; they also did further research and subsequent data entry where there was missing or incomplete data, to enrich the information provided by researchers.

Iteratively assessing our data in this way meant we could adapt and improve our data collection tactics and directly target groups, sectors and communities for which less data had been submitted.

Further details of our approach are given in Appendix 4: desk research / gap-filling.

Producing and running co-creation workshops with varied research communities, with a focus on overcoming barriers for research culture

We ran four workshops of 2.5 hours, conducted via Zoom, each with between 15 and 21 individuals. The workshops were designed to gain a variety of perspectives from people across the research system. Participants selected included underrepresented communities and diverse opinions.

We asked participants to provide feedback on specific areas of the framework, to discuss and critique existing work in this area, and to develop ideas for a future Good Practice Exchange.

Further details of the agenda for each workshop, recruitment methods and participant profiles are given in Appendix 5: co-creation workshops.

Reporting key findings, with recommendations and considerations to inform future championing of a healthy R&I culture

The project created the following data for analysis:

a) the literature review

b) data on existing initiatives from the call for evidence and desk research

c) transcripts from the workshops and associated Mural boards used for online notetaking and workshop tasks

Quantitative and categorical data collected in the call for evidence was cleaned, tabulated and analysed using Q Research Software. Cross tabulations and filters were created.

The transcripts and literature review were coded using textual analysis tool Atlas.ti. The code frame used aligned with the report structure, as agreed following analysis meetings with the project team. Our approach also allowed for emergent themes and codes. Coding was attentive to the nuance of sector, subject and theme, and related to the earlier developed framework and nomenclature.

Further details of our analysis approach is given in Appendix 7: analysis approach.

Notes on the data

Despite concerted efforts to gather comprehensive data on research culture in the UK, it is important to acknowledge that the findings are not fully representative of all initiatives in this field:

• this report recognises that there is a likely bias towards academia, and underrepresents non-academic initiatives. This may be partially attributed to the data collection process, both in how it was advertised – likely using language and methods more aligned with academia – and because it was being done for UKRI, an institution with strong academic ties
• smaller initiatives might have been less inclined to participate, perhaps due to concerns about their size or perceived eligibility. Alternatively, they may not have heard about it
• however, it is difficult to ascertain exactly how unrepresentative the sample is for either of these metrics

There was substantial variation in what the initiatives submitted in the call for evidence related to:

• of the four sections, how research is managed and undertaken was most varied, in that some behaviours within it had many initiatives submitted relating to them, but others had very few
• there was variation between public, private, and academic sectors, with the public sector standing out at times. For instance, the public sector was more likely to have initiatives relating to open research. The third sector, perhaps unsurprisingly, had far more initiatives relating to accessibility and inclusivity

Appendix 2: research culture framework methodology

Overview

A framework was developed to provide a nomenclature for the breadth of activities and understanding encompassed within the term research culture. It was then used to inform the search terms within the literature review and to structure the call for evidence, to categorise examples of practice to improve the research culture and environment, and subsequently identify any gaps.

Method

The framework is underpinned by the Royal Society’s definition of research culture and has been expressed in terms of appropriate values and behaviours. It has been developed through an iterative approach, initially through review of a range of documents and reports relating to aspects of research culture, such as the Science Europe Values Framework, the range of UK concordats and research culture statements and strategies from a range of 20 UK universities and further informed by the emerging evidence from the literature and desk research.

The term research culture is used extensively and what this term incorporates differs depending on the perspective or ‘lens’ from where research culture is being viewed, for example from a perspective of research integrity or EDI. The framework attempts to bring all these different perspectives together to take a holistic view, thereby maximising the scope and value of the mapping of good practice examples.

Validation

Further development, validation and refinement of the framework was through a series of interviews and correspondence with key stakeholders and networks representing a range of perspectives on research culture. In its early stages, the framework was also informed by, and informed, the literature review (appendix 8). The list of organisations involved is given below. As with other aspects of the project, the engagement of industry with the development of the framework has been challenging.

The framework was then refined further through the discussions within the co-creation workshops and the mapping of the examples of practice from the call for evidence, both of which activities included more industry and third sector involvement.

We are very grateful for the assistance of all those who advised us in this process.

Key stakeholders consulted

Stakeholders include:

• Concordats and Agreements Review
• EDI Office, University of the Arts
• National Centre for Universities and Business (NCUB)
• National Coordinating Centre for Public Engagement (NCCPE)
• National Association of Disabled Staff Networks (NADSN)
• National Association of Disability Practitioners (NADP)
• Race Equality Steering Group, UCL
• Researcher Mental Health Observatory (REMO)
• Researcher Development Concordat Strategy Group
• Research on Research Culture project
• The Royal Society
• UK Committee of Research Integrity (UK CORI)
• UKRI
• UKRI EDI Caucus
• Wellcome
• Women in Academia Support Network

Existing frameworks and statements related to research culture reviewed

Frameworks and statements include:

• A Values Framework for the Organisation of Research, Science Europe (2022)
• Coalition for Advancing Research Assessment (CoARA), initially driven by Science Europe, the European University Association (EUA) and the European Commission
• Research Concordats and Agreements review, Universities UK (2022), which includes DoRA, Leiden Manifesto, Athena Swan Charter, Race Equality Charter, Technician Commitment and seven other related concordats
• Concordat to Support the Career Development of Researchers, Universities UK (2018)
• Concordat to Support Research Integrity, Universities UK (2019)
• University research culture strategies and web pages:
  • Cardiff University
  • Imperial College London
  • London School of Economics and Political Science
  • Middlesex University
  • Newcastle University
  • Queen’s University Belfast
  • University College London
  • University of Aberdeen
  • University of Bath
  • University of Birmingham
  • University of Bristol
  • University of Cambridge
  • The University of Edinburgh
  • University of Glasgow
  • University of Leeds
  • University of Oxford
  • University of Southampton
  • University of St Andrews
  • University of Stirling
  • The University of Warwick

Research culture framework

How research is managed and undertaken

Effective research governance and management

The standards, structures and policies to ensure good research practice, integrity and equity:

• mechanisms to ensure transparent, accountable governance
• implementing effective policies and processes
• providing open, competent and effective research leadership
• providing appropriate, safe and accessible workspaces

Achieving the highest levels of research integrity

Undertaking research with integrity, honesty and rigour to ensure confidence in the methods and results:

• upholding the highest standards of rigour and integrity
• being accountable for all aspects of the research process
• being transparent and honest about all aspects of the research process
• caring and respecting the participants in and beneficiaries of research

Actively promoting sustainability

Minimising the impact of research on environmental, social and economic resources:

• using sustainable approaches to research
• effective use of resources to make the research system accessible to all
• ensuring the efficient use of finances, resources and infrastructure
• investing appropriately in talent and sustainable employment
• considering the impact of research on the environment and people

How research ensures value

Taking an open approach to research

Undertaking research that is openly accessible, collaborative and increases research integrity bringing public value and innovation:

• supporting open, collaborative, interdisciplinary and team science approaches to research
• ensuring research is understandable, explainable, reproducible and accessible
• engaging and partnering with potential beneficiaries
• co-creating and learning with research users and society
• being open, agile and responsive to new technologies and research approaches

Communicating research

Making research and knowledge available and accessible to all:

• connecting with others in accessible and inclusive language and media
• inspiring curiosity and learning
• sharing research, data and other outputs openly
• acknowledging and building on the research and knowledge creation of others
• open to new forms of communication methods and spaces

Realising impact

The translation of research into value for communities, society, culture and economy:

• understanding what value and impact means for different stakeholders
• advancing discovery and driving innovation
• capable of translation and innovation
• contributing to knowledge creation and teaching
• informing policy and practice
• developing a highly-skilled and engaged workforce

How people are supported

Employment and conditions

The recruitment, employment and progression of a diverse research workforce:

• providing transparent, equitable and competency-based recruitment and recognition, recognising diversity
• providing structured and varied progression routes
• providing appropriate remuneration and employment benefits
• ensuring healthy working conditions, accommodations and flexibility based on ongoing needs
• recognising wider contributions to research within job descriptions, workload models and progression criteria
• valuing the full range of experiences, skills and contributions of all who contribute to the research endeavour
• acknowledging and mitigating effects of career breaks and other disruptions, and inequalities

Recognition and assessment

Broadening what is recognised and valued as contributing to the research endeavour:

• valuing research wherever it is undertaken
• broadening the concept of excellence within the system research
• using appropriate qualitative and quantitative assessment methods
• valuing diverse approaches, methods and contributions
• recognising and valuing the diverse range of competencies needed for the research endeavour
• valuing failure and risk-taking as a healthy possibility of research

Embedding professional and career development

Integrating professional and career development into all career stages:

• valuing continued professional development
• addressing development needs at all career stages
• providing a wide range of professional and career development opportunities
• engaging in regular career development reviews
• enabling access to inspiring mentors and role models
• recognising and awareness of diverse career opportunities

Ensuring inclusive and healthy working environments

Environments where all individuals are free to be themselves, including feeling well supported and confident to express their views:

• ensuring the research environment is accessible, inclusive and equitable for all
• embracing and valuing diversity
• fostering psychological safety
• zero tolerance of and taking action against bullying and harassment
• supporting good mental health and wellbeing
• promoting balanced, flexible and achievable workloads

How individuals engage with others

Providing effective leadership and management

The performance and line management of individuals:

• providing responsive and empathetic line management
• providing honest and constructive feedback
• valuing and responding to differences in supporting others
• effective performance management
• being effective role models and mentors

Empowering individuals

Individuals having ownership and responsibility for their own careers:

• clear lines of responsibility, accountability and autonomy
• recognising motivations and ambitions, and facilitating professional visibility
• encouraging a culture of reflection and learning from experience
• enabling creativity and encouraging innovative, imaginative, entrepreneurial mindset
• generating confidence to speak out without repercussions
• encouraging all to invest in their continuing professional development

Building collegiality

The creation of healthy, inclusive, supportive communities:

• creating welcoming and inclusive communities for all
• recognising individual and diverse contributions, advocating for others
• engendering a sense of identity and belonging for all
• recognising that individuals’ behaviours shape cultures
• proving access to networks and communities

Appendix 3: call for evidence: detailed methodology

Background

Building on data around initiatives collected in the literature review, the call for evidence was designed as an online survey, through which those involved could submit data around their own initiatives or those of others in their sector.

Participants could submit information on up to 20 research culture initiatives.

Questions asked in the call for evidence

The data collection tool included approximately 15 questions on each initiative, with participants encouraged to submit multiple initiatives if they could (see Appendix 16: questions asked in the call for evidence). The call for evidence was linked to the agreed framework and nomenclature at that stage of the project. It should be noted that the framework evolved after this point and throughout the duration of the project, so there is no longer an exact match between the call for evidence data points and the current framework for initiatives.

The call for evidence questions were based on an early iteration of the framework. It is important to note that since the data collection phase, a new iteration has been developed, based on feedback and further research, with a slightly changed structure. Consequently, in the analysis and presentation of findings, the categories derived from the old framework have been mapped against the revised structure.

This approach allows for continuity and comparison, and means the existing data can be leveraged while accommodating changes or updates to the framework. However, it should be acknowledged that not all categories from the old framework had a direct equivalent in the new iteration. As a result, some mapping may require interpretation and adjustments to ensure accurate alignment.

For more information on how we mapped the two frameworks, see Appendix 19: framework mappings, used to map the call for evidence data to the final version of the research culture framework.

Engagement methods

We aimed to attract a wide potential group of contributors, with the main sectors to be covered identified broadly as:

• academic (universities, research institutes, funders, networks)
• industry, private sector (research institutes, independent research organisations (IROs), research and development departments, networks, human resources)
• public sector, including public sector research establishments (PSREs)
• third sector, including research-oriented charities and volunteering organisations

To ensure broad participation, the survey was distributed through various channels, including:

• direct communication: personalised emails and LinkedIn messages were sent to approximately 300 individuals in relevant roles. These individuals were identified through the literature review and desk research. They were encouraged to further distribute the call for evidence among their networks.
• LinkedIn advertising: targeted advertising campaigns on LinkedIn were launched, focusing on individuals in relevant roles. These campaigns resulted in over 160,000 impressions and 944 click-throughs.
• social media: the call for evidence was disseminated through UKRI, CRAC-Vitae, UKRN, and Shift social media platforms, including Twitter and LinkedIn, with specific groups targeted in paid LinkedIn advertising. A complete list of targeted groups is shown below.
• JISCM@il ListServs: posts were made on relevant JISCM@il ListServs to reach specific communities and networks.
• presentations: UKRI and the project team delivered presentations at relevant events, creating opportunities to engage potential contributors.

Snowballing was also used in this process, with sharing of the call for evidence encouraged. This method is cited by the UK government as valuable for recruitment from underrepresented groups.

Groups contacted via LinkedIn advertising, by group

The first audience targeted on LinkedIn were UK members of the following LinkedIn groups, identified by LinkedIn group membership:

• AACR Early-career Researcher Network
• Academic Keys: Higher Education Professionals
• Academic Research and Publishing – tips and resources
• Academic Researchers
• Academy of Medical Sciences Grant Awardees
• ADVANCE for Health Information Professionals
• Advance HE
• Centre for Research in Social Sciences and Humanities
• Deloitte and Universities Enabling Together (DUET – a network for women in supply chain)
• EEF Network
• ELRIG
• Equality, Diversity and Inclusion in STEMM Group
• Framework Programme for Research and Innovation
• Generative AI tech products
• Geospatial Insights at Innovate UK KTN
• Higher education and research
• Higher education management
• Horizon Europe
• Inclusive Innovation – Innovate UK
• Innovate UK’s Young Innovators
• Innovate UK EDGE South West
• Innovate UK’s Women in Innovation
• Innovation and entrepreneurship forum
• Innovation and research in IT for Healthcare
• Innovation Management Group
• JISC Webinars Networking Forum
• Knowledge exchange
• London Information & Knowledge Exchange
• Microbiome at Innovate UK KTN
• Neurodiversity in Business (NiB) Community Group
• NIHR Academy – Global Health Members
• NIHR CRN NENC Imaging Network Group
• Open Access Scholarly Publishing Association (OASPA)
• PLOS Communications
• PLOS Open Science Champions
• Publication ethics
• Quantum Technologies at Innovate UK KTN
• R&D Scientist Professional Network
• R&D
• Research Links
• Research Management
• Research Open Access
• Research, Methodology and Statistics in Higher Education
• Responsible Research in Practice
• Robotics & Artificial Intelligence (RAI) at Innovate UK KTN
• Science & Technology Policy and Management
• The Diversity, Equality and Inclusion Challenge
• The Professionalism and Integrity in Research Program (PI Program)
• UKRI TAS Hub

Groups targeted in LinkedIn advertising, by current employer

The second LinkedIn audience targeted was identified by current employer, with the following organisations selected:

• Academy of Medical Sciences
• Advance HE
• AGCAS
• Animal and Plant Health Agency
• Association of Research Managers and Administrators (ARMA UK)
• The Association of the British Pharmaceutical Industry (ABPI)
• Arts and Humanities Research Council (AHRC)
• Arts Council England
• Association of Medical Research Charities
• Association of University Administrators (AUA)
• Black and Brown Academics and Mentorship Program (B.B.A.M.)
• The British Academy
• British Neuroscience Association (BNA)
• The British Psychological Society
• Campaign for Science and Engineering
• Catalyst Editorial Limited
• Center for Research On Publication Ethics (C-ROPE)
• Committee on Publication Ethics (COPE)
• Education Endowment Foundation (EEF)
• Evaluation Support Scotland
• GW4 Alliance
• Health Education England
• Higher Education Academy
• H-Net: Humanities & Social Sciences On-Line
• Innovate UK
• JISC Campus
• The National Centre for Universities and Business (NCUB)
• The National News Agency of Ukraine (UKRINFORM)
• Natural Environment Research Council (NERC)
• Nesta
• NIHR Clinical Research Network
• NIHR Innovation Observatory
• Nonprofit & Voluntary Sector Studies Network
• Novartis
• Public Health England
• Public Library of Science (PLOS)
• Research Councils UK (RCUK)
• Responsible Research in Practice
• The Royal Society
• The Science and Technology Facilities Council
• Scottish Council for Voluntary Organisations (SCVO)
• The Social Mobility Foundation
• Society of Black Academics (SBA)
• University College Union (UCU)
• UK Research and Innovation
• UKRI GCRF Action against Stunting Hub

Appendix 4: desk research / gap-filling

During data collection, we used agile processes to identify and address gaps in responses and Shift team members searched for new initiatives in sectors with fewer responses. They also entered data about initiatives we knew of via the literature review, and enriched information about initiatives we knew of but for which we had incomplete data.

Iteratively assessing our data in this way meant we could adapt and improve our data-collection tactics and directly target groups, sectors and communities that had been less present in submitted data. In particular, we sought additional information around initiatives for industry.

New initiatives in underrepresented areas were found using social media, by phone calls, emails, networking and disseminating information at online and in-person events and conferences. This was carried out by members of Shift Insight, CRAC-Vitae, UKRN and UKRI.

Our focus was on filling gaps in underrepresented areas of the original list, such as the third sector and industry. It should be noted that here we were limited by information that was publicly available on websites and locatable through search terms such as ‘research culture’.

There are a huge number of activities, initiatives and networks, big and small, in this area across the R&I ecosystem. We recognise that we will only have captured a proportion of these in the time available for the study. In particular, there may be some smaller, less well-resourced or more private institutions whose activities we may not have captured. In addition, we may have gathered data around some initiatives in a single institution that may exist in others but were not publicised. Importantly, the difficulty in understanding the complete breadth and extent of activities in this area might also be faced by any Good Practice Exchange.

Appendix 5: co-creation workshops

Methodology

We ran four workshops of 2.5 hours each, conducted via Zoom and using Mural as a collaborative tool.

Participants were told that their input would be used “to shape the development of a future Good Practice Exchange, which aims to develop, test, evaluate and highlight ideas for improving research culture” as well as to develop a knowledge base to underpin UKRI’s wider portfolio of work within People, Culture and Talent.

Participants were told that we were interested in both their organisation’s views (if they were able to give these), but also their personal views as someone working in this area. They were encouraged to consult colleagues prior to the workshops if they wished, but this was not a requirement of participation.

Workshops 1-3 gave participants the opportunity to share their perceptions of the strengths and weaknesses of UK research culture, validate and refine the framework and prioritise areas for change and ideas for future actions.

Workshop 4 participants were given a summary of the previous workshops beforehand, with their session focused on the generation of ideas, particularly principles and priorities for a future Good Practice Exchange, as well as key questions about how the Good Practice Exchange should behave.

The workshops were recorded for analysis and reporting. Participants in workshops 1-3 shall remain anonymous and confidential, with their responses anonymised as much as possible in reporting. Participants in workshop 4 have their organisation credited below, but neither they nor their organisations are associated with any specific comments or inputs. A UKRI representative was present in workshop 4.

Structure of workshops 1 to 3

Part 1: Introduction to the session and the project, breakout groups: roles in research culture, whole group prioritisation exercise.

Part 2: Breakout groups: idea generation, using the centre for open science’s pyramid for behaviour change as a basis.

Part 3: Feedback on the breakout task, group discussion and summing up.

Structure of workshop 4

Part 1: Introduction to the session and project, discussion of findings from previous workshops, whole group prioritisation exercise.

Part 2: Breakout groups: each considering key questions around the creation of the good practice exchange.

Part 3: Feedback on the breakout task, group discussion and summing up.

Profile of participants

The workshops were designed to gain a variety of perspectives from people across the research sector, and participants were selected to involve diverse opinions from different communities. We attempted, where possible, to achieve a good spread of participants, including people with experience across all protected characteristics, as well as from across sectors and organisations. Details of participants are given below.

A total of 75 participants took part in the co-creation workshops, with workshop 4 having the largest number of participants.

Number of participants by workshop:

• 15 participants in workshop 1
• 17 participants in workshop 2
• 21 participants in workshop 3
• 22 participants in workshop 4

To respect privacy concerns and adhere to GDPR and MRS guidelines, it was not mandatory for participants to supply us with demographic information. This means that the numbers listed below may not add up to the total number of participants.

Ethnicity

Participants from a range of ethnicities were present in the workshops:

• 39 described themselves as white
• 6 described themselves as mixed race
• 4 described themselves as Asian
• 3 described themselves as Black
• 1 described themselves as Hispanic
• 1 described themselves as Jewish

Sexuality

When asked about their sexuality, participants identified as:

• heterosexual (40)
• homosexual (3)
• identified with another sexuality (3)
• chose not to disclose their sexual orientation (4)

Disability status

In total, 7 participants identified as living with various disabilities. These included:

• those living with a long-term medical condition
• chronic illness
• neurodivergence
• autism
• anxiety
• depression
• ADHD
• physical/dexterity/mobility impairments
• heart arrhythmia
• pernicious anaemia

Job roles

The intention was for the workshops to represent a large range of job roles. A wide range of job roles were represented across all four workshops. The breakdown is presented below.

Workshop 1:

• Lecturer x1
• Research managers x3
• Directors x2
• Researcher x1
• Policy managers x2
• Department heads x2
• Policy advisor x1
• Public engagement manager x1
• APVC x1

Workshop 2:

• Department heads x4
• Research managers x3
• University dean x1
• Associate director x1
• Academic engagement officer x1
• APVC x1
• Campaign lead x1
• Policy manager x1
• Research director x3
• Compliance officer x1

Workshop 3:

• Researchers x2
• Directors x2
• REF coordinator x1
• Department head x1
• HR director x1
• Lecturer x1
• Chief executive x1
• Research lead x2
• Research managers x4
• Open research specialist x1
• Associate x1
• PVC x1
• Policy lead x1
• AVP x1

Workshop 4:

• Policy lead x1
• Directors x3
• Lecturer x1
• Researchers x2
• Chairs x2
• Partner x1
• Assistant directors x2
• Chief executive x1
• Department heads x2
• Deputy director x1
• EDIS lead x1
• Policy specialists x2
• Research culture specialist x1
• Senior

Sectors

A wide range of quota groups were also represented across all four workshops, from third and public sector research organisations, to industry professional organisations. The breakdown of quota groups by workshop is displayed below.

Workshop 1:

• Researchers x1
• CPD organisations x1
• Research ethics and integrity specialists x2
• EDI leads and organisations x1
• University research culture specialists x5
• Third/public sector ROs x2
• PVCs for research x1
• Industry research managers x1
• Sustainability specialists x1

Workshop 2:

• Researchers x1
• CPD organisations x1
• Public engagement organisations x4
• EDI leads and organisations x1
• University research culture specialists x4
• Third/public sector ROs x2
• PVCs for research x1
• Industry research managers x1
• HR professionals x1

Workshop 3:

• Researchers x1
• University research culture specialists x8
• Research ethics and integrity specialists x1
• Public engagement organisations x1
• HR professionals x2
• PVCs for research x2
• EDI leads and organisations x1
• Trade unions x1
• Academic-industry collaborations x1
• Third/public sector ROs x1
• Industry professional organisations x1

Workshop 4:

• Academic – industry collaborations/ partnerships x2
• EDI leads and organisations x3
• Researchers x4
• University research culture specialists x1
• Third/Public sector ROs x1
• Industry research managers x1
• DSIT and/or Science and Technology Committee representatives  x2
• Academic groups x2
• Funder x3
• Sustainability specialists x1
• Public engagement organisations x1

Nationalities

All workshop attendees resided in the UK:

• 95% were from England (71)
• 4% were from Scotland (3)
• 1% were from Wales (1)

Appendix 6: recommendations around collaboration from the workshop

Key recommendations from the workshops relating to the theme of collaboration included:

• creating opportunities for professional support staff and other research enablers to become more involved in research, harnessing their diverse perspectives, life experiences and expertise from outside formal structures
• involving groups beyond higher education in developing frameworks for research integrity and evaluation
• learning from medical and health sciences, medical charities and the public sector in terms of work that has been done to encourage co-creation in research in practice
• supporting engagement with research users, society, and communities throughout the research process, and involving them in project planning, not just at later stages
• enabling communities to have a bigger role in deciding how funding is distributed and used, for example through participation in funding panels
• supporting team-based approaches to research, ensuring that external participants are rewarded in a timely manner
• making collaboration with stakeholders evident in funding applications and co-developed bids with non-academic partners
• forging strong connections between sectors, enabling movement between them
• encouraging early engagement with communities, using stakeholder input, and leveraging networks for knowledge-sharing
• allowing access to funding and promotions that recognise co-production approaches and accommodate diverse methodologies
• incentivising researchers to communicate with communities that might challenge their perspectives

Appendix 7: analysis approach

The project created the following data for analysis:

• the literature review
• data on existing initiatives from the call for evidence and desk research
• transcripts from the workshops and associated Mural boards

Quantitative and categorical data collected in the call for evidence was cleaned, then tabulated and analysed using Q Research Statistics software. Cross-tabulations and filters were created, enabling analysis by key elements of the framework (for example sector, theme, subject), which helped us:

• understand how the landscape of initiatives is distributed
• identify potential clusters of activity across each dimension, as well as gaps – for example, if a particular type of initiative was underrepresented in a particular sector, or if evaluated projects were rarely seen against a particular theme. This would then be linked to qualitative data from the call to evidence, collected in grid form – enabling us to identify precisely which initiatives fall into which clusters.

The transcripts and literature review were coded using textual analysis tool Atlas.ti, employing a code frame corresponding to the report’s structure as agreed following analysis meetings. Our approach also allowed for emergent themes and codes. Coding was attentive to the nuance of sector, subject and theme, and related to the earlier developed framework and nomenclature.

Appendix 8: research culture: a literature review

Lis Grey (University of Bristol) and Katherine East (King’s College, London), for the UK Reproducibility Network.

Background

UKRI have commissioned CRAC-Vitae, Shift Learning and the UK Reproducibility Network (UKRN) to undertake a state of the nation review on research culture. This project aims to develop a knowledge base to underpin UKRI’s portfolio of work within People, Culture and Talent. Specifically, it will inform the development of the ‘Good Practice Exchange’ outlined in the UK government’s People and Culture Strategy, which was proposed as a mechanism to build on the momentum within the sector and catalyse further collective activity to improve research culture.

Evidence about initiatives addressing research culture, and their evaluation, are likely to be widely scattered and with few public traces. The purpose of this literature review and desk research is to identify those traces and provide the project with either:

1. evidence directly contributing to the analysis, or
2. leads to closed information that can be followed up in subsequent direct engagement with the organisation (Stage C of the project to map the landscape of current and past activity focused on enhancing research and innovation culture across the academic, commercial, government, third and other sectors)

Aim

To conduct a review of academic, professional, trade, government and other sources to find evidence of initiatives addressing research culture, and their evaluation. Outputs include a brief literature review report and confirmation of the framework, nomenclature, and themes to be used in the next stage of the project and an initial list of target initiatives, contacts, and networks.

Definitions

We used the Royal Society’s definition of research culture: “the behaviours, values, expectations, attitudes and norms of our research communities. It influences researchers’ career paths and determines the way that research is conducted and communicated.”

A draft of a Research Culture Framework that was being developed within the project by CRAC-Vitae was used to guide our understanding of the types of activities that would be considered initiatives aimed at enhancing research culture. Example areas of focus for initiatives included:

• improving access to and participation in research, including postgraduate research study, for people from currently underrepresented groups
• furthering open research practices
• improving research conduct and reproducibility
• tackling bullying and harassment
• improving research leadership skills across all career stages
• creating routes for collaboration and exchange with businesses, third sector organisations and government
• securing and supporting the careers of researchers and associated professions
• diversifying recruitment, reward and recognition approaches at all career stages
• delivering new approaches to public dialogue and community-led research

Method

Search strategy

The search strategy was informed by clarifications from UK Research and Innovation (UKRI) and the project team, iterations of preliminary searches, and the evolution of the Research Culture Framework being developed as part of the project.

The database Scopus was chosen for identifying academic reports due to its extensive coverage of a wide range of disciplines. To identify grey literature, Business Source Complete, the CORDIS (database of EU-supported R&D activities), UKRI Gateway to Research and Overton databases were searched as well as running a Google Advanced search.

Searches were limited to reports published within the last five years and published in English language; if databases included a function to limit to reports authored in the UK, we also selected this limiter. All report types and study designs were considered. The data below show the search strategies and returns for each database.

Database search strategies and returns

Business Source Complete

Returns: 577

Terms: (research AND (culture OR environment OR climate OR ethos OR governance OR integrity OR open OR sustainable OR translation OR innovation OR impact OR excellence OR excellence OR support) ) OR “knowledge exchange”

CORDIS

Returns: 1

Terms: (‘research culture’)
Language: EN, Start date: 2018-01-01, Organisation country: UK

Gateway to Research

Returns: 34

Terms: (research AND (culture OR environment OR climate OR ethos OR governance OR integrity OR open OR sustainable OR translation OR innovation OR impact OR excellence OR excellence OR support) OR ( “knowledge exchange” ) )

Limited to 2018-2023

Only examined projects and publications – outcomes very broad and largely irrelevant

Overton

Returns: 487

Terms:

• TITLE-ABS-KEY ( ( “research culture” OR “culture of research” OR “research environment” OR “environment of research” OR “research climate” OR “climate of research” OR “research ethos” ) ) AND PUBYEAR > 2017 AND ( LIMIT-TO ( LANGUAGE , “English” ) ) AND ( LIMIT-TO ( AFFILCOUNTRY , “United Kingdom” ) )
• TITLE-ABS-KEY ( “research governance” OR “research integrity” OR “open* research” OR “sustainable research” ) AND PUBYEAR > 2017 AND PUBYEAR < 2024 AND ( LIMIT-TO ( LANGUAGE , “English” ) ) AND ( LIMIT-TO ( AFFILCOUNTRY , “United Kingdom” ) )
• TITLE-ABS-KEY ( “knowledge exchange” OR “research translation” OR “research innovat*” OR “research impact” OR “research excellence” OR “excellence in research” ) W/3 ( improv* OR better OR enhanc* ) AND PUBYEAR > 2017 AND PUBYEAR < 2024 AND ( LIMIT-TO ( AFFILCOUNTRY , “United Kingdom” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )
• TITLE-ABS-KEY ( “research career” OR “academic career” OR “career* in research” OR “career* in academi*” OR “research support” ) AND PUBYEAR > 2017 AND PUBYEAR < 2024 AND ( LIMIT-TO ( AFFILCOUNTRY , “United Kingdom” ) ) AND ( LIMIT-TO ( LANGUAGE , “English” ) )

Google Advanced

Returns: 320

Terms: allintitle: ( research AND (culture OR environment OR climate OR ethos OR governance OR integrity OR open OR sustainable OR translation OR innovation OR impact OR excellence OR excellence OR support) OR ( “knowledge exchange” ) )

Published 2018 – 2023, English language, UK

Setting and population

UK-based initiatives to support and promote research culture within research-performing organisations (that is, where research is conducted) were considered for inclusion. Initiatives in organisations outside the UK were excluded; however, initiatives involving multiple countries including the UK were within scope, provided the UK had more than an incidental role.

Initiatives/exposures

Initiatives were in scope if the reports referred to specific plans or actions intended to change the behaviours, values, expectations, attitudes and/or norms of the research workforce, and involved some combination of:

• co-creation: projects designed and conducted with community stakeholders, or where opportunities for partnerships/co-creation/collaboration may be present in the future
• evaluation: projects that have been self-assessed or independently evaluated or where there is potential for embedding evaluation mechanisms
• sharing: projects that have communicated findings, or are willing to communicate findings with the wider community

Reports of initiatives that do not explicitly aim to change culture and record the effect on behaviour, expectations, attitudes, norms or values with regards to research culture were excluded (for example reports of digital platforms that make data sharing easier, but with no discussion/findings given on behavioural changes associated with the platform).

Development work involved in creating position statements and policies on research culture was also considered for inclusion. Funding schemes designed to increase investment and resources to improve research culture were excluded.

Screening and data extraction

For articles that were not identified in Google, returned citations were imported to Endnote and duplicates removed. The remaining citations were single screened (that is, by one researcher only), on title and abstract/introduction/summary. Retained articles were then single screened on full text. For websites identified in Google, returned hits were imported to Microsoft Excel and single screened based on title/summary. Retained hits were then screened based on the landing page of the URL/website provided. Data extraction was conducted alongside full text screening using a Microsoft Excel datasheet to organise the data.

For articles that were not identified in Google, where available, we extracted information on the following:

a. Intervention name

b. Lead organisation

c. Other organisations involved

d. Region of UK

e. Sector(s) in which the work was done (academia, public sector, private sector, third sector)

f. Field (Arts and Humanities; Computer Sciences and Engineering; Health; Life Sciences; Physical Sciences; Social Sciences; Multiple fields)

g. Brief description of who within the organisation is leading

h. Completion status (ongoing, complete, unsure)

i. Completion date

j. Description of the intervention: aim, population involved (for example, ECRs, supervisors, policymakers, end users), methods, resources (for example, policy statement, teaching materials), sub-categories of the UKRN Research Culture Framework that the intervention addresses

k. Whether there was monitoring and evaluation, if so what was the scope, method, and findings

l. Cost in terms of people and money (including whether additional external funding is being used to fund it)

m. Limitations and potential sources of bias

n. Contact information or URL

For articles that were identified in Google, where available, we extracted information on the following:

a. Initiative name

b. Lead organisation

c. Sector(s) in which the work was done (academia, public sector, private sector, third sector)

d. Synopsis of initiative

e. Point(s) in framework that the initiative is relevant to

f. Whether there was monitoring and evaluation

g. URL

Data synthesis

A narrative synthesis of the extracted data was conducted, and is reported below.

Results

Overview

Of the 2,889 reports identified from Business Source Complete, CORDIS, Gateway to Research, Overton, and Scopus, a total of 29 initiatives were included for extraction (including those identified opportunistically or via snowballing); a breakdown of these according to database and primary sector involved is provided below.

Some reports recorded clusters of initiatives as part of one over-arching programme – in these cases the programmes have been counted as a single initiative. The main reasons for exclusion of returned reports were that they did not report an initiative, there was insufficient information on an initiative to extract, or the initiative was out of scope. Some full text reports were also not available (for example, conference abstracts) and so were excluded.

Numbers of initiatives aimed at improving research culture identified from Business Source Complete, CORDIS, Gateway to Research, Overton, and Scopus: total and by primary sector involved in the initiative

Business Source Complete

Total initiatives identified: 1

Academia: 1

CORDIS

Total initiatives identified: 0

Gateway to Research

Total initiatives identified: 1

Third sector: 1

Overton

Total initiatives identified: 3

Scopus

Total initiatives identified: 23

Academia: 14

Public: 9

Additionally, of the 320 website clusters identified from the Google Advanced search, the websites from 113 organisations were identified for extraction (including those identified opportunistically or via snowballing); a breakdown of these according to primary sector involved is provided in below.

Websites were explored for evidence of initiatives, including statements of intent or policies. Some initiatives occurred in clusters across organisations; for example, the Concordat to Support Research Integrity and Concordat to Support the Career Development of Researchers were included as initiatives, but so were individual reports provided by academic institutions to illustrate how the initiatives were being met. Therefore, the numbers presented below do not represent mutually exclusive initiatives, only mutually exclusive websites.

Summary of organisation websites identified from Google: total and by primary sector

Academia 71

Public 15

Private 4

Third 23

Total 113

As can be seen above, the majority of identified initiatives were based in academia, with some initiatives being discipline-specific (for example aimed at only at those working in computer sciences), but with many being cross-discipline (for example open research training and networking initiatives delivered by academic library services). Initiatives in the public sector predominantly focused on health and social care research culture. Some initiatives also spanned organisations and sectors; for example, the Research Excellence Framework (REF) and the Concordat to Support the Career Development of Researchers involved academic organisations and funders, while the San Francisco Declaration on Research Assessment (DORA) could be signed by individuals or organisations.

In general, initiatives were not well reported. Most of what was identified were statements of intent, policies, protocols, or advisory offices/research support teams. Descriptions gave insufficient detail to allow accurate coding according to the research culture framework and very few reports could be used to replicate an initiative. We applied broad interpretations of the three inclusion criteria (co-creation, evaluation and sharing) because description of these aspects was not always clear. For example, the individuals or teams involved in creating initiatives were not often reported but ‘co-creation’ was assumed if there was evidence that more than one person or stakeholder group had contributed to the development. The ‘sharing’ criterion was assumed from the fact that a report of the initiative, plans or actions was available in the public domain.

Brief descriptions and extracted data from all included initiatives are provided in Appendix 1 and 2. Appendix 3 provides an overview of the findings from the 18 evaluations of initiatives that were identified. Below, we summarise findings according to the research culture framework. The list of initiatives identified is not exhaustive due to the nature of this rapid review; rather, the list should be viewed as examples of initiatives that have been implemented to improve research culture in the UK since 2018.

How research is managed and undertaken

Initiatives addressing aspects of research culture in the form of how research is managed and undertaken tended to take the form of policy statements and protocols, forming advisory committees or research support teams/offices, and providing training and tools on research governance or open research. Specific examples are provided below. Numerous academic and non-academic institutions also published strategies which mentioned governance, integrity/honesty, and openness, although with little description of how these strategies would be evaluated.

There was substantial overlap between two aspects of the framework under this category: ‘achieving the highest levels of research integrity’ and ‘taking an open approach to research’, mainly due to both involving the open reporting of research.

Sustainability of initiatives was referred to in multiple ways, including sustainability in terms of funding, workforce retention, access to outputs/data/publications, and the environmental sustainability/impact of research initiative. For example, some academic organisation strategies referred to environmental sustainability as a key objective, while others referred to sustainability in terms of ensuring that outputs and the workforce are sustained.

Examples of initiatives under framework item ‘how research is managed and undertaken’

Effective research governance and management

• LSHTM Research Governance Statement
• UUK Managing risks in international research and innovation report
• Keele University Research Governance Toolkit
• NHS UK Policy Framework for Health and Social Care Research (public sector)
  Achieving the highest levels of research integrity
• Queen Margaret University Research Ethics and Governance team
• University of Nottingham Code of Research Conduct and Research Ethics
• UEA Research Integrity Report and Guidelines on Good Practice in Research
• FOSTER project

Taking an open approach to research

• University of Leeds Open Research Hub
• LSE Open Access Publications Policy
• RIOT Science club
• FORRT: Framework for Open and Reproducible Research Training
• UKRN Open Research Programme

Considering the sustainability of research

• EAUC Sustainability Leadership Scorecard (third sector)
• Anglia Ruskin University statement on Sustainability
• The University of Hull ‘A fairer, brighter, carbon neutral future: Strategy 2023’
• M-KEN: Marine Knowledge Exchange Network (academia and private sectors)

How research ensures value

Initiatives addressing dissemination and valuing of research included collaborative ventures between academia and private industry, patient and public engagement training and support for researchers, open research support, and research assessment frameworks (for example the Research Excellence Framework; REF). Example initiatives aimed at addressing this aspect of research culture are provided in below.

There was considerable overlap between the sub-categories ‘realising impact’ and ‘communicating research’ as impact often required communicating research.

Examples of initiatives under framework item ‘how research is managed and undertaken’

Effective research governance and management

• LSHTM Research Governance Statement
• UUK Managing risks in international research and innovation report
• Keele University Research Governance Toolkit
• NHS UK Policy Framework for Health and Social Care Research (public sector)

Achieving the highest levels of research integrity

• Queen Margaret University Research Ethics and Governance team
• University of Nottingham Code of Research Conduct and Research Ethics
• UEA Research Integrity Report and Guidelines on Good Practice in Research
• FOSTER project

Taking an open approach to research

• University of Leeds Open Research Hub
• LSE Open Access Publications Policy
• RIOT Science club
• FORRT: Framework for Open and Reproducible Research Training
• UKRN Open Research Programme

Considering the sustainability of research

• EAUC Sustainability Leadership Scorecard (third sector)
• Anglia Ruskin University statement on Sustainability
• The University of Hull ‘A fairer, brighter, carbon neutral future: Strategy 2023’
• M-KEN: Marine Knowledge Exchange Network (academia and private sectors)

How the research workforce is supported

The majority of initiatives identified in this review were focused on this part of the framework. Initiatives included research training schemes, networking events/platforms, mentoring and peer support schemes, academic career coaching, and EDI position statements. Specific examples are provided below.

There was overlap between all sub-categories in this part of the framework, but particularly between ‘supporting career progression’ and ‘empowering individuals’, and between ‘ensuring inclusive, supportive and healthy environments’, and ‘building collegiality’. Initiatives addressing ‘ensuring inclusive, supportive and healthy environments’ could often be separated into two types: those addressing researcher wellbeing and those aiming to increase equality and diversity of the workforce. Initiatives that fitted within this category of research workforce also overlapped with other categories, for example open approach, governance, largely because training in other aspects of the framework would also fit under supporting the workforce.

Examples of initiatives under framework item ‘how the research workforce is supported’

Supporting career progression

• The University of Sheffield 2020 to 2025 strategic plan
• UUK Research and innovation conference 2022
• The Concordat to Support the Career Development of Researchers
• Clinical Trials Scholarship (CTS) initiative at Birmingham, Warwick and Keele clinical trials units (public sector)

Providing effective leadership and management

• LSE Career Development Review (CDR) Toolkit
• Every Researcher Counts Toolkit (third sector)

Empowering individuals

• University of Northampton Changemaker Hub
• SFC Saltire Emerging Researcher Scheme (third sector)
• Research and Impact Accelerator Programme (RIAP) – UWTSD

Ensuring inclusive, supportive and healthy environments

• Imperial College London EDI strategy
• Résumé for Researchers (third sector)
• Athena SWAN
• ‘Healthcare Professionals in Research’ (HPiR) Facebook group (public sector)

Building collegiality

• University of Leeds Research Culture Cafés
• EMDoC (East Midlands Doctoral Network)
• UK Reproducibility Network
• GENMAC (Gender, Markets, and Consumers)

Evidence of evaluation of initiatives

Thirty-four reports of research culture initiatives either included evaluations or explicitly stated that evaluations were planned. Where evaluations were reported, these tended to be poorly described (for example not explaining methods in sufficient detail for replication or reporting all results) and of low quality (for example using unvalidated surveys with low response rates). In this review, we have included initiatives even if no evaluation was reported or that provided only reflective commentaries on initiatives but did not count these as having been evaluated.

Where findings from evaluations were reported (N=20), they were mostly positive: feedback from training courses, mentoring schemes, workshops and networking events endorsed the initiatives and suggested that researchers valued the initiatives. There was evidence of increased applications for funding and grant income generated following initiatives, and some reports highlighted that new research and innovation collaborations had been established. See Appendix 9: Overview of evaluations of research culture initiatives (literature review) for more.

There were no reports of initiatives that had failed or had no positive outcomes; however, a few reports highlighted that the sustainability of the initiatives was uncertain given the need for ongoing funding, and most reports were produced by the organisation(s) who designed and/or implemented the initiative and so may be subject to bias.

Summary and recommendations

This report provides a snapshot of the available records on initiatives to enhance research culture in the UK. The search strategy was broad to include grey literature across the public, private and third sectors as well as academia. Overall, we find evidence that many organisations across different sectors are aware of the importance of, and are making efforts to improve, factors that affect research culture. Perhaps unsurprisingly, the vast majority of research culture initiatives are within or targeted at universities.

In general, the initiatives were poorly described and there was little evidence of (robust) evaluation. Where evaluations were reported they tended to be in the form of service evaluations, for example, providing data on attendees at networking events, research grants or outputs attributed to the initiative or participant satisfaction and feedback surveys. It could be argued that these evaluations thus only focused on the behavioural aspects of ‘research culture’.

Future work should look to also assess the impact of initiative on the “values, expectations, attitudes and norms of our research communities” to give a more comprehensive understanding of research culture. Further, evaluations were often conducted shortly after initiatives were implemented – examining the longer-term effects will be important, particularly as cultural change can be slow to accomplish.

Limitations

• While we tried to keep search strategy broad, the lack of consensus both within and across sectors on terminology and definition around research culture means relevant reports may have been missed.
• Only single screening and extraction was possible with the resources available, limiting the reliability of the review.
• The poor quality descriptions of initiatives meant that assessing which parts of the research culture framework they aimed to address involved a degree of subjectivity.
• The rapid nature of this review limited our ability to snowball sample, which would be a preferable strategy for this relatively unexplored field.

Recommendations

• To best enable sharing of good practice and further our understanding of research culture, all research active organisations could be encouraged to report, in publicly accessible formats, what initiatives they have in place to support research culture. It may be helpful to provide guidance on what information to include and how, in order to standardise reporting across sectors.
• There is some conceptual overlap in the research culture framework. For example, inclusivity appears in both ‘ensuring inclusive, supportive and healthy environments’ and ‘building collegiality’; if a programme provides training on research methods/integrity, should it be coded under research management theme or support for researchers theme or both?; does the sustainability sub theme cover environmental impact, or financial/human resource sustainable capacity to keep the initiative running, or both? This overlap may make it difficult for organisations to develop and evaluate initiatives, and for initiatives in different organisations to be compared and best practice shared. It therefore would be beneficial in future iterations of the framework to condense or refine categories to make them distinct and limit the possibility of them being interpreted differently by different organisations/individuals.
• To better enable evaluation of research culture initiatives and promote sharing of good practice, it would be helpful to develop and validate a set of research culture measures that assess all aspects of the research culture framework and that can be easily applied in different contexts. While routinely collected data (for example grant income, workforce demographics, etc.) can provide useful information on the outcomes of an organisation’s research culture activity, assessment of researchers’ “values, expectations, attitudes and norms” is currently lacking. Increasingly, academic organisations are attempting to assess the wellbeing of their workforce – this could also usefully contribute to evaluation of research culture initiatives, although validated measures should be used to enable cross-organisation comparisons.
• Robust evaluation of initiatives should be promoted – ideally this would be planned at the development stage of an initiative and involve pre- and post-assessment using standardised, validated measures as well as routine data (for example workforce demographics, grant income) and qualitative methods. Given the often slow nature of culture change, long-term follow-up assessment is also important.

Appendix 9: overview of evaluations of research culture initiatives (literature review)

Of the research culture initiatives studied, 34 either included evaluations, explicitly stated that evaluations were planned or that the organisation was committed to evaluating its research culture initiative(s). Evaluations were generally of low intensity (for example very few conducted an assessment before implementing an initiative to enable pre-post comparison) and were minimally reported (for example insufficient detail to allow replication, with only narrative summaries provided of selected data).

The most common evaluation technique was a feedback survey of recipients/users (16), followed by feedback interviews/focus groups (7). Validated measures of impact were used in three evaluations (3). Other evaluations reported included: usage/attendance data (4); research output data, such as number of publications (2); case studies (3); and data on grant applications and awards (1). Some reports provided a general review of resources used and changes in the demographics of staff members (6) or an overview of achievements (12), such as implementing a new policy on fixed-term contracts.

Below, we provide some examples of the initiatives that were evaluated and their findings.

The University of Kent Academic Repository Research Support Team

This initiative involved forming a new team – comprising members from the library and professional services teams plus new posts – to create a sustainable environment where support for Open Access, and open scholarship more widely, is embedded in the Library Collections team and which provides the best support to researchers for Open Access. A single point of access email address was set up for researcher enquiries to the team. Evaluation involved gathering feedback (assumed to be via a survey) from research staff at the university. Feedback was “overwhelmingly positive” and the service increased the amount of contact from researchers regarding Open Access.

The Clinical Improvement Scholarship (CIS) at Western Sussex Hospitals NHS Foundation Trust

This initiative aimed to provide support for practitioners stepping up to the Clinical Academic Programme (CAP) for increasing research engagement among nursing, midwifery and allied health professionals. CIS candidates received salary support to release their clinical time two days a week over 12 months to work on a quality-improvement project in the trust. They also received personalised professional development, with a specific focus on leadership and research capabilities – including internal taught content from local clinicians, externally accredited modules, practical research project experience, action learning sets, and coaching.

Evaluation consisted of a feedback survey and interviews with the first cohort of CIS recipients and supervisors. Findings indicated that the scholars “observed an improvement in both their personal and professional confidence, particularly with regard to accessing and using research evidence to challenge and support change in practice”. This was endorsed by supervisors, and the authors of the evaluation reflect that there had been an increase in research engagement within the trust attributed to the work of the scholars.

The FOSTER project

This collaboration of EU-based research trainers/educators with an interest in Open Science aimed to create a community of practice for trainers to exchange ideas and materials. The project offered training workshops and one-day bootcamps, as well as infrastructure, resources and support for trainers – including an online toolkit and training handbook (Open Science Training Handbook, developed using the ‘book sprint’ method). FOSTER has also developed badges to reward trainers, trainees and platform users for completing certain tasks. Evaluation included reporting on event attendance and toolkit usage, as well as feedback surveys.

FOSTER reached 8,211 people between May 2017 and May 2019, including researchers from different domains, policy makers, research administrators and Open Science trainers in 41 countries. 78% reported that the workshops had been helpful to them becoming a better trainer. As a result, hundreds of training sessions have been held between these organisations, with many more planned. A total of 254 people attended 10 bootcamps; as a result, they organised 107 training events in their organisations. The bootcamps were positively received, particularly in providing a friendly environment and including a diverse range of people.

As of November 2018, 20,800 people had used the toolkit and 744 badges had been awarded. Sustaining the toolkit online has presented some challenges – some software used is now unsupported – but the use of analytics to identify problems and ensure the community can develop resources helps to keep it sustainable.

Community of Practice for Early Career Computer Science Academics in the UK

This initiative aimed to support the development needs of early career computer science academics, and involved three key activities: (i) developmental/training sessions; (ii) cross-university mentoring; (iii) cross-university buddying. Evaluation included records of event attendance and membership, as well as feedback surveys. The initiative’s events saw increasing interest and feedback was mostly positive (although response rates to the survey were low).

Networking opportunities were particularly valued, but survey respondents thought the events could have been more interactive and scheduled for less busy times in the academic calendar. Mentoring started in 2021, with 12 early career academics who requested mentoring in multiple areas (research, career planning, professional registration, education). Buddying began in summer 2021, with an initial group of six early career academics.

Healthcare Professionals in Research (HPiR) Facebook Community

This initiative aimed to facilitate self-directed and confidential online peer support for doctoral and postdoctoral healthcare professionals (HCPs) across the UK who conduct research. The Facebook site enables doctoral and postdoctoral HCPs to share advice and support each other as they complete their research and training. The community is monitored by members who were trained in the professional curation and moderation of the HPiR Facebook forum.

The initiative was evaluated via an online survey to all members that was pilot-tested and assessed by non-member HCPs to check face and content validity. Approximately one-third of members completed the survey. Most members joined for networking (88%) and peer support (82%) purposes. Members valued the opportunities that the HPiR community provides for peer support and connection with fellow HCPs.

Appendix 10: case studies and examples

A full breakdown of initiatives from the call for evidence by framework is given in Appendix 18: Complete list of initiatives. A small number of examples are given below.

Examples of initiatives targeting demographic groups

BME Early Career Researcher Hybrid Conference; Royal Society of Biology

The Black Minority and Ethnic (BME) Early Career Researchers (ECR) conference was founded by Dr Bernadine Idowu-Onibokun with the aim of equipping and empowering BME ECRs with the tools and skills required to remain and thrive in academia. Beginning in 2016, the conference has grown from strength to strength as evidence of its unique contribution to enhancing the wellbeing and professional lives of ECRs.
https://www.rsb.org.uk/

NADSN STEMM Action Group; University of Kent

NADSN is a super-network with a mission to connect and represent disabled staff networks. They are an “unincorporated association, non-governmental, independent, and self-determining, made up of impassioned people”. They act as a collective platform to share experiences and good practice and examine challenges and opportunities. They focus on the tertiary education sector (that is, universities and colleges) and are open to any individual and organisation interested in the equality of disabled staff.

The NADSN STEMM Action Group is an active steering group comprised of people with experience of disability, who are working in and around the STEMM disciplines. Since its inception in 2020, members have worked to develop a set of recommendations for funders and institutions to remove and ameliorate barriers faced by those who are marginalised due to disability.

Examples of initiatives relating to ‘How research is managed and undertaken’

Effective research governance and management

Nottingham Trent University puts on an annual programme of events for their Supervisor Education and Development Programme. It is for doctoral supervisors across eight schools, and is aimed at providing them with opportunities to reflect on their supervision practices, learn from research and from peers, and make changes that will create more inclusive, engaging and supportive supervision for candidates from a diverse range of backgrounds.

Achieving the highest levels of research integrity

The Wellcome Sanger Institute, in partnership with Catalyst Editorial, launched a mandatory training programme on good research practice for all scientific staff.

This focuses on:

• the responsibilities and behaviours that support research integrity and good research practice
• the drivers, costs and consequences of research misconduct
• the skills required to recognise and navigate ethical issues in research

Actively promoting sustainability

The University of Leeds are partnering with industry to promote open science and boost innovation in achieving net zero. It is funded by Horizon Europe and EPSRC. The project aims to address the challenges of sustainable agriculture by using low-cost nature-based solutions for soil health restoration and nutrient/carbon cycling.

Examples of initiatives relating to ‘How research ensures value’

Taking an open approach to research

Keele University produced an information resource for open data, called Making Data Open. It aligns to the ‘FAIR Guiding Principles for scientific data management and stewardship’. The resource page provides some guidance for the presentation and distribution of open data.

Communicating research

Responsible Research in Practice Ltd has produced Statistical Analysis for In Vivo Scientists Training. This training course is designed to support life science researchers to select the most appropriate method of statistical analysis for their experimental design, implement it, analyse, and report it in a rigorous and reproducible way that is in accordance with best practice.

Realising impact

The Research and Innovation Department, within the NHS, runs an initiative called Supporting your Research: From Idea to Impact. It supports clinical research by developing research questions, advising on grants and funding; providing study support services; and providing financial support. This initiative aims to create impact from completed projects.

Examples of initiatives relating to ’How people are supported’

Recognition and assessment

Close the Gap: Fair admissions in postgraduate research at Oxford and Cambridge seeks to understand and change the fact that on average, a Black British, British Bangladeshi, or British Pakistani candidate has been around half as likely as a White candidate to receive an offer for doctoral study at either Oxford or Cambridge.

This project, an equal partnership between Oxford and Cambridge, is researching formal and informal admissions cultures, systems and practices, and will develop and test disciplinary-specific, race-literate modifications and refinements to selection processes that are designed to bring about meaningful change in doctoral candidate selection systems.

Through working with volunteer departments across both Universities, drawing on the expertise of internal and external stakeholders, and engaging creatively with the lived experience of postgraduate students from ethnically minoritised groups, the project’s goal is to halve the current Offer Gap in the Pilot sites by the end of 2025.

Employment and conditions

The University of Liverpool established the Making an Impact (MAI) series in 2018, which aims to provide rich, varied and high-quality development opportunities for researchers, academics and research-related professional services staff at all stages of their career with opportunities offered across all faculties. The series includes sessions that are tailored to (and open for) postgraduate researchers, postdoctoral researchers, mid-career researchers, managers of researchers/principal investigators, professors, as well as research technical professionals and clinical research staff.

Sessions in the series are mapped to the MAI Framework, which has been created in response to University of Liverpool’s researcher development and knowledge exchange capacity-building needs.

Embedding professional and career development

The International Creative Research Methods Conference was founded to provide a home for the creative research methods community and help to develop the field and improve the research culture of this often-overlooked group of researchers. The first conference will be held Sept 2023 and has received 120 proposals from around the world (with 25 more late submissions). The director is an independent researcher and financing it themselves as they are not eligible to apply for UKRI funding.

Ensuring healthy working environments

GSK have an initiative focusing on supporting collaborative science through: attractive incentives and pay-grading people fairly and equally; building team culture around a common goal; minimising unhealthy competitive culture that develops around competing for funding; and promoting a belief that you cannot do science completely on your own (no public link available).

Examples of initiatives relating to ’How individuals engage with others’

Providing effective leadership and management GuildHE has a Research

Enhancing Research Culture Activity Support programme, which is a series of workshops, training opportunities, and discussions designed for smaller and specialist universities and colleges to enhance their understanding of good research culture and practice. It comprises of: two group coaching sessions, Leaders in Solidarity and Leadership and Shifting Culture, building on allyship principles, effective leadership, and personal resilience; workshops on participatory research methods, and creating effective and equitable research collaborations; and a series of research culture cafés based on the Wellcome Trust’s work.

Empowering individuals

The ‘PGR to Professor: Building Inclusive Research Culture in Interdisciplinary Environmental Research’ project at the University of Bristol brings PGR students, professional service members and academics from across the community to collaboratively re-imagine approaches and tools to enable strong, inclusive research connections, communities, and culture.

The project will reveal mechanisms that can bridge the gaps between academic and postgraduate communities to build an inclusive environmental-research eco-system which cultivates a sense of ‘belonging’ in higher education and environmental spaces. The project has been funded through Research England’s ‘Enhancing Research Culture Funding’.

Building collegiality

The British Science Association has a project titled The Ideas Fund. The Ideas Fund was set up to enable the UK public (individuals, community groups and charities) to receive funding directly and support them in working with researchers to explore ideas related to mental wellbeing.

The fund is delivered in four areas of the UK, working closely with local universities, with a key premise being that projects focus on community aspirations as the starting point, rather than a research/researcher agenda. Many projects with this funding have formed partnerships between community groups and researchers for the first time.

Examples of evaluated initiatives

Details of evaluated initiatives found in the literature review is given in Appendix 8: Research culture: A literature review. We offer a selection of summaries here, illustrating the variety of evaluation approaches. Below, we provide some examples of the initiatives that were evaluated. The widespread use of feedback surveys is apparent here, as in the literature review.

Examples of initiatives which have been evaluated, with evaluation information

The Clinical Improvement Scholarship (CIS) at Western Sussex Hospitals NHS Foundation Trust

This initiative aimed to provide support for practitioners stepping up to the Clinical Academic Programme (CAP) for increasing research engagement amongst nursing, midwifery and allied health professionals.

Evaluation consisted of a feedback survey and interviews with the first cohort of CIS recipients and supervisors.

The FOSTER project

This collaboration of UK and EU-based research trainers/educators with an interest in Open Science aimed to initiate a community of practice for trainers to exchange ideas and materials.

Evaluation included reporting on event attendance and toolkit usage, as well as feedback surveys.

Community of Practice for Early Career Computer Science Academics in the UK

This initiative aimed to support the development needs of early career computer science academics in the UK.

Evaluation included records of event attendance and membership, as well as feedback surveys.

Healthcare Professionals in Research (HPiR) Facebook Community

This initiative aimed to facilitate self-directed and confidential online peer support for doctoral and postdoctoral healthcare professionals (HCPs) across the UK who conduct research.

The initiative was evaluated via an online survey sent to all members.

Examples of initiatives with collaboration

The nature of collaborations varied across different submissions, as shown below. The types shown below are those most commonly represented in the call for evidence.

Examples of initiatives which are collaborative

Regionally focused collaborations – sometimes bringing together different sectors at a regional level:

• Universities for Nottingham Co(l)laboratory Research Hub
• Developing chief investigators within the NHS: the West Midlands clinical trials scholars programme
• Converge Challenge

Cross-institutional partnerships or initiatives in one sector – often found in the higher education sector:

• Prosper: unlocking postdoc career potential
• Close the Gap: fair admissions in postgraduate research at Oxford and Cambridge

Cross-sector partnerships or initiatives – often higher education or industry partnerships:

• Broadening Horizons in the Chemical Sciences (RSC)
• IGNITE Network+

Policies and concordats created with input from multiple stakeholder groups:

• The Concordat to Support the Career Development of Researchers
• Concordat for Engaging the Public with Research

Cross-organisational, or internal initiatives which cross-functional boundaries:

• The Kent Academic Repository
• Western Sussex NHS foundation Trust Clinical Improvement Scholarship (CIS)
• University of Leeds, Belonging & Success Research group

Collaborations between over-arching organisations:

• An equitable future for research and innovation (Young Foundation and UKRI)

Appendix 11: additional figures

Initiatives submitted to the call for evidence, by scale and coverage

Responses to the question ‘what is the scale of the initiative’:

• organisation or institutional level 45%
• cross-sector 27%
• sector level 17%
• department or team level 7%
• other 4%

Responses to the question ‘what is the coverage of the initiative’:

• regional or UK-wide 70%
• international or global 12%
• institutional or college 8%
• unsure 7%
• other 3%

Note for figures: base n = 347.

Initiatives submitted to the call for evidence, by target beneficiary

Responses to the question ‘are there any direct target beneficiaries of the initiative’:

• those involved in research activities 75%
• early career 48%
• mid-career 36%
• students 35%
• managerial staff or supervisors 29%
• senior leadership 25%
• other 21%
• technicians 19%
• part-time workers 15%
• HR 9%
• PhD 2%
• all 2%
• the public/ local communities 2%
• health and social care professionals 1%
• institutions and funders 1%
• EDI general 1%
• patients 1%
• independent researchers 1%

Notes to figures: base n = 272 to 437, 75 missing. Note that this was a multiple-choice question so the number of responses will not total the number of submissions.

Further target groups, mentioned less than 1% of the time, were:

• members of membership bodies
• research participants
• cross-sector collaborators
• new researchers or re-engagers in research

Proportion of initiatives by whether they have been shared

Responses to the question ‘has the initiative been shared with the wider R&I community / is it publicly available’:

• yes 62%
• no 22%
• no, but it will be 10%
• not applicable 6%

Source: call for evidence, base n = 347

Proportion of initiatives by the ways in which they have been shared

• on institutional website 40%
• report 25%
• events including webinars 23%
• blog post 12%
• other 12%
• social media 10%
• published article 10%
• initiative’s own website 9%
• resources 8%
• newsletter 4%

Source: call for evidence, base n = 210, those who indicate an initiative is shared.

Question: Please give details of how the initiative has been shared

Proportion of initiatives, by collaboration level

Extent of collaboration in research culture initiatives

• yes – designed with multiple organisational partners 35%
• yes – conducted with multiple organisational partners 33%
• no – 20%
• no – but there are opportunities for organisational or community partnerships 12%
• unsure – 9%
• other – 7%

Source: call for evidence, base n = 347

Proportion of initiatives by evaluation status

Responses to the question ‘has the project been evaluated’:

• unsure 27%
• self-assessment 20%
• not evaluated 14%
• not yet evaluated – but plans for this in the future 14%
• independent evaluation 8%
• not applicable 6%
• other 6%
• not yet evaluated and no plans to 5%

Source: call for evidence, base n = 347

Please note that initiatives may have been evaluated in more than one way; therefore these proportions do not add to 100%.

Framework section 1: Elements and behaviours

Effective research governance and management: 40%

Of which:

• N = 109: Implementing effective policies and processes
• N = 69: Mechanisms to ensure transparent, accountable governance
• N = 49: Providing open, competent and effective research leadership
• Providing appropriate, safe and accessible work spaces (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Achieving the highest levels of research integrity: 30%

Of which:

• N = 88: Upholding the highest standards of rigor and integrity
• N = 73: Caring and respecting the participants in and beneficiaries of research
• N= 72: Being transparent and honest about all aspects of the research process
• N = 62: Being accountable for all aspects of the research process

Actively promoting sustainability 15%

Of which:

• N = 39: Considering the impact of research on the environment and people
• N = 32: Ensuring the efficient use of finances, resources and infrastructure
• N = 23: Using sustainable approaches to research
• Effective use of resources to make the research system accessible to all (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)
• Investing appropriately in talent and sustainable employment (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Framework section 2: Elements and behaviours

Taking an open approach to research 43%

Of which:

• N = 107: Supporting open, collaborative, interdisciplinary and team science approaches to research
• N = 106: Co-creating and learning with research users and society
• N = 76: Ensuring research is understandable, explainable, reproducible and accessible
• N = 60: Being open, agile and responsive to new technologies and research approaches
• N = 56: Engaging and partnering with potential beneficiaries

Communicating research 41%

Of which:

• N = 90: Sharing research, data and other outputs openly
• N = 75: Connecting with others in accessible and inclusive language and media
• N = 72: Inspiring curiosity and learning
• N = 56: Open to new forms of communication methods and spaces
• N = 54: Acknowledging and building on the research and knowledge creation of others

Realising impact 40%

Of which:

• N = 96: Informing policy and practice
• N = 84: Contributing to knowledge creation and teaching
• N = 81: Capable of translation and innovation
• N = 79: Advancing discovery and driving innovation
• Understanding what value and impact means for different stakeholders (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)
• Developing a highly skilled and engaged workforce (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Framework section 3: elements and behaviours

Recognition and assessment 23%

Of which:

• N = 65: Recognising and valuing the diverse range of competencies needed for the research endeavour
• N = 58: Broadening the concept of excellence within the system research
• N = 58: Valuing diverse approaches, methods and contributions
• N =42: Valuing failure and risk-taking
• Valuing research wherever it is undertaken (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)
• Using appropriate qualitative and quantitative assessment methods (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Employment and conditions

This is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence.

Of which:

• N = 82: Recognising wider contributions to research within job descriptions, workload models and progression criteria
• N = 72: Providing structured and varied progression routes / Providing appropriate remuneration and employment benefits
• N = 72: Valuing the full range of experiences, skills and contributions of all who contribute to the research endeavour
• N = 57: Providing transparent, equitable and competency-based recruitment and recognition, recognising diversity
• N = 48: Ensuring healthy working conditions, accommodations and flexibility based on ongoing needs
• N = 43: Acknowledging and mitigating effects of career breaks and other disruptions, and inequalities

Embedding professional and career development 46%

Of which:

• N = 119: Providing a wide range of professional and career development opportunities
• N = 116: Addressing development needs at all career stages
• N = 78: Recognising and awareness of diverse career opportunities
• N = 72: Valuing continued professional development / Engaging in regular career development reviews
• Enabling access to inspiring mentors and role models (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Ensuring inclusive and healthy working environments 37%

Of which:

• N = 104: Embracing and valuing diversity
• N = 80: Ensuring the research environment is accessible, inclusive and equitable for all
• N = 76: Supporting good mental health and wellbeing
• N = 62: Zero tolerance of and taking action against bullying and harassment
• N = 58: Promoting balanced, flexible and achievable workloads
• Fostering psychological safety (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Framework section 4: elements and behaviours

Providing effective leadership and management 24%

Of which:

• N = 55: Being effective role models and mentors
• N = 46: Effective performance management
• N = 33: Providing honest and constructive feedback
• Providing responsive and empathetic line management (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)
• Valuing and responding to differences in supporting others (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Empowering individuals 32%

Of which:

• N = 66: Enabling creativity and encouraging an innovative, imaginative, entrepreneurial mindset
• N = 63: Encouraging a culture of reflection and learning from experience
• N = 59: Recognising motivations and ambitions, and facilitating professional visibility
• N = 52: Clear lines of responsibility, accountability and autonomy
• Generating confidence to speak out without repercussions (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)
• Encouraging all to invest in their continuing professional development (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Building collegiality 44%

Of which:

• N =118: Proving access to networks and communities
• N = 116: Creating welcoming and inclusive communities for all
• N = 105: Recognising individual and diverse contributions, advocating for others
• N = 81: Engendering a sense of identity and belonging for all
• Recognising that individuals’ behaviours shape cultures (this is a new addition to the framework and was not able to be mapped to previous framework used in the call for evidence)

Appendix 12: priority areas for change

Areas identified as most urgently requiring action

Participants in workshops 1-3 were asked to look at the research culture framework relevant to their workshop theme and vote on those which were most urgently in need of action. These are listed below. This is obviously far from a robust survey of the sector.

Workshop participants by sector

Workshop 1 participants by sector

• Researchers x1
• CPD organisations x1
• Research ethics and integrity specialists x2
• EDI leads and organisations x1
• University research culture specialists x5
• Third/public sector ROs x2
• PVCs for research x1
• Industry research managers x1
• Sustainability specialists x1

Workshop 2 participants by sector

• Researchers x1
• CPD organisations x1
• Public engagement organisations x4
• EDI leads and organisations x1
• University research culture specialists x4
• Third/public sector ROs x2
• PVCs for research x1
• Industry research managers x1
• HR professionals x1

Workshop 3 participants by sector

• Researchers x1
• University research culture specialists x8
• Research ethics and integrity specialists x1
• Public engagement organisations x1
• HR professionals x2
• PVCs for research x2
• EDI leads and organisations x1
• Trade unions x1
• Academic-industry collaborations x1
• Third/public sector ROs x1
• Industry professional organisations x1

Note that these areas map onto a previous version of the framework which was subsequently amended.

The research culture frameworks voted most urgently in need of action

Workshop 1: How research is managed and undertaken

1. Investing appropriately in talent
2. Providing open, competent and effective leadership
3. Engaging and co-creating with research users and society
4. Supporting open, collaborative, interdisciplinary and team science approaches to research
5. Considering the impact (in terms of sustainability) of research on the environment and people – fewer associated initiatives in the call for evidence
6. Being accountable for the research process and confident to speak out without repercussions

Workshop 2: How research is disseminated and valued

1. Understanding what value and impact means for different stakeholders
2. Broadening the concept of excellence within the system research
3. Valuing diverse approaches, methods and contributions
4. Recognising and valuing the diverse range of competencies needed for the research endeavour
5. Communicating in accessible and inclusive language and media
6. Open to new forms of communication methods and spaces – fewer associated initiatives in the call for evidence.

Workshop 3: How the research workforce is supported

1. Ensuring the research environment is accessible, inclusive and equitable for all
2. Valuing the range of experiences, skills and contributions of all who contribute to the research endeavour
3. Providing appropriate remuneration and employment benefits
4. Effective performance management
5. Promoting balanced and achievable workloads
6. Acknowledging wider contributions to research within workload models and progression criteria

From the call for evidence, we also identified the following framework areas as having relatively less submitted activity:

• Providing honest unbiased feedback
• Valuing failure and risk-taking
• Acknowledging and building on the research of others
• Considering the sustainability of research
• Providing open, transparent and merit-based recruitment and recognition
• Ensuring appropriate working conditions and other benefits
• Addressing precarity of employment

However, these were not selected as requiring the most urgent attention in the workshops.

How research is managed and undertaken

Each area for urgent action was discussed in a breakout group. A summary of responses is given below.

Investing appropriately in talent

Participants stated that, in the academic sector, promotion criteria had become too strongly aligned with metrics around publication, as publication record and associated ability to attract funding are now a key promotion criterion.

Participants stated that team-working has great benefits for research, though incentives were seen to be constructed in such a way as to make team-working less attractive. Additionally, the group mentioned the need to elevate and recognise the wider group of research enablers. They said that small grassroots organisations often had talented teams and individuals to co-create knowledge with, but some saw it as difficult to invest appropriately and responsively in them due to administrative and bureaucratic barriers. Issues around precarity of employment were also mentioned here in the academic sector specifically.

Providing open, competent and effective leadership

Discussions here focused on the need to enable leadership at all levels, highlighting the need for better training, alongside best practice exchanges between sectors (especially between organisations with smaller research capacity learning from organisations with more infrastructure). Funders were seen to have a key role in working with business and skills experts when evaluating fellowship applicants’ competencies, as well as with regulatory bodies on developing new policies attached to grant funding.

There was support for building on the momentum that is already behind the narrative CV, including via training, so that it is effectively embedded and doesn’t bring in additional biases. The idea of a summit or series of activities on research leadership was suggested, developing an understanding of what research leadership means, including a code or standards. Learning from other sectors and organisations was thought to be important here.

Engaging and co-creating with research users and society

The group stated that good work was present in this area, which could be shared more widely, particularly the Charities Research Involvement Group and community-based participatory research being conducted around public health interventions where patients are involved not just in co-creation of research but also of research questions. Participants thought challenges lay in embedding this in normal practice across sectors, helping researchers understand the value of doing so, and making it easier for them.

Engaging the public in research was also seen as a challenge here. Training on how to engage with communities and stakeholders was needed, alongside more sharing of skills and approaches. Participants stated that funders could also require the use of Responsible Research and Innovation frameworks.

Supporting open, collaborative, interdisciplinary and team science approaches to research

The group highlighted the need to disrupt processes and systems, particularly the activities of funders. This included ideas such as incentives for proposals that do not have a single principal investigator but a team of researchers, and more long-term and interdisciplinary funding, with a greater focus on ‘grand challenges’ facing society than specific disciplines. The group also mentioned that REF’s set-up was important, and called to reform the organisation so that it rewards inter-institutional and interdisciplinary activity.

Participants also noted that processes and systems more generally needed to value the role of partners in research to larger extent. Stronger skills development in interdisciplinary research might be required. The group also highlighted that this would be a space to learn from best practice in interdisciplinary working across different sectors, and industry was felt to have much to contribute here.

Considering the impact (in terms of sustainability) of research on the environment and people

Participants acknowledged that the term ‘sustainability’ could be defined in many ways in the context of research culture – while in some institutions, it could refer to sustainability of research and the workforce, for others, it clearly referred to sustainability in terms of the environment. Discussion here focused on the means through which sustainability considerations, for both definitions, could be brought into research without creating additional bureaucracy or workload, keeping it within current impact assessment processes.

Both resources and competence were also seen as barriers. The work of the Royal Society of Chemistry in supporting sustainable laboratories was specifically mentioned. Senior leadership support was seen as crucial.

Issues around EDI considerations in research were also discussed, with reverse mentoring for senior leaders mentioned specifically, along with better training on equality impact assessments for researchers, involving non-higher education groups in devising frameworks for research integrity and evaluation, and sharing case studies around working with diverse groups.

Being accountable for the research process and confident to speak out without repercussions

Participants discussed a range of potential interventions here, including embedding allyship training so that people are able to effectively support their colleagues. However, a more fundamental need to ‘disrupt’ systems was also seen as key, reforming research systems in ways that make them less hierarchical – which would make it easier for people to speak out.

More practically, there was a lot of discussion around the need for example to allow communities and external partners to have a greater role in committees – specifically in making decisions on funding, and the direction of research programs. Co-creating a route for researchers to report misconduct was also mentioned.

Suggestions for action

Suggestions for action in this workshop could be grouped under the following headings:

Inclusion

• enable flexible working arrangements, normalising personal development, fostering psychological safety, and promoting diverse representation in senior leadership
• require institutions to divorce maternity and sick leave from school or other budgets
• create opportunities for professional support staff to be more involved in research, harnessing their diverse perspectives, life experiences and expertise from outside formal structures
• produce training on Equality Impact Assessments and use inclusive language
• consider extending these assessments to include an environmental impact assessment and sustainability plan
• generate and share case studies highlighting the benefits of diversity in research
• involve groups beyond HE in developing frameworks for research integrity and evaluation
• learn from work done in medical and health sciences, medical charities and the public sector to encourage co-creation in research in practice

Leadership and support

• reform research processes and systems, to involve people at all levels and roles in decision-making and ensure their views are heard
• encourage and incentivise senior leadership to embrace change and create a safe environment for challenging existing practices
• encourage the broadening and extension of mentorship programs, particularly reverse mentoring for senior leaders
• give research group leaders and/or principal investigators training to effectively manage students and postdoctoral researchers in universities
• provide support and guidance around writing narrative CVs, to embed this effectively and equitably
• enable leadership at all levels and encourage more decision-making autonomy
• fully embed allyship training
• co-develop a sector-wide understanding of research leadership and how to foster good research leadership

Recognition

• support engagement with research users, society and communities throughout the research process – involve them in project planning, not just at later stages
• enable communities to have a bigger role in deciding how funding is distributed and used, that is, through participation in funding panels
• recognise and reward contributions beyond traditional academic outputs
• reward contributions throughout the research process, for example, by acknowledging the contribution of technicians and research enabling staff in reporting outputs
• provide training for evaluating impact creatively, and adopt competency-based assessment approaches, including sustainability considerations
• encourage use of intersectional support tools
• support team-based approaches to research, ensuring that external participants are rewarded
• adapt systems to make it easier to reward external organisations

How research ensures value

Each area for urgent action was discussed in a breakout group. A summary of responses is given below.

Understanding what value and impact means for different stakeholders

This group was concerned with the lack of public trust in science from an early age, and talked about the need to engage in a dialogue that extended past just policymakers, civil society and academia. Participants mentioned that for this to happen, it would be necessary to bring underrepresented voices into the research environment, through: involving more diverse groups in knowledge creation; and improving and creating infrastructures which allow citizens and residents to develop their understanding and express their opinion about matters that matter to them.

The group said that check lists and resources on how to conduct public engagement might be useful, but more crucial was having funder support for community engagement – as well as the necessity for researchers to both recognise the power that came from their position and be prepared to cede some of this power. They also stated that it was important to recognise that participatory methods weren’t appropriate for all disciplines.

Broadening the concept of excellence within the system of research

Participants discussed how excellence might mean different things to different stakeholders and that it was important to understand this clearly. Some saw it as important that there were multiple ways in which different kinds of excellence were rewarded, outside the formal world of metrics. The importance of embracing negative results and replications was also highlighted.

Participants said that institutions had a role here to celebrate success at every opportunity – and awards might also play a role in recognising wider researcher achievements, rather than just at landmark events like the REF. As one participant put it, “the more people feel valued the easier it will be to do more”.

Valuing diverse approaches, methods and contributions

The group discussed the wide range of definitions of diversity and the importance of defining this clearly – seeing diversity in terms of methods, contributions and approaches. Participants also discussed how policies mattered here and that they could both promote diversity through mandatory requirements, (for example through Athena Swan) and inhibit it (for example, via rules around who could be named as principal investigators in grant applications).

They also thought better communication was required to encourage diverse contributions. Randomisation of funding was offered by one participant as a potential solution worth trialling, as was encouraging the public celebration of achievements by organisations.

Recognising and valuing the diverse range of competencies needed for the research endeavour

Participants mentioned that recruitment practices were key here – some suggested that values, experience, and background with supporting peer development should be more central to recruitment criteria, rather than just publications records. Within the university sector specifically, participants saw promotional practice and PhD programs as being in need of change, the latter of which might need to adapt their structure to accommodate new kinds of PhD candidates. Acknowledgement of knowledge exchange expertise was also mentioned in this context.

Others discussed the importance of recognising the contributions made by all members of research teams, for example, technicians. Participants also mentioned the importance of working with people with lived experiences and recognising their expertise alongside ‘traditional’ qualifications when doing inclusive research.

Communicating in accessible and inclusive language and media

The group agreed this was an issue, and welcomed training, support and guidelines. It was noted that for some groups, such as women and minority populations, communicating via social media could attract abuse and that institutions needed to support individuals.

Participants also reflected that communicating accessibly goes beyond simple processes and, in the future, must be about broadening the kinds of outputs that are valued in research. Feedback was given that any initiatives to improve accessible communication should go beyond simply “seeing impact”, and instead see it as central to the research – and something that creates value within itself.

Open to new forms of communication methods and spaces

For new forms of communication to flourish, participants thought a number of preconditions were required. These focused on skill development and early career training, valuing excellence in alternative formats, engaging communities earlier in the research process rather than leaving communications to research outputs, and matching communication methods to audience through other forms such as animations and video. Developing communities of practice and exercising humility and increasing willingness to share and discuss were also highlighted. A number of examples of good engagement were given including the National Centre for Academic and Cultural Exchange and the University Policy Engagement Network.

One participant critiqued framing this area as ‘communications’:

I do feel quite strongly that by calling this communication, we open up to a slightly old-fashioned, slightly deficit model, potentially – although communication doesn’t have to be like that. In terms of how to bring people together, then this is about methodology. This is about ways of creating powerful, inclusive knowledge, so where are the communities of practice that can be brought together?

Participant, Workshop 2

Suggestions for action

Suggestions for action in this workshop could be grouped under the following headings:

Meaningful engagement

• make collaboration with stakeholders evident in funding applications and co-develop bids with non-academic partners
• forge strong connections between sectors, enabling movement between them
• encourage early engagement with communities, using stakeholder input and leveraging networks for sharing knowledge
• allow access to funding and promotions that recognise co-production approaches and accommodate diverse methodologies
• value knowledge exchange more explicitly through working groups, good practice sharing, transparent methodologies and diverse communication networks
• train ECRs to disseminate research in a range of forms to reach wider audiences
• help the public value science and research by stepping down from elevated positioning and speaking in lay language
• use alternative stakeholder communities, for example, ScotPEN, to act as a channel between researchers and the public

Recognition, rewards and resourcing

• change recognition and reward criteria to reflect the diversity of research outputs: promoting and recognising effective communication beyond journal articles, appreciating the value of diverse outputs. and moving away from using journal prestige as the sole measure of quality
• understand and address the consequences of the REF on what is valued in research and research culture more widely
• encourage publishers to request easy-to-read abstracts and support public engagement and outreach activities
• incentivise researchers to communicate with communities that might challenge their perspectives
• base recognition and awards on key metrics like innovation and usability of research outputs
• encourage researchers to embrace failure, publishing negative/null results, and review the REF
• provide funding for less well-resourced universities to invest in open research approaches and platforms

Inclusive research practices

• enhance accessibility and inclusive communication, implementing mandatory screen-reader accessibility, guidelines for inclusive language, and measures to address adverse consequences
• broaden the concept of excellence to promote diverse research practices and recognition
• train researchers in data curation and open research, and in implementing inclusive recruitment and promotion policies
• change practices and policies in recruitment and promotion, prioritising diversity, considering broader competencies
• share case studies of successful individuals with diverse skillsets
• create tools to help identify more diverse speakers and panellists
• challenge assumptions and incentivise engagement with diversity and inclusion initiatives
• rethink PhD programmes – potentially restructuring them to allow for greater diversity
• engage non-academic reviewers to build partnerships and diversify expertise

How the research workforce is supported

Each area for urgent action was discussed in a breakout group. A summary of responses is given below.

Ensuring the research environment is accessible, inclusive and equitable for all

It was clear to participants that inequalities exist and there was desire to move to past data collection to decisive action. They felt that more active campaigns were needed in terms of action plans and targets – perhaps with funding contingent on implementation – and sanctions and accountability enforced, with clear and timely complaints and resolutions systems. However, there was an overall strong consensus that another action plan on its own would not be enough; that future work on research culture needs to be more than a ‘tick box’ exercise.

A culture of individualism and hierarchical systems were felt to contribute to these issues. Participants suggested training was needed at all levels, and particularly for anyone involved in gatekeeping in this area. More widespread mentoring was also suggested. Wider questions such as precarity and flexibility of employment were also said to play a role.

Valuing the range of experiences, skills and contributions of all who contribute to the research endeavour

Participants explored key areas of need like looking at guidance and training for those writing and reviewing narrative CVs, developing promotion criteria that recognises teamwork, in addition to individual success, and encouraging recruitment criteria that recognises and values inter-sector experience, particularly for academic roles. Participants stated that changes to job specifications, promotional pathways and career routes were required here, particularly for research enablers. Again, support from senior leadership and/or management was thought to be key. Additionally, finding ways to both reward external partners and to identify potential partners was thought to be necessary, and participants said the REF should place increased value on this in the academic sector, as well as associated support from funders.

Providing appropriate remuneration and employment benefits

In terms of looking at providing appropriate remuneration and employment benefits, participants mentioned the degree to which the power was in the hands of funders and policy makers:

You could have restrictions placed on organisations if they didn’t meet minimum remuneration expectations… if UKRI said tomorrow, if you’ve got anyone on a fixed-term contract you cannot have any of our funding anymore, no-one’s got a fixed-term contract any more.

Participant, Workshop 3

Participants saw a multiplicity of different initiatives, some not well known. They thought there needed to be a mechanism to access this information in a much more digestible and accessible form. This was another area in which there was potential for the academic sector to learn from other sectors – particularly in terms of progression and promotion, and making these kinds of decisions transparent from the outset of people’s careers. Research enablers were seen to have fewer progression routes, and there was a need for the introduction of these and more standardisation across organisations. Clarifications about what expectations were within different job roles in progression was also seen as important.

Effective performance management

Effective performance management was felt to be lacking in many settings, with large differences between institutional practices. The practice of automatic salary progression was one that came under particular scrutiny – some considered there to be a lack of accountability in some institutions.

Participants thought there may be lessons to be learned in relation to performance management from the private sector – they gave examples of individual performance management practice where EDI, collegiality and research culture measures were embedded. Again, they emphasised that more sharing of best practice might be valuable.

Promoting balanced and achievable workloads

Workload was seen as a particular issue in research – in that it can be difficult to have a bounded workload because there is always, in theory, more that you can do. It was also acknowledged that research workload doesn’t exist in isolation from other forms of workload and was an individual decision within an institutional context. Participants suggested that benchmarking could play an important role, to understand what is happening across the sector, via creating a collective understanding of the workload impacts of specific activities such as involvement in committees.

Working time guidance for leaders was suggested, to enable a transparent and honest conversations about how long specific tasks and activities really take. This was already underway at one of our participant’s institutions:

Something we’ve piloted where I work at the research institute is that we… So last year we created a working time guidance specifically for our group leaders, which broke up the time allocations that the institute would expect of group leaders at different levels of seniority, so estimating how much time would be spent on things like internal committee work or peer review, and all these activities, and supervising.

Participant, Workshop 3

Acknowledging wider contributions to research within workload models and progression criteria

Participants saw a lot of positive initiatives happening in this area – with citizenship, knowledge exchange and wider engagement increasingly being included in promotion criteria. Discussion focused on whether there should be standards here and more sharing of practice. It was felt to be important to acknowledge impact on workloads here and resource it properly. The revised REF framework was felt potentially to be positive here.

Participants mentioned that work should build on progress already made – one suggestion was for an open framework or an exchange for workload models and progression criteria, so that people can see what is available, and use this to further develop and refine workload models and criteria within their own organisations and institutions.

Suggestions for action

Suggestions for action in this workshop could be grouped under the following headings:

Recruitment and progression

• formalise and review progression pathways for both researchers and research enablers. The GPE could have a role in helping HEIs to learn from business and industry practices here
• create recruitment criteria that better rewards a range of life experiences and skills
• change promotion and progression to better reward teamwork over individualism. Similarly, funding could recognise these things over a focus on publication records
• include contributions to EDI and support for the development of others as part of performance review processes
• address instability and lack of flexibility in job roles, identified as a key issue in research culture

Learning and sharing

• develop training in a number of areas to ensure better implementation – for example in reviewing and writing narrative CVs
• encourage consistency and transparency by identifying mechanisms to share good practice, for example in recruitment and progression processes
• make benchmarking around workload available, to give institutions a better sense of where they sit in terms of their own practices
• develop realistic guidance on how long different activities take

Systemic change

• change should be evidence led – however, it is important that changes are made where substantial evidence already exists around issues in research culture, for example, in the case of lack of representation
• address interactions with other systems – issues such as workload are about research culture but also how they interact with teaching and learning and other activities. Funders, employers and individuals working together is key
• there should be consequences for lack of compliance or poor standards – accountability is vital. Funders have an important role to play here
• examine who runs initiatives – recruitment for these roles should be transparent and competitive with a diverse range of people involved
• make existing work easy to access, bringing information and initiatives together to make them more accessible. Current initiatives are ‘indigestible’ and there are many of them

Appendix 13: cluster analysis method

Cluster analysis: Introduction to methods used

Cluster analysis is a multivariate technique that groups individuals or items into clusters. Individuals or items in the same cluster are more similar to one another than to those in other clusters. In this project, it was used to explore the data around initiatives collected in the call for evidence.

We used Q research software, employing a mixed mode cluster analysis technique, a simplification of latent class analysis. We selected this as it is a method which could both account for the amount of non-continuous data we collected and also ensured that initiatives were linked to a specific cluster.

Mixed-mode cluster analysis makes a technical assumption of constant priors for class sizes. That is, it groups items to the cluster that they are closest to, whereas latent class analysis groups items based both on their similarity to the clusters and the size of the clusters. Second, mixed-mode cluster analysis assumes people are in exactly one cluster, whereas latent class analysis allows for people to be partially in multiple clusters.

Data cleaning and preparation

Prior to running the clusters, we adjusted the data. This included:

• grouping some of the variables in questions so there were fewer options for bigger questions. For example, for initiative category, we grouped training, workshops and coaching together.
• turning some ordered categorical variables into ranked variables, for example funding
• excluding missing data, as we were unable to impute values due to missing data being non-random

Running the analysis

Analysis was run using different combinations of the following variables as inputs: Name Question Additional data processing

SEG1

How would you categorise this initiative?

Additional data processing: Grouped and missing data removed

SEG2

Which sector(s) does this initiative relate to?

Additional data processing: Grouped and missing data removed

SEG3

What is the scale of the initiative?

Additional data processing: Grouped, missing data removed, turned into a ranking question

SEG4

What is the coverage of this initiative?

Additional data processing: Grouped, missing data removed, turned into a ranking question

SEG5

Collaboration

Additional data processing: Grouped and missing data removed

SEG6

Has the initiative been shared with the wider R&I community?

Additional data processing:  Grouped and missing data removed

SEG7

Has the project been evaluated in any of the following ways?

Additional data processing: Grouped, missing data removed, turned into a ranking question

SEG8

Is/was the initiative funded?

Additional data processing: Grouped, missing data removed, turned into a ranking question

Initial analyses were run with analysis set to use 1000 iterations when processing the data and 100 start points. Different combinations were run and discussed with the team and the cluster groupings were cross-tabulated against other variables to establish their usefulness in describing the data. After running multiple clusters, we settled on one which had a low Bayesian information criterion (BIC) of 2,658.12. A lower BIC was found with clusters that excluded data entered by Shift, as this data entry was based on publicly available information only and so these initiatives had more missing data associated with it. However, a decision was taken to include this data for completeness.

Similarly, we decided to exclude the variables SEG1, SEG2 and SEG4, as these made the groupings less distinct and describable. We also excluded any set of clusters with more than five groupings, as the dataset was not large enough for us to be confident in the validity of them.

Our final groupings are given below. It should be noted that while group descriptors are indicative, not all the conditions in the description will hold in each case for every initiative. The number of cases is small for an analysis of this kind and contains large biases around missing data, so these clusters should be seen as indicative and worthy of further exploration rather than robust or definitive.

Group 1

Number of initiatives: 69

Percentage of initiatives: 20%

Description: Mainly large cross-sector projects, that are shared and involve collaboration

Sector, scale and coverage: More likely to:

• be present outside academia
• have UK-wide or international focus

Framework elements: More likely to:

• be focused on ‘realising impact’

Category: More likely to be:

• collective activity for example community, or special interest group, network, conference or committee

Sharing with the wider R&I community: 77%

Involving collaboration: 93%

Independent evaluation?: 14%

Some funding: 83%

Group 2

Number of initiatives: 73

Percentage of initiatives: 21%

Description: Projects at organisational or department-level, all funded, some of which are shared, but are largely self-assessed.

Sector, scale and coverage: More likely to:

• be at organisation level or regional level

Framework elements: No key differences

Category: More likely to be:

• mentoring, coaching, training or workshops

Sharing with the wider R&I community: 58%

‎Involving collaboration: 100%

Independent evaluation?: 10%

Some funding: 100%

Group 3

Number of initiatives: 69

Percentage of initiatives: 20%

Description: Largely organisation or department level projects with little collaboration, sharing or evaluation.

Sector, scale and coverage: More likely to:

• be at organisation level or regional level

Less likely to be outside academia

Framework elements: Less likely to:

• address framework 2 “how research is disseminated and valued”

Category: Less likely to be:

• collective activity or funding

Sharing with the wider R&I community: 4%

Involving collaboration: 0%

Independent evaluation?: 5%

Some funding: 78%

Group 4

Number of initiatives: 81

Percentage of initiatives: 23%

Description: Mix of organisational, departmental and cross-sector, with no collaboration though often shared. Little funding.

Sector, scale and coverage: Mix of organisational level and cross sector

Framework elements: No key differences

Category: More likely to be:

• policies, procedures, strategies, statements or commitments

Sharing with the wider R&I community: 84%

Involving collaboration: 7%

Independent evaluation?: 14%

Some funding: 51%

Group 5

Number of initiatives: 55

Percentage of initiatives: 16%

Description: Largely sector-level projects which, often collaborative and shared.

Sector, scale and coverage: More likely to be:

• at sector level
• at UK-wide or UK nation-wide level

Framework elements: No key differences

Category: Less likely to be:

• mentoring, coaching and related
• research, journals or resources

Sharing with the wider R&I community: 89%

Involving collaboration: 81%

Independent evaluation?: 21%

Some funding: 75%

All initiatives submitted, base n = 347

Appendix 14: the project team

CRAC-Vitae partnered with Shift Insight and UKRN to conduct the project. Each organisation contributed based on their specific area of expertise – providing a comprehensive and multi-faceted approach. An overview of each organisation and their involvement is displayed below.

CRAC-Vitae

A non-profit program under the Careers Research and Advisory Centre (CRAC), Vitae has been supporting researchers’ professional development for over 50 years. They work closely with higher education institutions to establish and promote best practices in employing and developing research talent. By becoming a member of Vitae, institutions can enhance their researchers’ skills and support their research goals.

Vitae believes that researcher development is crucial for improving research culture, and encompasses wellbeing, ethics, integrity, leadership, and equality, diversity, and inclusion. Vitae membership also aids institutions in engaging with the HR Excellence in Research Award process, by providing guidance and sharing practices for action planning, implementation, and evaluation – aligning with the principles of the Researcher Development Concordat, which aims to foster an inclusive and supportive research environment.

Bringing experience of working directly on research culture in the higher education sectors, Vitae provided input and guidance across key stages of the research. This included scoping, developing the framework, feeding into the synthesis of evidence gathered, and reviewing of report drafts. Vitae provided networks within R&D to lead on communication and engagement as part of Stage C (see Appendix 2 for more detail on the stages of the project).

Shift Insight

Shift Insight, established in 2002, has become a prominent research consultancy. While initially focused exclusively on education, they have expanded into sustainability and membership. They offer comprehensive research services – including primary and secondary research – using qualitative, quantitative and desk research methods. Their specialised sub-brand, Shift Learning, focuses on education, and the scientific and publishing sectors, possessing in-depth knowledge of these fields – including sector structures, terminology, challenges and sensitivities.

Shift’s clients comprise notable organisations including non-profits, funders, higher education institutions, professional bodies, scholarly societies, academic publishers and awarding organisations.

As a consultancy with deep experience in the education and scientific sectors, Shift led on project management, data collection, analysis and reporting. Shift also acted as the main contact and conduit between the organisations involved – responsible for the quality and timeliness of the project and day-to-day reporting.

UKRN

UKRN is a national, peer-led consortium focused on ensuring the UK remains a hub for world-leading research. It investigates factors contributing to robust research, promotes training activities and shares best practices.

With nearly 70 local networks of researchers, 30 institutions with senior representatives, and over 50 external stakeholders, including major companies and associations, UKRN fosters coordination and collaboration across the research sector. Previously, they have conducted various projects and activities relevant to this project, including:

• a literature review on research integrity; convening universities to share good practices
• providing leadership training for researchers at different career stages

Bringing research expertise, UKRN led on the literature review and desk research as part of Stage B (see Appendix 8: Research culture: A literature review).

Additionally, UKRN’s extensive sector network contributed to engagement in Stage C.

Individuals

The key project team is outlined in the table below. Both Shift and UKRN were supported by wider teams for relevant administration, fieldwork and literature review tasks, such as research assistants. All work was undertaken by the three organisations and quality-assured in-house.

Project roles and responsibilities

Jane Powell Shift Insight, Managing Director, Project supervisor

Jane’s role included overseeing the quality of the research, and co-writing the final report and deliverables. She was also directly involved in workshops and presentations.

Elsie Lauchlan Shift Insight, Associate Director Project director

Elsie led the day-to-day running of the research. She was responsible for project management, development of fieldwork materials, analysis and reporting, and the smooth running of key phases. She served as the day-to-day contact and conduit, managing the relationship across the partners involved. Elsie left the team during the project and her role was adopted by Emily Britton-Drewry.

Emily Britton-Drewry Shift Insight, Research Manager Project director

Emily led the day-to-day running of the research, following Elsie’s departure. She was responsible for project management, development of fieldwork materials, analysis and reporting. She had direct involvement in workshops, served as the day-to-day contact and conduit, managing the relationship across the partners involved.

Georgia Woollett Shift Insight, Junior Research Executive Project administrator

Georgia’s role included providing support to the research team regarding analysis and reporting, and had direct involvement in workshops.

Andy Dzro, Lorna Flutter, Ben Porter, Jack Wilson, Alex Jaworzyn and Kate Miah

Shift Insight colleagues Various roles at Shift Insight

Colleagues who gave support in recruitment, dissemination, data collection, editing, administration and other tasks which were invaluable for this project.

Clare Viney CRAC-Vitae, CEO Project advisor

Clare’s role included guiding all stages of the research. She led engagement with key stakeholders and relevant organisations as part of Stage C. She directly fed into analysis plans report structures for final deliverables. Clare also supported recruitment and facilitation of workshops alongside Shift.

Janet Metcalfe Vitae, Principal Project advisor

In addition to providing guidance across all stages of the research, Janet was fundamental to the framework’s development. She supported on key stakeholder engagement, co-facilitated workshops, as well as directly fed into analysis plans and report structures for final deliverables.

Yolana Pringle Vitae, Policy and Evidence Manager Project advisor

Yolana’s role included advising during all stages of the research. Yolana supported on key stakeholder engagement and directly fed into analysis plans and report structures for final deliverables. Yolana co-facilitated workshops alongside Shift and supported with recruitment for this stage.

Marcus Munafò, UKRN Chair of Steering Group, and Associate PVC for Research Culture, University of Bristol Lead for UKRN engagement activities

Marcus led on engagement as part of Stage C.

Neil Jacobs UKRN, Head of UK Reproducibility Network – Open Research Programme, project lead for the literature review

Neil oversaw the literature review with other senior colleagues from UKRN. The literature review was undertaken by two postdoctoral researchers, Katherine East and Lis Grey, supported with engagement as part of Stage C.

Appendix 15: glossary

Acronyms

AACR: American Association of Cancer Research

ABPI: Association of the British Pharmaceutical Industry

AGCAS: The Association of Graduate Careers Advisory Services

AHRC: Arts and Humanities Research Council

APVC: Associate Pro-Vice Chancellor

ARMA: UK Association of Research Managers and Administrators UK

AUA: Association of University Administrators

AVP: Assistant Vice President

B.B.A.M.: Black and Brown Academics and Mentorship

BME: Black Minority and Ethnic

BNA: British Neuroscience Association

CAP: Clinical Academic Programme

CDR: Career development review

CFE: Call for evidence

CIS: Clinical Improvement Scholarship

CoARA: Coalition for Advancing Research Assessment

COPE: Committee on Publication Ethics

CORDIS: The Community Research and Development Information Service

CRAC: Careers Research and Advisory Centre

C-ROPE: Center for Research On Publication Ethics

CTS: Clinical Trials Scholarship

DoRA: The San Francisco Declaration on Research Assessment

EAUC: The Alliance for Sustainability Leadership in Education

ECRs: Early Career Researches

EDI(S): Equality, Diversity and Inclusion (in Science)

EEF: Education Endowment Foundation

EEN: Education Endowment Network

ELRIG: The European laboratory research and innovation group

EMDoC: East Midlands Doctoral Network

EPSRC: Engineering & Physical Sciences Council

EUA: The European University Association

FAIR: Findability, Accessibility, Interoperability, and Reusability

FORRT: Framework for Open and Reproducible Research Training

FRAPs: Future Research Assessment Programmes

GDPR: General Data Protection Regulation

GENMAC: Gender, Markets and Consumers

GPE(x): Good Practice Exchange

HCPs: Healthcare professionals

HE: Higher education

HPiR: Healthcare Professionals in Research

HR: Human resources

IROs: Independent research organisation

KTN: Knowledge Transfer Network

KTP: Knowledge Transfer Partnerships

LGBTQ+: Lesbian, gay, bisexual, transgender, queer and other terms

LSE: London School of Economics

LSHTM: London School of Hygiene & Tropical Medicine

MAI: Making an Impact

M-KEN: Marine Knowledge Exchange Network

MRS: Market Research Society

NADP: National Association of Disability Practitioners

NADSN: National Association of Disabled Staff Networks

NCCPE: National Coordinating Centre for Public Engagement

NCUB: National Centre for Universities and Business

NHS: National Health Service

NiB: Neurodiversity in Business

NIHR: National Institute for Heath and Care Research

NIHR CRN NENC: National Institute for Heath and Care Research Clinical Research Network North East and North Cumbria

OASPA: Open Access Scholarly Publishing Association

PGR: Postgraduate Researcher

PLOS: Public Library of Science

PSREs: Public sector research establishments

PVC: Pro-Vice Chancellor

R&D: Research and development

R&I: Research and innovation

RAI: Robotics and artificial intelligence

RAND: Research and development

RCUK: Research Councils UK

REF: Research Excellence Framework

REMO: Researcher Mental Health Observatory

RFARA: Responsible and Fair Approaches to Research Assessment

RIAP: Research and Impact Accelerator Programme

RIOT: Reproducible Interpretable Open Transparent

RSC: Royal Society of Chemistry

SBA: Society of Black Academics

ScotPEN: The Scottish Public Engagement Network

SCVO: Scottish Council for Voluntary Organisations

SFC: Scottish Funding Council

SSO: Single Sign-On

STEM(M): Science, Technology, Engineering, and Mathematics (and Medicine)

TEF: Teaching Excellence Framework

UCL: University College London

UCU: University College Union

UEA: University of East Anglia

UK CORI: UK committee of Research Integrity

UK KTN: United Kingdom Knowledge Transfers Network

UKRI: UK Research and Innovation

UKRI GCRF: UK Research and Innovation Global Challenges Research Fund

UKRI TAS: UK Research and Innovation Trustworthy Autonomous Systems

UKRN UK: Reproducibility Network

UUK: Universities UK

UWTSD: University of Wales Trinity St David

Definitions

Academic research

Used here to refer to research conducted within a university or affiliated institution by scholars, researchers or academics to advance knowledge of various disciplines and fields.

Academic sector

Used here to describe research activities at universities, colleges, research institutions, academies, and other educational establishments that offer formal instruction, conduct research, and contribute to the advancement of various disciplines and fields of study.

Beneficiaries

A beneficiary is a person, organisation, or entity that is designated to receive benefits, assets, or advantages from a particular arrangement, agreement, policy, will, trust, or any other legal or financial instrument. Used here, this term refers to the individuals, groups, or entities that stand to gain or benefit from the outcomes, results, or applications of research.

Call for evidence

An information-gathering exercise that seeks expertise from people, organisations and stakeholders with knowledge of a particular issue. The call for evidence in this work consisted of an online survey seeking to directly engage with the R&I sector, allowing us to gather evidence of initiatives which seek to improve research culture.

Categorical data

Non-numeric data that is divided into groups or categories based on qualitative characteristics.

Co-creation workshops

Facilitated workshops where participants or stakeholders with different roles align to offer diverse insights to guide a design process. In this work, the co-creation workshops.

Disability

Used here as an overarching term to describe a range of long-term health conditions, impairments or physical or mental illness which have an impact on day-to-day life.

Disaggregation

The process of separating compiled information into smaller units to gain a deeper understanding and find underlying trends and patterns.

Equality, diversity and inclusion (EDI)

Ensuring fair treatment and opportunity for all, with the aim of eliminating prejudice and discrimination based on an individual group’s character traits.

Ethnicity

Used here to refer to cultural background, skin colour or ethnic origins.

External validity

The extent to which the findings, conclusions, or results of a research study can be generalised or applied to populations, settings, or conditions beyond the specific context of the study.

Gender

Used here to describe how a person identifies, whether that is male, female or another identity. As opposed to the term ‘sex’, gender is not used here to describe binary forms but a range of identities or experiences.

Good Practice Exchange

As outlined in the government’s People and Culture Strategy, the aim is to collaborate with individuals from various sectors, gathering insights from the community to develop, test, evaluate and highlight ideas to improve culture. This will look at interventions across talent, including bullying and harassment, diversity and inclusion, recruitment and leadership.

Grassroot

A group or movement that originates and operates at the local level, typically driven by individuals within a community who share a common cause, interest, or goal.

Grey literature

Information resources such as reports, studies, and publications, that are produced and distributed outside of traditional commercial publishing channels.

LinkedIn

A social media platform designed for business professionals to share work-related information.

Nationality

Used here to describe the legal and cultural affiliation or identity of an individual to a specific country or nation.

Non-academic research

Used here to refer to research that is conducted outside of formal educational or scholarly settings, often undertaken by individuals, organisations, or entities for various purposes beyond advancing knowledge within a specific field.

Private sector

Used here to describe research activities at commercial, industrial, and entrepreneurial organisations that are driven by profit motives.

Public evaluation

The assessment, analysis, and review of programs, policies, projects, or initiatives by involving the general public or a wide range of stakeholders in the evaluation process. It aims to gather input, opinions, and feedback from members of the public who are affected by or have an interest in the subject being evaluated.

Public sector

Used here to describe research activities at government organisations, agencies, institutions, and services that provide public goods, essential services, and regulatory functions to serve the interests and wellbeing of society.

Qualitative data

Non-numeric data that is descriptive and provides insights into qualities, characteristics, attributes, and opinions (such as categorical data).

Quantitative data

Numeric information that is collected through systematic measurement or counting, allowing for mathematical analysis, statistical computations, and objective comparisons.

Research connectors

Individuals, organisations, platforms, or technologies that facilitate and enhance connections, collaborations and interactions within the research community.

Research culture

In the context of this work, we use the definition of culture developed by the Royal Society: “Research culture encompasses the behaviours, values, expectations, attitudes and norms of our research communities.”

Research integrity

The adherence to ethical and professional principles in the conduct of research. It involves maintaining honesty, transparency, and accuracy throughout all stages of the research process.

Screen reader

A form of assistive technology that conveys text and image content as speech or braille output, often used by those who are visually impaired.

Sexuality

Used here to describe a person’s identity in relation to the gender or genders they are attracted to (their sexual orientation).

Snowballing

A sampling technique where existing participants provide referrals to recruit further participants required for a research study.

Stakeholders

An individual, group, or organisation that has an interest, concern, or investment in a particular project, organisation, or system. Stakeholders can include a wide range of parties, such as employees, customers, shareholders, suppliers, government agencies, local communities, and even the general public. Stakeholders here refer to groups or organisations in R&I, such as Universities, public sector research establishments and research-orientated charities.

Taxonomy

A hierarchical system for classifying and organising items or concepts based on their shared characteristics and relationships.

The People and Culture Strategy

Defined by the UK government as an “ambition to build the research and innovation workforce the UK needs, working in a positive and inclusive culture”.

The research concordats and agreements review

Published by UUK, UKRI and the Wellcome Trust, this review assesses the collective effect of concordats and agreements on the research cultures and environments in the UK.

The Science Europe Values framework

Science Europe describe this framework as: “a set of shared values to serve as a reference for the policies and practices implemented by Science Europe Member Organisations and as a foundation for collaboration on actions to further embed these values as part of the research system”.

Third sector

Used here to describe research activities at any non-profit, charitable, and voluntary organisation that works towards social, cultural, environmental, or community-based objectives that are neither part of the government (public sector) or driven by profit motives (private sector).

Trade literature

A category of written materials that are specifically created for professionals within a particular industry or trade. These publications are designed to provide industry-specific information, news, trends, and insights that are relevant to the practitioners, businesses, and organisations operating within that industry.

UKRINFORM

A state information and news agency and international broadcaster of Ukraine, also known as The National News Agency of Ukraine.

Appendix 16: questions asked in the call for evidence

1a. Which of the following sector(s) do you work in?

a. Academia

b. Industry / private (including IROs)

c. Public sector (including NHS and public sector research establishments (PSREs))

d. Third sector (including research-oriented charities and volunteering organisations)

e. Other, please specify

1b. Please tell us what your research discipline or industry is, if applicable.

1c. What is your job title?

<with single choice option: ‘This is not applicable to me.’>

1d. What organisation/company/institution do you work at?

<with single choice option: ‘This is not applicable to me.’

1. What is the name or description of the initiative?

Name:

Brief description:

Name of organisation leading the initiation:

Link to the initiation or information:

2. How would you categorise this initiative? Please use the ‘other’ box if the following categories aren’t suitable – we appreciate this list is not comprehensive of all plans and actions.

a. Campaign group

b. Commitment or concordat

c. Community or special interest group

d. Conference

e. Funding

f. Journal and/or publications

g. Mentoring or coaching

h. Network

i. Policy

j. Procedure or process

k. Research study

l. Resource

m. Special interest group

n. Training

o. Workshops

p. Other, please specify

3. Which sector(s) does this initiative relate to?

a. Academia

b. Industry / private

c. Public sector

d. Third sector

e. Other, please specify

4. What is the scale of the initiative?

a. Department / team level

b. Organisation / institutional level

c. Sector level

d. Cross-sector

e. Other, please specify

5. What is the coverage of this initiative?

a. Regional level

b. UK-wide level

c. Unsure

d. Other, please specify

6. We are looking to identify initiatives which are collaborative or there are opportunities for this in the future. Was the initiative designed and/or conducted with multiple organisation partners? Please select all that apply.

a. Yes – designed with multiple organisational partners, please give details

b. Yes – conducted with multiple organisational partners, please give details

c. No – but there are opportunities for organisational or community partnerships, please give details

d. No

e. Unsure

f. Other, please specify

7. Has the initiative been shared with the wider R&I community / is it publicly available? Note that any initiatives you provide will be added to a publicly-available list.

a. Yes – please give details of how the initiative has been shared (for example report, resource, event, etc.)

b. No

c. Not applicable

8. Will the findings be shared / made publicly available in the future, or are there opportunities for this?

a. Yes

b. No

c. Unsure

d. Other, please specify

9. Has the project been evaluated in any of the following ways or is there a plan or opportunities to evaluate it?

a. Self-assessment

b. Independent evaluation

c. Not yet evaluated – but plans for this in the future, please give details

d. Not evaluated

e. Not applicable

f. Unsure

g. Other, please specify

10. Are you formally involved with this initiative?

a. Yes – as an organiser or administrator

b. Yes – as a participant

c. Other please specify

d. No

11. Is/was the initiative funded?

a. No

b. Yes, less than £10,000

c. Yes – £10,000-£30,000

d. Yes – £30,001-£75,000

e. Yes – £75,001-£125,000

f. Yes – £125,000+

g. Other, please specify

h. N/A

i. Unsure

12. We’re aware that some groups are more likely to be negatively impacted by poor research culture, therefore we want to find out if initiatives are focused on any specific groups.

Are there any direct target beneficiaries of the initiative? Please select all that apply.

Staff/employees

a. Those involved in research activities

b. Students

c. HR

d. Technicians

e. Early career

f. Mid-career

g. Managerial staff/supervisors

h. Senior leadership

i. Part-time workers

j. Other, please specify

Demographic groups

a. Ethnic minorities

b. Women or gender minorities

c. Those with a disability or long-term health condition

d. LGBTQ+

e. International researchers

f. Those with caring responsibilities

g. Other, please specify

13. Is the initiative ongoing?

a. Yes

b. No

c. Other, please specify

14. We now want you to help categorise the themes of the initiative. Please select relevant topics below across the 3 core areas.

How research is managed and undertaken:

a. Effective research governance and management for example the standards, structures and policies to ensure good research practice, integrity and equity

b. Achieving the highest levels of research integrity for example undertaking research with integrity, honesty and rigour to ensure confidence in the methods and results

c. Taking an open approach to research for example undertaking research that is openly accessible, collaborative and increases research integrity bringing public value and innovation

d. Considering the sustainability of research for example minimising the impact of research on environmental, social and economic resources

e. Other, please specify

15. How research is disseminated and valued

a. Realising impact for example the translation of research into value for society, culture and economy

b. Using appropriate assessment for example broadening what is recognised and valued as contributing to the research endeavour

c. Communicating research for example making research and knowledge available and accessible to all

d. Other, please specify

16. How the research workforce is supported

a. Supporting career progression for example the employment, recruitment and development of the research workforce

b. Providing effective leadership and management for example the performance and line management of individuals

c. Empowering individuals for example individuals having power and control over their own lives

d. Ensuring inclusive, supportive and healthy environments for example all individuals are free to be themselves, feel well supported and confident to express their views

e. Building collegiality for example the creation of healthy, supportive communities

f. Other, please specify

17. Are any of the following sub-themes relevant to the initiative?

Effective research governance and management

a) Ensuring good governance

b) Implementing effective policies and processes

c) Providing strong capable leadership

d) Other, specify

Achieving the highest levels of research integrity

a) Upholding the highest standards of rigour and integrity

b) Being accountable for the research process and confident to speak out without repercussions

c) Being transparent and honest about all aspects of the research process

d) Caring and respecting the participants in and beneficiaries of research

e) Other, specify

Taking an open approach to research

a) Using open, collaborative and interdisciplinary approaches to research

b) Ensuring research is understandable, explainable and reproducible

c) Involving and engaging with business

d) Engagement with research users, society

e) Being open, agile and responsive to new approaches

f) Other, specify

Considering the sustainability of research

a) Using sustainable approaches to research

b) Ensuring the efficient use of talent, resources and infrastructure

c) Considering the Impact of research on the environment and people

d) Other, specify

Realising impact

a) Advancing discovery and driving innovation

b) Capable of translation and innovation

c) Informing policy and practice

d) Other, specify

Using appropriate assessment

a) Broadening the concept of excellence within research

b) Valuing diverse approaches and methods

c) Acknowledging diverse range of contributions

d) Valuing failure and risk-taking

e) Other, specify

Communicating research

a) Communicating in accessible and inclusive language and media

b) Contributing to knowledge creation and teaching

c) Inspiring curiosity and learning

d) Sharing research, data and other outputs openly

e) Acknowledging and building on the research of others

f) Open to new forms of communication methods

g) Other, specify

Supporting career progression

a) Providing open, transparent and merit-based recruitment and recognition

b) Ensuring appropriate working conditions and other benefits

c) Recognising development needs

d) Providing access to professional and career development opportunities

e) Addressing precarity of employment

f) Valuing careers within and beyond academia / diverse career paths

g) Valuing diverse experiences /mobility

h) Other, specify

Providing effective leadership and management

a) Providing honest unbiased feedback

b) Facilitating personal growth

c) Effective performance management

d) Being effective role models

e) Other, specify

Empowering individuals

a) Clear lines of accountability and responsibility

b) Encouraging a culture of learning and healthy competition

c) Enabling creative autonomy and encouraging innovation

d) Facilitating professional visibility

e) Other, specify

Ensuring inclusive, supportive and healthy environments

a) Accessible to all

b) Achieving equity

c) Creating enabling environments and effective hierarchies

d) Embracing diversity and respectful of all

e) Intolerance of bullying and harassment

f) Supports good mental health and wellbeing

g) Promoting work life balance and achievable workloads

h) Other specify

Building collegiality

a) Creating inclusive communities

b) Recognising individual and diverse contributions

c) Engendering a sense of identity and belonging

d) Proving access to networks and communities

e) Other, specify

18. If there are any documents you wish to upload relating to this initiative, please do so here.

Please note, you should only upload these if they are already publicly accessible.

19. Is there any other information you want to provide about this initiative?

20. Thank you for providing this information. Are there any other initiatives you want to provide details on?

a. Yes

b. No

If yes selected – repeat above questioning

21. Which aspects of research culture are addressed well in current initiatives available in your sector? This might be topics, audiences, sub-sectors or activity types.

22. Where do you think there are gaps?

23. Finally, are there any networks or channels we think we should be engaging with for this call of evidence?

Please provide the name of the initiative.

24. Thank you for taking part in the survey. Are there any more comments you’d like to share with us about the topic or the survey?

25. We will be conducting a series of paid online workshops in June to help us further explore gaps in initiatives as well as ways to overcome challenges within the system. Would you be interested in taking part in this part of the research?

Please note that by ticking yes, you are agreeing for Shift to use the information provided in this survey to contact you for future research.

a. Yes, I’d be interested in hearing more about this

b. No

26. Please fill in your contact details below.

Please note, we only ask you to provide details you are comfortable with sharing. These fields are not compulsory. Note that Shift adheres to the Market Research Society Code of Conduct (PDF, 593KB) and you will not be contacted for sales or marketing purposes. Please be assured that your survey responses will not be directly linked to your contact details in our analysis. For more information, please read our privacy policy.

Appendix 17: example email used to invite people to submit to the call for evidence

We used a number of different versions of this email to recruit and remind people to respond. This is an example of the basic script.

Dear [Name],

UKRI Call for Evidence: help us improve UK research culture and environment

Why are we gathering evidence?

The government’s People and Culture Strategy (PDF, 446KB) put forward a Good Practice Exchange to develop, test, evaluate and highlight ideas for improving culture. This is sourced from the community, bringing together people from across the sector to work creatively. To underpin a future Good Practice Exchange, UKRI have set up a taskforce of The Careers Research and Advisory Centre (CRAC) who manage Vitae, the UK Reproducibility Network and my research agency, Shift Learning, to explore initiatives to improve the UK research culture and environment across the research and innovation (R&I) sector. This Call for Evidence, to which you’re warmly invited, will allow us to draw on best practice and map out effective initiatives, while understanding how they can be better supported.

What evidence are we gathering?

We’re keen to hear about initiatives at your workplace – or those you’ve heard of elsewhere – intended to change the behaviours, values, expectations, attitudes and/or norms of the R&I workforce. The introduction to our Call for Evidence gives more detail on scope and how we define research culture.

How can you take part?

This is an opportunity for you to share initiatives you’ve been involved in with the R&I community.

Read more or take part

Answering the questions should take approximately 10 minutes, but will depend on how many initiatives you choose to describe.

Why are we inviting you?

We want to gather knowledge and expertise from a wide range of contributors, including all specialisms across academia, the private sector, the NHS and the third sector. We are contacting you as a key player in R&I, with vital insights to this area. However, if you know of anyone else in the sector who could also make relevant contributions to this evidence base, please forward on this email.

The Call for Evidence will close on Wednesday 3 May 2023. You can provide multiple initiatives and complete it again if you think of others at a later date. Insights from this project, including the full list of initiatives described, will be published following completion. If you have any questions about the research, please email emily.britton-drewry@shift-insight.co.uk or call +44(0)207 253 8959.

Best wishes

Emily Britton-Drewry
Research Manager
Shift Insight

Shift Insight are contacting you on behalf of UKRI, CRAC-Vitae and UKRN. Shift Insight adheres to the Market Research Society Code of Conduct and the Data Protection Act 2018. You have the right to access, rectify or withdraw your data from this research, or to restrict or object to its processing. You can find out more about us on our website or to find out how we handle your data please visit our policy page. Please feel free to check our validity by calling the Market Research Society UK Freephone verification service free* on 0500 39 69 99. If you do wish to make a complaint, please visit our website to review this policy.

Appendix 18: complete list of initiatives

The full list of initiatives can be found in the accompanying Excel spreadsheet for ease of reading, sorting and filtering, and a PDF document for readability.

Note that there may be duplicate initiatives in the data. These are because the submitted information for the initiatives differed and offered different insights.

Appendix 19: framework mappings, used to map the call for evidence data to the final version of the research culture framework

This appendix shows:

• the version of the framework that the call for evidence was based on
• the version of the framework that was used during the writing of the report
• and the ways we mapped the previous framework onto the report version of the framework

See the framework (PDF, 330KB)

Appendix 20: recommendations for the Good Practice Exchange, mapped to UKRI’s principles for change

This project has identified the following principles for the development of the Good Practice Exchange, its organisation and governance. We have organised these around the UKRI four principles for change.

Diversity

• the Good Practice Exchange will need to be inclusive of a diverse range of stakeholders to be credible. This includes diversity in term of sector, seniority, personal characteristics and job roles. This diversity should also be reflected in the leadership and governance of the Good Practice Exchange.
• championing and amplifying existing work will be essential. There should be emphasis on looking to , smaller scale or unfunded initiatives, working at organisational level and/or from outside higher education; this could be done alongside encouraging effective evaluation for these kinds of initiatives.
• incentives to participate will need to be considered carefully, particularly to enable underrepresented groups, including those outside higher education, to participate.
• it is important that organisations as well as individuals are clear about the benefits of being involved.
• the Good Practice Exchange will also need to be mindful of the unequal burden placed on minoritised groups to change research culture and ensure that there is adequate compensation for this labour.

Connectivity

• practice-sharing should reach across the research ecosystem, maximising the potential for learning between institutions and sectors. It should support a de-centring of academic experiences and work
• the Good Practice Exchange should foster a place for openness and transparency, including failure
• to gain credibility and influence, it will be important for the Good Practice Exchange to help translate practice into policy. This could include connecting those most affected by poor research culture with those seeking to address it through policymaking

Resilience

• the Good Practice Exchange must be agile and flexible, with mechanisms to monitor and review what’s important, what’s working, and what’s changing
• the Good Practice Exchange needs to save time. It can do this by building on existing work and considering how processes and initiatives can be streamlined.
  Engagement
• a common language for research culture is needed for engagement, planning and strategy development. The research culture framework developed to map the interventions for this project provides a useful tool here
• the Good Practice Exchange needs to be action-based and be seen to enable actual change on the ground
• work must be inclusive and fully accessible, both in terms of activities and outputs
• evaluation considerations should be built in from the start
• the Good Practice Exchange will need a compelling vision both for its own activities and for research culture. This will need to fully engage with a diverse range of stakeholders to be relevant and effective.

Appendix 21: project reflections

We wanted to gain a comprehensive understanding of research culture across a diverse range of sectors, industries and voices – our approach placed this at the forefront. We used a variety of methods, communication strategies and safeguarding practises, in line with UK government guidelines around inclusive social research. We recognise the importance of collecting data from the widest and most diverse pool of contributors to ensure the fullest dataset possible and one that reflects the R&I sector.

The call for evidence and workshops were designed by the project team, which comprised researchers from organisations with a wealth of expertise in designing diverse and inclusive research (see Appendix 14: The project team for details). However, we recognise that the core project team was not particularly diverse and this will have influenced the methodology and limited our ability to engage meaningfully with some communities. At every stage of the project, we actively sought out engagement with a diverse range of stakeholders. The following strategies were specifically employed:

• working with a wide range of contributors through consultation and including expert input to develop, validate, and refine a framework that allows a consistent and inclusive way of mapping research culture initiatives (see Appendix 2: Research culture framework methodology)
• using techniques like snowballing (often cited as being a valuable method for recruitment from underrepresented groups) to promote engagement and responses from underrepresented groups for the call for evidence (see Appendix 3: Call for evidence: Detailed methodology and Appendix 4: Desk research / gap-filling)
• running co-creation workshops with varied research communities, specifically targeting and including diverse opinions through oversampling and guidance from community advisors. Incentives were offered for workshop participation where appropriate (see Appendix 5: Co-creation workshops)
• ensuring that data collection practices and reporting are accessible to all. This was done in various ways, for example: consideration during recruitment to additional needs; outputs that can be accessed via screen-reader
• foregrounding issues of inclusion in the workshops as an explicit area of consideration in discussions, prioritisation and idea generation
• identifying and addressing gaps in representation during data collection, with a focus on external validity (see Appendix 4: Desk research / gap-filling)
• analysing data to ensure meaningful disaggregation and emphasise specific voices (see Appendix 7: Analysis approach)
• publishing an open-access synthesis report to ensure accessibility for all

Despite the consistent outreach and adaptation of methods, it remained difficult to engage those outside UK HEIs. There are several reasons for this: commercial settings often use different language to describe these sorts of activities, may see these activities as private and commercially sensitive, and may have less time available/interest to join such activities. They may also be less likely to be involved in groups or channels connected to UKRI and the project team, or see the value of engaging in the project.

Activities conducted by other groups outside higher education, such as small grassroots organisations, may also be underrepresented, perhaps because they are less well-resourced. Smaller projects may also lack resources for public evaluation or sharing.

Perhaps most significantly, research culture is still often seen as a higher education-specific matter. While the term ‘research culture’ is becoming established in higher education, it does not translate easily to other sectors, where terms such as ‘organisational culture’ or ‘workplace culture’ are more common. Therefore, the use of this term may have inadvertently excluded participants. Care needs to be taken to mitigate against this in future communications, activities and outputs.

It will be important to consider how to boost engagement with, and understanding of, research communities outside of higher education, as well as how to make involvement at an individual and corporate level worthwhile. Fully inclusive project leadership and co-design should be at the forefront of any considerations for further projects and for a future Good Practice Exchange.

Minimising the work required to share information, providing incentives to do so and engaging leaders in R&I, and particularly those not in academia, will be important moving forward. Moreover, the potential benefits from doing so must be clear and compelling to researchers and communities, and people must have trust in them.

Page viewed: 1:10 am on 22 December 2024