A guide for Science and Technology Facilities Council (STFC) public engagement grant holders when compiling evaluation data through Researchfish and metrics spreadsheet.
Introduction
Effectively and appropriately evaluating public engagement is important to STFC. In publishing the STFC public engagement evaluation framework, we aim to not only make our own approach transparent, but to aid our community in undertaking their own engagement activities.
Our evaluation framework lays out a clear series of measures by which we measure, review, and reflect on the effectiveness of the STFC public engagement programme. Within this programme, our range of public engagement grant schemes enables the STFC community to undertake high quality engagement across the UK. Grant holders extend the reach of STFC science and technology to new audiences remote from our own STFC laboratories and campuses.
Our evaluation framework was explicitly designed to allow STFC to evaluate our engagement programme, not just our individual activities. Our grant schemes are part of this programme and your data plays an important role in our understanding of how that overall programme is delivering.
Our grant holders report their data via Researchfish and a metrics spreadsheet. This guidance document is designed to help that process. We hope that this will help grant holders better understand our expectations for the evaluation of public engagement and help them record and report their work over the lifetime of their award, and beyond.
We have updated our evaluation framework to align with the refreshed STFC public engagement strategy published in 2024. To align with the new framework and strategy, we have updated Researchfish and the metrics spreadsheet. It contains some new sections that provide us with more information about your public engagement grant project. Guidance for completing these questions is covered in this document.
Be aware, as part of this change, data submitted in previous Researchfish submissions will no longer be accessible during the reporting process.
Monitoring and reporting on STFC public engagement grants
STFC monitors its public engagement funded activity through Researchfish and a metrics spreadsheet. All STFC public engagement grant holders must submit their data through these two monitoring systems. We recommend that engagement data is captured on the metrics spreadsheet as the project progresses to make it easier to collate data at the end of the calendar year for submission during the Researchfish window in February and March.
Metrics spreadsheet
The metrics spreadsheet is designed to capture data as the project progresses and will help sum data to enter into Researchfish. It also requests some additional information that Researchfish does not ask for. This includes, for example, the post codes of any schools engaged with during the year.
This information allows us to look at the data from multiple award holders and see what the total reach is for our grant programmes. Include as many details as you can on the metrics spreadsheet, even if they do not appear to be asked for on Researchfish.
Submit the metrics spreadsheet by email to stfcpublicengagement@stfc.ac.uk by the Researchfish submission deadline each year.
Researchfish
While all UKRI grant holders complete the questions relevant to engagement in the ‘common outcomes’ question set, STFC public engagement grant holders have an additional, bespoke set of mandatory questions. These are labelled ‘STFC Engagement Activities’.
Completing the metrics spreadsheet
The metrics spreadsheet has a number of worksheets within it. This guidance will go through them one by one. Not all of them may be relevant, but read the guidance in full to help understand what to complete.
Report on the complete previous calendar year as is requested in the Researchfish submission.
Worksheet 1: Grant details
Include the details of your grant, as entered on the Je-S form or the Funding Service.
Worksheet 2: Event reach and outcomes
This is for keeping track of details and outcomes of face to face or online events.
If your project is mainly about producing resources or creative pieces then the first worksheet ‘event reach & outcomes’ may not be relevant. However, if there are occasions when people are invited to see or engage in your resource, then do add these events with as much information as you can to the Worksheet.
Complete the ‘All Events’ fields and the fields that are relevant to your event type (for example public, community, schools, training). Provide this information for every event you do.
Event details
This is summary data about your event.
Location of event (postcode or online)
This is for face-to-face or online events. For on demand videos or content, include the number of views/downloads in the ‘Resources’ tab.
Event name and type of event
Give the name of your event and the type of event you ran. This can include number of interactions, delivery methods, methodology for developing content.
How many events is this
This would usually be ‘1’. Occasionally you may wish to group a few events together. If so, report the total number of events and ensure your answers are the sum the total attendees and their responses.
Reach
All events
- Total number of attendees. Individuals who attend more than one event should be counted at each event they participated in.
- Percentage of whole audience from the Wonder demographic (not just 8 to 14-year-olds). This can be an estimate. See the evaluation framework for more guidance on how to report Wonder groups.
Public event
Number of children aged 8 to 14 (can be an estimate).
Community groups event
Number of groups who attended (add details of the groups to Worksheet 3: Schools and groups).
Training or CPD event
- Number of groups or schools who attended, if applicable (add details of schools or groups to Worksheet 3: Schools and groups).
- Total number of people completing the training.
Schools event
- Number of schools attending (add details of schools to Worksheet 3: Schools and groups).
- Number of students attending.
- Number of students from each age range:
- upper primary (Key Stage 2/Second level, Years 3 to 6 in England and Wales, Years 4 to 7 in NI, P5 to P7 in Scotland)
- lower secondary (Key Stage 3/Third and Fourth level, Years 7 to 9 in England and Wales, Years 8 to 10 in NI, S1 to S3 in Scotland)
- upper secondary (Key Stage 4/Senior phase, Years 10 and 11 in England and Wales, Years 11 and 12 in NI, S4 to S6 in Scotland)
Participant outcomes
These questions relate to the participant outcomes from STFC’s public engagement evaluation framework. Completing this worksheet will help you to collate outcome data for your events so you can report your impacts on Researchfish Question 4.
The participant outcomes section is split into two to enable reporting of quantitative and qualitative data. Some fields are marked as essential. This means that you should aim to provide this information for every event you do, as we need to report on these outcomes across all STFC Public Engagement strands. The training outcomes are required for training events only.
Appendix 2 of the evaluation framework provides guidance on developing quantitative and qualitative questions linked to each outcome.
For qualitative data (such as interview responses, focus group notes, questionnaire responses, delivery team observations, quotes or comments), record these for each linked outcome along with a non-identifying description of who said them (for instance, ‘teacher’, ‘Farm Dale primary school’, ‘Mrs J, age 65’, or ‘Amelia, age 5’).
Worksheet 3: Schools and groups
This sheet captures the names and the postcodes of any schools or groups (for example community groups or uniformed groups) that attended your events or who were targeted for receiving resources. Indicate if this is a new group you are working with.
Do not duplicate schools or groups. Each one should only appear once on this sheet.
Note: For data protection purposes, do not include both the full name and postcode for home educated groups, as this uniquely identifies individuals. As alternative, put ‘home educator’ as the group name and include their postcode.
Worksheet 4: Resources
We are keen to know if you have produced any new resources and how widely they are distributed. Use this worksheet to record the name and a link to the resource, if available, and the number of times the resource was distributed, downloaded, viewed or visited.
We are particularly interested in how many resources reach schools. If you have targeted schools for receipt of resources, then add those schools to the list of schools on Worksheet 3: Schools and groups.
If you have published any papers or given any conference talks linked to your STFC-funded public engagement activity, also add the details on this worksheet.
Worksheet 5: Partners
This sheet captures any organisations or groups that you partnered with.
Public engagement is often delivered in partnership, which can take many different forms. Your project may have included a formal partnership or informal collaboration on activity development. Partners may be with national, regional, local organisations or networks and could include STFC staff, researchers or facility users. Only partnerships that substantially shape your work should be reported.
Provide the name of the partner organisation, their postcode and a short description of the nature of the partnership or collaboration. We are particularly interested in any co-production approaches that were used in your partnerships, so include these if applicable.
This sheet also contains partnership outcomes. The first outcome, ‘Do you feel your partnership with STFC adds value to your work?’, is essential and relates to your partnership with us. This is an opportunity for you to give feedback, both positive and negative. This data will be anonymised in reporting and will be used to improve STFC’s partnerships with its grant holders.
The second outcome relates to your partnership with community groups, teachers or partners you work with. It is required if you develop relevant relationships or partnerships. If you have developed these partnerships, ask your partners directly and report their response in the field provided.
Worksheet 6: STFC staff
To help us to understand how STFC-funded staff and researchers have contributed to public engagement, list all STFC scientists, engineers, researchers or students who contributed to the development or delivery of public engagement activities and resources. This includes anyone who works in a department that received STFC funding, or a facility user on an STFC facility.
Do not list individuals more than once. Undergraduate students should not be counted here. Indicate how many hours each individual has spent on the project and the main role that they played. If ‘other’, write a short description of the nature of their involvement.
If individuals played a similar role, they can be grouped together. For example, six post-docs from the same department could be in a single row, grouped together as ‘Post Docs’ under name or job role. Include how many are in the group and how many hours total they spent of the project.
Worksheet 7: Fellowships – capacity-building
If you have been awarded an STFC Leadership Fellow in Public Engagement Fellowship, it is mandatory to complete this tab. Other grants may include capacity-building activity, and if this the case, we strongly encourage you to fill in this tab with relevant data even though it is optional.
We have included some examples of leadership and capacity-building activities, but appreciate they can be varied and take many forms. Include details of individuals or groups involved and a brief description of the activity. If the activity if a training event, also include in this event in Worksheet 2: Event reach and outcomes.
If your capacity-building has led to the development of another project, activity or event, give an estimate of the participants reached if possible. Also provide any other information you feel is relevant to reporting the activity, such as web links, longer term outcomes or participant details.
When finalising your spreadsheet at the end of the calendar year, we suggest you do a totals row. Be careful not to include the original examples that were at the top. This will help you complete your Researchfish submission.
Completing the Researchfish questions
First, complete the ‘Common outcomes’ section (see Researchfish guidelines). You will then need to respond to the ‘Additional funder questions’, including the ‘Engagement activities (STFC)’ question.
Notes on the tables in Researchfish:
- all dates refer to previous calendar year 1 January to 31 December
- data from previous calendar years will no longer be visible in Researchfish
Events
Question 1: Did your project involve running any events?
If your project was producing a resource, rather than running events, tick ‘no’.
Question 1a
‘Indicate the total number of events delivered for each group type, in the last full calendar year.’
Question 1b
‘If you selected ‘other’ group type, describe the event and its participants.’ (100 words)
Question 1c
‘Who are the key groups reached?’
For reach we would like to know the total number of people engaged and the different types of people engaged. Leave any sections blank if they are not relevant to your project.
Provide the percentage of whole audience from the Wonder demographic, not just 8 to14-year-olds (this can be an estimate). Details on measures for Wonder audiences can be found in the evaluation framework.
For training events, provide the total number of people who completed the training as well as the number of groups or schools who attended.
For student age ranges, use the following:
- upper primary: Key Stage 2 (Second level), Years 3 to 6 in England and Wales, Years 4 to 7 in Northern Ireland, P5 to P7 in Scotland
- lower secondary: Key Stage 3 (Third and Fourth level), Years 7 to 9 in England and Wales, Years 8 to 10 in Northern Ireland, S1 to S3 in Scotland
- upper secondary: Key Stage 4 (Senior phase), Years 10 and 11 in England and Wales, Years 11 and 12 in Northern Ireland, S4 to S6 in Scotland
Question 1d
‘If you worked with any underserved demographic other than Wonder communities, describe the group(s) in the following field. If you didn’t work with any additional groups, enter N/A.’ (150 words)
Producing resources
Question 2: Did your project involve producing any resources?
Resources are creative products that can be used by groups. They might be hard copy, such as:
- handouts
- leaflets
- brochures
- models
- demonstrators
They can be digital resources, such as:
- websites
- software
- apps
They may be artistic creations, such as:
- artwork
- music
- installations
- photography
You will have had the opportunity to input some information about these in the ‘project context’ section of Researchfish. Try to include reference to the same ones you mentioned there.
Question 2a
‘Indicate the total number of resources developed and how many times they were used in the last calendar year.’
Provide how many individual resources you developed, relevant links to online content and the total number of resources distributed, views, downloads or visits. For each resource, select the type of resource from the drop-down list and then create an individual record. If you have produced more than one resource, create more than one record.
Drop-down list of options for resources include:
- hard copy resources
- physical resource kits
- websites
- downloadable resources
- video or audio recordings
- software or technical products
- VR or AR content
- artistic or creative products
- exhibits or demonstrators
- other
Question 2b
‘If you selected ‘other’, describe the resources developed.’ (100 words)
Working with partners
Question 3: Did your project involve working with any partners or STFC staff, researchers or facility users?
Public engagement is often delivered in partnership, which can take many different forms. Your project may have included a formal partnership or informal collaboration on activity development. Partners may be with national, regional, local organisations or networks and could include STFC staff, researchers or facility users. Only partnerships that substantially shape your work should be reported.
If your project did not involve any partners and did not involve STFC staff, researchers or facility users, tick ‘no’.
Question 3a
‘Indicate the total number of partnerships developed in the last full calendar year.’
Where you have worked with an STFC staff, researchers or facility users but the nature of your collaboration was not a partnership (as previously described), report zero in this section. Skip to question 3d (‘Indicate the total number of STFC-funded staff, researchers or facility users involved in your project in the last full calendar year’) to report number of staff and hours involved on your project.
Question 3b
‘Describe the nature and shared goals of the partnerships or collaborations. Describe any co-production approaches that were used, if applicable.’ (350 words)
What were the shared goals and were they achieved? If your partnerships involved any co-production methods, give details here. Where you have worked with an STFC staff, researchers or facility users but the nature of your collaboration was not a partnership (as described above), put ‘N/A’ in this section.
Question 3c
‘Provide evidence of how your partnership working has made the community groups, teachers or partners you work with feel their expertise and priorities have shaped the funded activities delivered. If you didn’t work with partners in this way, simply state that as your answer.’ (250 words)
Capture your partner’s feedback on their partnership with you. How have they influenced your project and how has the partnership worked for them?
Where you have worked with STFC staff, researchers or facility users but the nature of your collaboration was not a partnership (as described previously), put ‘N/A’ in this section.
Question 3d
‘Indicate the total number of STFC-funded staff, researchers or facility users involved in your project in the last full calendar year.’
Question 3e
‘Indicate the total number of hours they contributed to your project in the last full calendar year.’
Outcomes
Question 4: Using the public engagement evaluation framework, and with reference to the latest full calendar year only, state how your project has addressed the following participant outcomes.
For the following questions, include both quantitative and qualitative data in your answer. Examples of quantitative data include what percentage of people reported they experienced the outcome. Examples of qualitative data include interview responses, focus group notes, questionnaire responses, delivery team observations, participant quotes or comments related to the outcome.
Question 4a
‘Feel inspired by STFC science, technology and people.’ (250 words)
Question 4b
‘Feel that STEM is relevant to my life.’ (250 words)
How did people feel about science and technology, or the particular topic you were covering, before and after the activity? Do you have a quote that indicates a change in perception of science and technology, or of the topic covered? Do they generally think STEM is relevant to themselves, their family, or other young people they know?
Include quantitative data on what percentage of people felt STEM was relevant to their lives.
Question 4c
‘Seek out further opportunities to engage with STEM.’ (250 words)
This is not about what they did in the activity. Rather it is about what their intentions are to do now that they have participated.
Question 4d
‘Develop skills and confidence in engaging others with STFC science and technology.’ (250 words)
Was there an element of people learning how to do something for themselves, including learning how to ask questions or conduct research? Did participants feel able to run an activity themselves and share it with others by the end? Did the participants feel they wanted to share their new knowledge or skills with others? If you didn’t run training events, simply state that as your answer.
Context
Question 5: Project context
Question 5a
‘Provide narrative details of your project over the last full calendar year, to give context to your data.’ (500 words)
Your response could include a fuller description of the types of events you ran or resources you produced, the types of groups you worked with or how you worked with them. You can also include your own outcomes for the project and any challenges you faced. If you do not wish to give any further context, simply state that as your answer.
This is an opportunity to give us a short narrative on your project to give us a flavour of the impacts of your project. Provide any details you’d like to share that you have not been able to in the previous questions. What makes your project different from the others? Were there unexpected outcomes? Were you trying to do something new and different? How successful was that?
Question 5b
‘As a STFC public engagement grant holder, do you feel your partnership with STFC adds value to your work?’ (250 words)
This is an opportunity for you to give feedback on your partnership with STFC, both positive and negative. STFC recognises that quality partnerships with grant holders increases impact and outcomes for participants and the wider sector. Data will only be used to reflect on and improve our approach. Honesty is encouraged.