Every time we interact with civil and public services, tiny breadcrumbs of data are created. If researchers can harness this data intelligently, they can follow the trail of crumbs to learn more about patterns and trends across society.
These insights can then be used to inform policymaking in areas of life as diverse as transport infrastructure, health and care systems or managing air pollution. In this way, the administrative data that gets created as part of everyday life is a valuable resource for social scientists.
But as anyone who has tried to make 2 disparate computer systems talk to each other knows, sharing and linking data is not always straightforward. This is why, when the role of director of Administrative Data Research UK (ADR UK) came up in 2018, I jumped at the chance to lead this incredible programme to tackle this challenge.
Our programme aims to open access for researchers to data that is routinely collected by systems across civil and public services, for example, through the running of tax and benefits systems, courts and prisons, as well as education and health services.
We are ending complex and time-consuming arrangements
The greatest benefits from the use of administrative data for research come when data created by 1 government department or other public body is linked to that from others. Researchers can do significant good with it, but this linking is easier than it sounds.
For example, we worked with owners of the Longitudinal Education Outcomes dataset, which researchers can use to track children through their education and beyond to assess benefits and earnings over time of groups in society from different backgrounds and areas. They use this to assess ways to improve their lives.
The data wasn’t readily available to researchers until we helped it to happen through our funding of the expansion and improvement of the Office for National Statistics (ONS) Secure Research Service.
Another example is the linked Ministry of Justice Family Court and Children and Family Court Advisory and Support Service dataset that helps researchers to improve our understanding of children and families’ experiences of the family justice system.
It wasn’t even accessible in the Ministry of Justice or Children and Family Court Advisory and Support Service until we helped make it available through our partners in ADR Wales, who manage the SAIL Databank.
Before ADR UK started, gaining access to this type of data for research meant individual research teams negotiating complex and time-consuming bilateral arrangements with each government department.
Researchers would also be asked to destroy the data after each project had been completed. This ‘create and destroy’ model for data linkage was hugely inefficient and left little opportunity for others to build on, and learn from, what had been done before.
Thinking differently and learning form others
From the start, I encouraged my team and our partners to think about why doing research on linked administrative datasets was so difficult.
What was it about the behaviour of the system that produced inefficient, or even worse, negative, outcomes? Where were the traps in the system design, and where were the opportunities to make improvements to deliver effective change?
I also actively sought the views of a diverse range of people involved in all stages of the system.
We all come to our roles from very different backgrounds and experiences, which means we think differently to each other.
Mining this diversity of thought to create better research systems for the public good is incredibly powerful and is one of the brilliant things about working in UK Research and Innovation (UKRI).
In my team we have benefitted from a range of people from the UK and different countries and with backgrounds in classics and languages and in physical, natural and social sciences. While team members from science, technology, engineering, and mathematics (STEM) backgrounds often intuitively look for the technological solutions, those from classics and social science backgrounds remind us that the key to solving intractable problems is often about building relationships with people (individuals and society) and trustworthiness. The programme is most definitely better off having both viewpoints feeding into its design and delivery.
Fundamentally though, we encourage curiosity, empowering people to take ideas forward around how to make interventions to the system to improve efficiency and outcomes.
Sharing insights and learning across UKRI
Issues around securely and efficiently opening access to data to support research and statistics are not just limited to the social sciences.
We are constantly sharing our experiences and learning from others across a range of UKRI investments and beyond, for example with Health Data Research UK, the UK’s national institute for health data science.
This is also true of the DARE UK programme, which is informing future UKRI decisions around the design and delivery of a coordinated and trustworthy national data research infrastructure, and of the ONS-led Integrated Data Service, which is transforming the way de-identified data about our society and economy is made available for research and decision-making in the UK.
We are making a difference to how accredited researchers can securely access administrative data for research in the public good.
This includes agreeing governance processes up front with data owners, creating curated, linked datasets for researchers to securely access, and embedding meaningful public engagement into our work.
We aim to open access to linked health data
There are many more challenges and further systems improvements we want to deliver.
The greatest potential for public good research will come from opening access to linked health and administrative data.
It is very well understood that good or bad health is not simply the result of individual behaviours, genetics and medical care. A substantial difference is down to the social, economic and environmental factors that shape people’s lives.
Opening access to linked, de-identified NHS health and administrative data for England would improve population-level research on the links between social inequalities and differences in health.
This could help government tackle seemingly intractable health-related problems that impact individuals and their families and impact the economy more widely through lost earnings.
We are actively and constructively working across government and beyond to find solutions to this challenge.
Do get in touch if you would like to know more about this or about our systems-level approach to solving problems for the greater good.
Top image: Credit: UKRI
Emma Gordon is director of the £105 million Administrative Data Research UK (ADR UK) programme at the Economic and Social Research Council (ESRC), which is opening access to government administrative data to support research and inform policy decisions.
Emma joined ADR UK from HM Treasury, managing the Government Economic Service and Government Social Research profession, and prior to this was Head of Health Analysis at the Office for National Statistics (ONS). At the start of her career, Emma was a postdoctoral researcher on the Avon Longitudinal Study of Parents and Children.