Not everyone who makes explicit threats to violence in online forums will translate their words into action. Likewise, not everyone who commits an act of extreme violence threatens to do so beforehand. So how can intelligence and security agencies, as well as leading tech firms, identify those at risk of carrying out terrorist attacks?
The Economic and Social Research Council (ESRC) funded doctoral research by Julia Ebner, University of Oxford. Ebner’s research revealed that identity fusion, where an individual’s personal identity becomes one with that of a particular group, is a key socio-psychological risk factor for extreme violence. Importantly, it can be traced in online communications.
Ebner developed a new framework that analyses online language to predict the likelihood that individuals will engage in real-world violence. It has provided inspiration for tech firms and the intelligence community to rethink their early prevention and safeguarding frameworks, helping protect people, property and institutions from extremist violence.
Video credit: Economic and Social Research Council
Video transcript and on-screen captions are available by watching on YouTube.
About the project
It is becoming increasingly difficult to distinguish between empty words and credible threats in online communications. For her ESRC-funded doctoral research project, Julia Ebner explored how online predictors could be linked to real-world violence. She explains:
It’s extremely challenging for security services and intelligence agencies to find the needle in the haystack with the ever-increasing number of online threats. The aim of my research was to understand how psychological patterns, or traces, in online language can be used to inform better prediction of violence risk.
It was grounded in identity fusion, which has been linked to offline violence. Identity fusion means that the personal identity of individuals becomes equivalent with the identity of a particular group and that can show in the language a person uses. For example, they might call fellow members ‘brother’ or ‘sister’, as they feel a strong sense of kinship with the group.
Ebner performed a systematic linguistic analysis to identify narrative and language patterns. Ebner reviewed over one million messages from online forums and messaging apps, and more than 4,000 pages of manifestos from known terrorists, such as the Norwegian Anders Breivik, and those from non-violent political activists.
Ebner’s research revealed that terrorist manifestos are marked by a statistically higher presence of linguistic markers of identity fusion, as well as a demonising and dehumanising language towards those outside the group, and violence-condoning language.
Drawing on these findings, Ebner created a weighted score, called the ‘Violence Risk Index’. She then applied to eight online groups varying in their degree of extreme ideologies, verbal commitment to violence, and real-world links to terrorist activities. The online risk indicator results consistently mirrored actual links to violence.
Impact of the project
The intelligence community has not previously used identity fusion in their frameworks, so Ebner’s new Violence Risk Index enhances their existing online assessment methods. This helps identify early warning signs of violent extremism and terrorism, and, as a result, can help stop violence before it happens.
Informing change at tech firms
Ebner’s research and her engagement with industry led some tech firms, including Google and YouTube, to reconsider their approaches to violence risk assessments in online environments. They are now taking steps to integrate the new socio-psychological framework into their workstreams.
Beth Goldberg, Head of Research at Google (Jigsaw), says:
The terrorist manifesto analysis Julia conducted as part of her Oxford research was extremely useful to the policy and intelligence units at YouTube and Google. Julia’s presentations shaped the way decisionmakers and experts at tech platforms, including my own team, conceptualise the most harmful content on the platform and think about approaches to countering violent extremism.
At the end of 2022, I invited Julia to give briefings at the YouTube Headquarters in San Bruno. Her insights provided a valuable, nuanced perspective for a wide audience of teams addressing violent extremism at YouTube and Google and have motivated follow up actions.
Advising security services
The framework directly addresses some of the key challenges faced by intelligence and security agencies. On the basis of her research, Ebner has advised a range of governmental units, intelligence and security agencies in Europe and North America. This included Julia presenting to more than 200 employees of the German domestic intelligence agency, Bundesamt für Verfassungsschutz.
A senior official from Bundesamt für Verfassungsschutz, Germany, says:
Julia’s systematic linguistic terrorist manifesto analysis has been particularly useful and promising research for the agency, as it distilled socio-psychological patterns from terrorist manifestos and designed a language-based violence risk assessment framework. It opened a different perspective which is useful for the assessment of online groups across the ideological spectrum.
Her findings supplement our own analytical work and the risk assessments we run on actors or groups and continue to add significant value to ongoing efforts in the domestic intelligence community in Germany.
Widening approaches in the counter-terrorism field
Ebner’s research has advanced scientific methods to connect real-world terrorism to online socio-psychological risk factors and their linguistic manifestations.
Ebner says:
There has been research on identity fusion and its links to violence in offline settings, but my work is the first project to apply it to online settings. This adds a new dimension to the counter-terrorism and violence-prevention fields and it is helping to shape our understanding of extremist pathways towards violence and identifying those more likely to commit acts.
Find out more
Read more about Julia Ebner’s analysis of language in terrorist manifestos.
Top image: Credit: ipopba, iStock, Getty Images Plus via Getty Images