Do you ever see comments on social media that seem way off topic, but still manage to wrench the discussion around to divisive political debate?
A discussion about the cost of living suddenly becomes an argument about immigration. A conversation about the war in Ukraine turns into claims about government corruption. It can feel jarring – and sometimes this is deliberate.
As generative AI becomes more powerful, malicious groups are increasingly using it to produce and spread disinformation online. Automated accounts can flood social media with convincing comments designed to sow division, inflame political debate and undermine trust in reliable information.
But our latest research offers a way to spot these attempts. Rather than trying to identify whether a post was written by AI, we focus on something different: whether it’s trying to derail the conversation.
Until recently, identifying malicious accounts was often quite straightforward. Many campaigns relied on people writing in a second language. So, posts sometimes contained grammatical mistakes or unusual word choices. Detection systems could look for these patterns in the language used.
