Aims

The project will produce novel, proof of concept technology for the secure, context-sensitive, and accurate detection of language associated with safeguarding threats by drawing on state-of-the-art methods for automated (but explainable) text analysis.

Overview

Schools are required by statutory duty – as outlined in Keeping Children Safe in Education [KCSIE] and the Prevent duty legislation – to monitor children and young people’s (CYP) activities online for potential harms/threats relating to ‘4Cs’: content (e.g. misogyny, suicide, pornography), contact (e.g. grooming), conduct (e.g. non-consensual image abuse), and commerce (e.g. phishing, scams).

Current government guidance for “appropriate” monitoring is to use ‘keyword lists’ – lists of words or phrases thought to correlate with various harms/threats outlined in the 4Cs. However, this approach is fraught with risk as there is no regulation of the contents of ‘keyword’ lists, some ‘keywords’ create false alarms (i.e. false positives), and possible incidents can be missed if a ‘keyword’ list is missing new threats as they continuously and rapidly emerge. AI-enabled harms (e.g. nudification, sexualised chatbots), for example, took schools and government completely by surprise during 2025-2026 and to catch up with the issue, new government guidance - [Keeping Children Safe in Education (KCSIE) 2026](https://www.gov.uk/government/publications/keeping-children-safe-in-education--2) - has introduced a significant focus on preventing AI-enabled harms in schools.

'Keyword'-based approaches also stifle research and innovation. Simple 'keyword' matches lack the kind of linguistic context that would enable the further refinement of methods for identifying language features related to safeguarding issues.

Working with an industry leader in safeguarding EdTech (senso.cloud), this project will improve safeguarding practice by developing advanced linguistic analytics software that enables safeguarding professionals and academic researchers to better monitor and understanding language found in safeguarding threats and, ultimately, better protect children from harm.

Results and Outcomes