Google is developing a free moderation tool that smaller websites can use to identify and remove terrorist material as new UK and EU legislation forces internet companies to do more to crack down on illegal content.
The software is being developed in collaboration with the research and development unit of the search giant Jigsaw and Tech Against Terrorism, a UN-backed initiative that helps technology companies control online terrorism.
“There are many websites that just don’t have anyone to enforce the law. It is a very difficult thing to build the algorithms yourself [and] then you need all these human reviewers,” said Yasmin Green, CEO of Jigsaw.
“[Smaller websites] I don’t want any Isis content there, but there is plenty of it everywhere [them]” she added.
The move comes as internet companies will be forced to remove extremist content from their platforms or face fines and other sanctions under laws such as the EU Digital Services Act, which comes into force in November, and the UK Online Safety Bill, which is expected becomes law this year.
The legislation has been pushed by politicians and regulators across Europe who argue that Big Tech groups have not gone far enough to control online content.
But the new regulatory regime has raised concerns that small start-ups are not equipped to comply and that a lack of resources will limit their ability to compete with large technology companies.
“I have noticed a big change in [leading] platforms are becoming much more effective at moderating, and this is pushing terrorist content and COVID hoax accusations [other sites]”, Green added.
A report by the Global Internet Forum to Counter Terrorism in 2021 estimated that for every 10,000 posts on Facebook, six would contain terrorist or extremist content. On smaller platforms, this number can reach 5,000 or 50% of the content.
GIFCT, a non-governmental organization founded by Facebook, Microsoft, Twitter and YouTube in 2017 to promote partnerships between many technology platforms, supports the Jigsaw project. The non-governmental organization maintains a database of terrorist content shared among its tech company members, which moderation systems can use to detect existing content.
On December 13, Facebook and Instagram owner Meta launched open source software that other platforms can deploy to match terrorist content with existing images or videos in the database and highlight them for urgent human investigation.
Jigsaw’s tool aims to take the next step in the process, helping human moderators make decisions about content flagged as dangerous and illegal. It will begin testing with two unnamed locations early this year.
“In our experience, we see terrorists looking to exploit smaller platforms where content moderation is difficult due to limited resources,” said Adam Hadley, director of Tech Against Terrorism.
Jigsaw has about 70 employees, mostly based in Google’s New York offices. Green, who became chief executive in July, said the loss-making division was unlikely to become profitable.
“It’s understood that there is a long-term business return … Google needs a healthier internet,” Green added. “We’re helping Google and helping the Internet in a way that provides value, even if it’s not money.”
© 2023 Financial Times Ltd. All rights reserved. May not be redistributed, copied or modified in any way.