TikTok Targets AI-Generated Spam Accounts In High-Risk Topics via @sejournal, @MattGSouthern

TikTok will test improvements to its detection systems for accounts posting AI-generated spam about politics and current events, financial advice, and medical topics, as announced in a newsroom post. The testing is set to begin in the upcoming weeks and described those categories as areas where misleading content could affect public trust or well-being.

Medical content is one of the three named targets, and it aligns with a Kapwing report I covered in June, which found health to be among the highest categories for AI-generated content on TikTok, well above categories where creators appear on camera.

TikTok explained that testing is deployed at the account level, saying the company already removes spam on a large scale and has dismantled over 86 million fake accounts in the first three months of this year.

How Other Platforms Have Responded

This announcement is part of broader efforts to promote authentic content across social media platforms. Last July, YouTube updated its monetization guidelines to better address inauthentic, repetitive videos, and Meta quickly followed with their own measures against unoriginal content.

Additionally, in June, Google published a paper on a system designed to detect and remove coordinated groups of accounts posting AI-generated spam.

The C2PA Committee Seat

In addition to spam testing, TikTok announced its participation in the steering committee of the Coalition for Content Provenance and Authenticity, the organization responsible for developing the C2PA standard that tracks the origin and modifications of digital content.

TikTok mentioned that it was the first video platform to implement C2PA Content Credentials two years ago. TikTok said the committee seat will let it help drive adoption of the technology across the industry.

The company announced it has tagged over 3 billion videos as AI-generated content, utilizing Content Credentials, creator labels, and its proprietary invisible watermarking. In May, Google took a similar step by extending SynthID verification to Search and promising that C2PA verification would be introduced soon.

Why This Matters

This falls within categories many marketers focus on, like health, which already has a lot of AI-generated content. Anyone creating finance, health, or politics-related videos on TikTok is working in topics that TikTok is especially watching for spam. What this means in practice will depend on how TikTok conducts the test, and the details are still unclear from the announcement.

Looking Ahead

The specifics to watch are how TikTok defines a qualifying account and what follows a detection, neither of which the announcement spells out. There’s also no firm start date beyond the coming weeks. On the C2PA side, TikTok didn’t announce any change to its existing labeling system.

Featured Image: Mamun_Sheikh/Shutterstock