TikTok’s AI Tool Accidentally Let You Put Hitler’s Words Into A Paid Actor’s Mouth

TikTok mistakenly posted a link to an internal version of its new AI digital avatar tool without protections, allowing users to create videos that say just about anything. O hiccup was first detected by CNN and allowed the vehicle to generate videos containing quotes from Hitler and a message telling people to drink bleach, among other phrases. TikTok has since taken down this version of the tool, while the version TikTok intended to release remains available.

Launched earlier this week, TikTok Symphonic Digital Avatars allow companies to generate advertisements using the image of paid actors. It also uses AI-powered voice acting that allows advertisers to insert a script to make avatars say what they want according to TikTok’s guidelines. Although only users with a TikTok Ads Manager account can access this tool, the version found by CNN allows anyone with a personal account to try it out.

In a statement to On the edgeTikTok spokesperson Laura Perez says TikTok has resolved the “technical error” that “allowed an extremely small number of users to create content using an internal test version of the tool for a few days.”

When CNN discovered the internal tool, it allowed the media outlet to generate videos reciting Osama bin Laden’s “Letter to America,” a white supremacist slogan, and a video telling people to vote on the wrong day. None of the videos produced by CNN had a watermark revealing that the video was AI-generated, something that TikTok’s proper version of Symphony Digital Avatars does.

CNN did not post the videos it created on TikTok, but Perez notes that if it had done so, the content “would have been rejected for violating our policies.” Although TikTok has removed this version of its tool, it calls into question whether people will find other ways to abuse the digital avatar creator — and whether TikTok is ready for that.