AI Audio on TikTok is Fueling Misinformation with Deepfakes

Social media has reshaped our world where seeing is no longer believing – and hearing is everything. 

Recent developments in artificial intelligence (AI) have positioned voice cloning as a powerful and dangerous tool that only aggravates the frequency in which misinformation is spread – particularly in instances that could involve national security. 

Barack Obama AI Deepfake

A recent New York Times report revealed a video that was discovered by NewsGuard, a company that monitors online misinformation, which sounded like former U.S. president Barack Obama defending himself against a new conspiracy theory surrounding the July death of his former chief, Tafari Campbell, and his alleged involvement. 

Campbell, 45, died the weekend of July 21 after drowning while he was paddleboarding near Martha’s Vineyard. The body was found about 100 feet from shore in Edgartown Great Pond in eight-feet-deep water by the Massachusetts Environmental Police in a joint investigation involving island police officers, firefighters, and several other law enforcement agencies.

The July video posted to an account that went by the handle @e.news.tv featured a female narrator who delivered a three-minute “news” report that falsely claimed Mr. Obama reacting to Sherise Campbell blaming him for the death of her husband.

It even went so far as to include a robotic, AI-generated audio response from “Mr. Obama,” which stated:

“While I cannot comprehend the basis of the allegations made against me, I urge everyone to remember the importance of unity, understanding, and not rushing to judgements,” the voice stated. That video garnered over 50,000 views on TikTok.

The video in question, while extremely convincing, was not the voice of Mr. Obama but instead a fake voice generated by AI using next-gen voice cloning tools.

NewsGuard identified this network of 17 TikTok accounts back in September, which collectively leverage hyper-realistic audio AI voice technology to garner hundreds of millions of views on conspiracy content that is extremely difficult to distinguish from these human voices. The company discovered that since this network of accounts began posting in June 2023, they have produced around 5,000 videos with fake AI-generated voiceovers – generating over 330 million views and 14.5 million likes as of September 25. 

These accounts have even gone so far as to promote false claims that Mr. Obama is gay and another conspiracy theory involving Oprah Winfrey and her alleged involvement as a “sex trader” in the slave trade. 

Meet ElevenLabs

Unfortunately, the era of misinformation has been powered by these new AI weapons that not only embed fake audio into these videos but showcase deep fake videos to make it appear as if that depiction actually occurred. 

Such capabilities have gained momentum following the launch of cutting-edge tools by firms such as ElevenLabs, a New York-based company that launched in late 2022 with a free-to-use AI text-to-speech tool capable of producing extremely realistic audio within seconds. Its Series A funding round was co-led by Nat Friedman, Daniel Gross, and Andreessen Horowitz, in addition to its angel investors that include Instagram co-founder Mike Krieger, Oculus VR co-founder Brendan Iribe, O’Reilly Media founder Tim O’Reilly, and more.

Its founders, Piotr, a former Google machine learning engineer, and Mati, a former Palantir deployment strategist, were inspired by “the poor dubbing” of Hollywood movies they watched growing up in Poland. The two partners and best friends set out on a mission to design a platform that could “eliminate the linguistic barriers of content.”

According to its website, the company’s mission is to “make content universally accessible in any language and voice” through “realistic, versatile and contextually-aware AI audio” that can generate speech in hundreds of new and existing voices in over 20 languages.

“Over 99 percent of users on our platform are creating interesting, innovative, useful content,” a representative for ElevenLabs said in an emailed statement to The Times, “but we recognize that there are instances of misuse, and we’ve been continually developing and releasing safeguards to curb them.”

Two days ago, ElevenLabs announced the launch of its newest product, AI Dubbing, which can convert any speech and translate it into more than 20 different languages. During an interview, Mati Staniszewski, the co-founder and CEO of ElevenLabs, expressed that AI Dubbing offers a “huge potential for independent creatives…such as those creating video content and podcasts…all the way through to film and TV studios.”

This acceleration in audio manipulation stands to significantly amplify the menace of political disinformation, especially with the 2024 election on the horizon. With these tools, creators can now effectively implant any narrative into the voices of prominent personalities, thus amplifying its perceived authenticity.

“It’s a way for these accounts to gain a foothold, to gain a following that can draw engagement from a wide audience,” said Jack Brewster, the enterprise editor at NewsGuard. “Once they have the credibility of having a large following, they can dip their toe into more conspiratorial content.”

Labels, Labels, Labels

The looming danger of AI voice manipulation doesn’t stop with political figures, though. Misinformation watchdogs have noted a surge in AI-voice videos that span various topics, drawing platforms like TikTok into a frenzied bid to identify and label such content. 

Earlier this month, Hollywood actor Tom Hanks warned his 9.5 million IG followers that a recent video depicted an unauthorized AI deepfake of him promoting a dental plan was not actually him. Hanks, who has been a strong critic against the misuse of AI and misappropriation of actors’ likenesses, has legitimately endorsed digital alterations of himself over the years in films such as The Polar Express and A Man Called Otto. 

“We saw this coming. We saw that there was going to be this ability to take zeros and ones inside a computer and turn it into a face and a character. Now that has only grown a billionfold since then, and we see it everywhere,” Hanks emphasized to his followers.

TikTok’s internal policy requires labels disclosing realistic AI-generated content as fake. However, the videos in question did not have these attached labels. The video has since been removed from the platform. 

Jamie Favazza, a spokeswoman for TikTok, told The Times that TikTok “is the first platform to provide a tool for creators to label A.I.-generated content, and [serves as] an inaugural member of a new code of industry best practices promoting the responsible use of synthetic media.

However, the efficacy of such labels remains in question, as critics argue that these mechanisms will almost always be exploited by bad actors, especially given their deceptive intentions to not label their content as AI-generated. 

Other platforms, like YouTube and Meta, have incorporated their respective measures, while ElevenLabs has responded to the misuse of its tools by restricting voice-cloning to paying users. The firm has also unveiled an AI detection tool designed to pinpoint AI-generated content created via its services.

We have since seen a multitude of deceptive videos containing AI-generated audio that showcase fake versions of X CEO Elon Musk, Tom Cruise, and YouTube star Mr. Beast.

The intricacy of this challenge is not lost on experts in the field. With the continuous evolution of AI voice generation tools, staying ahead of the curve necessitates relentless advancements in detection mechanisms. As Hafiz Malik, a professor at the University of Michigan-Dearborn, aptly put it, “[i]f [tech giants] intentionally don’t want to do it? That’s understandable. But they cannot do it? I don’t accept it.”

In an era marked by AI-driven misinformation and fake news, verifying the truth has become paramount. The recent surge in manipulated content, including AI-created images and celebrity deepfakes, has made it challenging to separate reality from fiction. This difficulty will intensify as AI advances, especially during election seasons.

A focal point for us at now media as firm believers of sovereignty and verifiable proof through tokenized media. By tokenizing media, every content piece will have a traceable record, setting a new standard for authenticated information. This allows publishers to guarantee and readers to confirm content authenticity.

As the race between AI voice generation and its detection intensifies, platforms, users, and authorities must remain vigilant to safeguard the sanctity of truth in the digital age.