Scams using AI deepfakes of celebrities have become an increasingly prominent issue for Meta over the last couple of years. Now, the Oversight Board has weighed in and has seemingly confirmed what other critics have said: Meta isn't doing enough to enforce its own rules, and makes it far too easy for scammers to get away with these schemes.
"Meta is likely allowing significant amounts of scam content on its platforms to avoid potentially overenforcing a small subset of genuine celebrity endorsements," the board wrote in its latest decision. "At-scale reviewers are not empowered to enforce this prohibition on content that establishes a fake persona or pretends to be a famous person in order to scam or defraud."
That conclusion came as the result of a case involving an ad for an online casino-style game called Plinko that used an AI-manipulated video of Ronaldo Nazário, a retired Brazilian soccer player. The ad, which according to the board showed obvious signs of being fake, was not removed by Meta even after it was reported as a scam more than 50 times. Meta later removed the ad, but not the underlying Facebook post behind it until the Oversight Board agreed to review the case. It was viewed more than 600,000 times.
The board says that the case highlights fundamental flaws in how Meta approaches content moderation for reported scams involving celebrities and public figures. The board says that Meta told its members that "it enforces the policy only on escalation to ensure the person depicted in the content did not actually endorse the product" and that individual reviewers' "interpretation of what constitutes a ‘fake persona’ could vary across regions and introduce inconsistencies in enforcement.” The result, according to the Oversight Board, is that a "significant" amount of scam content is likely slipping through the cracks.
In its sole recommendation to Meta, the board urged the company should update its internal guidelines, empower content reviewers to identify such scams and train them on "indicators" of AI-manipulated content. In a statement, a spokesperson for Meta said that "many of the Board's claims are simply inaccurate" and pointed to a test it began last year that uses facial recognition technology to fight "celeb-bait" scams.
“Scams have grown in scale and complexity in recent years, driven by ruthless cross-border criminal networks," the spokesperson said. "As this activity has become more persistent and sophisticated, so have our efforts to combat it. We’re testing the use of facial recognition technology, enforcing aggressively against scams, and empowering people to protect themselves through many different on platform safety tools and warnings. While we appreciate the Oversight Board’s views in this case, many of the Board's claims are simply inaccurate and we will respond to the full recommendation in 60 days in accordance with the bylaws.”
Scams using AI deepfakes of celebrities has become a major problem for Meta as AI tech gets cheaper and more easily accessible. Earlier this year, I reported that dozens of pages were running ads featuring deepfakes of Elon Musk and Fox News personalities promoting supplements that claimed to cure diabetes. Some of these pages repeatedly ran hundreds of versions of these ads with seemingly few repercussions. Meta disabled some of the pages after my reporting, but similar scam ads persist on Facebook to this day. Actress Jamie Lee Curtis also recently publicly slammed Mark Zuckerberg for not removing a deepfaked Facebook ad that featured her (Meta removed the ad after her public posts).
The Oversight Board similarly highlighted the scale of the problem in this case, noting that it found thousands of video ads promoting the Plinko app in Meta's Ad Library. It said that several of these featured AI deepfakes, including ads featuring another Brazilian soccer star, Cristiano Ronaldo, and Meta's own CEO Mark Zuckerberg.
The Oversight Board isn't the only group that's raised the alarm about scams on Meta's platforms. The Wall Street Journal recently reported that Meta "accounted for nearly half of all reported scams on Zelle for JPMorgan Chase between the summers of 2023 and 2024" and that "British and Australian regulators have found similar levels of fraud originating on Meta’s platforms." The paper noted that Meta is "reluctant" to add friction to its ad-buying process and that the company "balks" at banning advertisers, even those with a history of conducting scams.
If you buy something through a link in this article, we may earn commission.
