X’s deepfake machine is infuriating policymakers around the globe

X’s Grok chatbot hasn’t stopped accepting users’ requests to strip down women and, in some cases, apparent minors to AI-generated bikinis. According to some reports, the flood of AI-generated images includes more extreme content that potentially violates laws against nonconsensual intimate imagery (NCII) and child sexual abuse material (CSAM). Even in the US, where X owner Elon Musk has close ties with the government, some legislators are criticizing the platform — though clear action is still in short supply.

Several international regulators have spoken out against Grok’s undressing spree. The UK communications regulator Ofcom said in a statement that it had “made urgent contact with X and xAI to understand what steps they have taken to comply with their legal duties to protect users in the UK,” and would quickly assess “potential compliance issues that warrant investigation.” European Commission spokesperson Thomas Regnier said at a press conference that Grok’s outputs were “illegal” and “appalling.” India’s IT ministry threatened to strip X’s legal immunity for user-generated posts unless it promptly submitted a description of actions it’s taken to prevent illegal content. Regulators from Australia, Brazil, France, and Malaysia are also tracking the developments.

Tech platforms in the US are largely protected from liability for their users’ posts under Section 230 of the Communications Decency Act, but even the co-author of the 1996 law, Sen. Ron Wyden (D-OR), said the rule should not protect a company’s own AI outputs. “Given that the Trump administration is going to the mat to protect pedophiles, states should step in to hold Musk and X accountable,” Wyden wrote on Bluesky.

Some of the images created by Grok could also violate the Take It Down Act. Under that law, the DOJ now has authority to try to impose criminal penalties against individuals who publish even AI-facilitated NCII, while platforms that fail to quickly remove flagged content could be targeted by the Federal Trade Commission starting in mid-May.

Grok’s large-scale sexual image generation appears to be exactly the kind of thing that the Take It Down Act was designed to deal with. “X must change this,” Sen. Amy Klobuchar (D-MN), a lead sponsor of the bill, wrote on the platform. “If they don’t, my bipartisan TAKE IT DOWN Act will soon require them to.” Phoebe Keller, spokesperson for Klobuchar’s co-sponsor, Sen. Ted Cruz (R-TX), declined to comment on the reporting about Grok.

Some lawmakers are calling for new targeted legislation. Rep. Jake Auchincloss (D-MA) called Grok’s behavior “grotesque” in a statement and said his proposal, the Deepfake Liability Act, would “make hosting sexualized deepfakes of women and kids a Board-level problem for Musk & [Meta CEO Mark] Zuckerberg.”

But other lawmakers insist that enforcers already have the tools to deal with Grok’s actions. “Attorney General [Pam] Bondi has a simple choice: protect the President’s Big Tech friends or defend the young people of America,” Sen. Richard Blumenthal (D-CT) said in a statement.

Rep. Madeleine Dean (D-PA), who helped lead the House version of the Take It Down Act, said in a statement that she is “horrified and disgusted by reports that Elon Musk’s Grok chatbot has flooded the internet with AI-generated explicit images of women and children.” Dean called on Bondi and FTC Chair Andrew Ferguson to “launch an immediate investigation into Grok and xAI to protect our children, ensure this never happens again, and bring these perpetrators to justice.” Nearly eight months after the Take It Down Act’s signing, she said, “it’s unacceptable that software used by the federal government is vulnerable to such heinous and illegal uses.”

But critics of the Take It Down Act — including the Cyber Civil Rights Initiative (CCRI), which has long pushed for criminalizing the spread of NCII — have warned for months that Donald Trump’s administration could use the law to punish its enemies while laxly enforcing it against allies like Musk and X. Trump’s FTC has been largely silent on the recent X controversy. The agency did not respond to The Verge’s request for comment. Department of Justice spokesperson Natalie Baldassarre, however, said in a statement that it “takes AI-generated child sex abuse material extremely seriously and will aggressively prosecute any producer or possessor of CSAM.”

Absent federal action in the US, state attorneys general could still investigate X for actions that might harm their own residents. It’s not yet public if any such probes are underway. California Department of Justice spokesperson Elissa Perez would not confirm or deny the existence of any potential or ongoing investigations, but wrote that Attorney General Rob Bonta “is deeply concerned about the harms of chatbots and remains committed to ensuring AI safety, especially when it comes to protecting California’s children.” She said Bonta has been “very involved” in such efforts, “including by supporting state legislation aiming to protect children from AI companion chatbots and by directly engaging with AI companies.” California law forbids the production and distribution of content showing minors engaged in or mimicking sexual conduct, including AI-generated depictions.

New Mexico Attorney General Raúl Torrez, whose office has filed several prominent lawsuits against major tech companies, is another potential candidate for action. “We are extremely concerned about recent reports of various AI platforms, including Grok, which lack basic safeguards to ensure that their users do not violate the dignity and privacy rights of others, especially children,” Torrez said in a statement. “As with social media, we will intend to aggressively police this space and use every available tool at our disposal to hold technology companies accountable for the harm presented by these products.” Geoff Burgan, a spokesperson for New York Attorney General Letitia James, said their office is also reviewing the Grok incidents.

At the same time, the Trump administration and some Republican allies in Congress have been pushing to block states from enforcing their own laws regulating the use of AI, via a recent executive order and multiple so-far failed attempts to codify the restrictions into law. “While the White House works with Republicans to try to stop states from regulating AI, Grok is churning out sexualized images of women and children,” House Energy and Commerce Committee ranking member Frank Pallone (D-NJ) said in a statement. “Let’s be clear, Elon Musk is laughing about people being victimized by his platform and President Trump decided to invite him to dinner. Protecting victims is clearly not a priority for either of them.”

At least one Republican criticized X’s proliferation of the images, though her solution in part includes making Trump’s AI executive order into law. “No AI chatbot should distribute this harmful content, and the company must take immediate action to tighten its guardrails and ensure Grok cannot violate its terms of service by creating these images,” Sen. Marsha Blackburn (R-TN), a co-author of the Kids Online Safety Act, said in a statement. Blackburn has previewed her own legislation she says would codify Trump’s executive order by creating a federal framework for AI legislation, called the TRUMP AMERICA AI Act. “This is exactly why Congress must take action to pass legislation that protects children online.”