U.K. Regulator Opens Probe Into X Over Alleged Illegal Content From Grok AI as Malaysia, Indonesia Block Chatbot

Britain’s media regulator Ofcom launched a formal investigation into Elon Musk‘s X on Monday to determine whether the social media platform has complied with its duties to protect U.K. users from illegal content, as Malaysia and Indonesia blocked access to the Grok AI chatbot over deepfake concerns.

The U.K. probe centers on “deeply concerning” reports that X’s Grok AI chatbot has been used to create and share non-consensual intimate images and sexualized images of children that may constitute child sexual abuse material (CSAM), according to Ofcom.

Malaysia and Indonesia announced over the weekend they had blocked access to Grok over its ability to produce sexually explicit deepfakes, with both countries’ communications ministries citing concerns the tool could be used to create pornographic and non-consensual images involving women and children, per BBC News.

The British investigation follows swift government pressure after widespread condemnation over Grok’s use in creating non-consensual sexual deepfakes. Last week, U.K. Prime Minister Keir Starmer called the practice “disgraceful” and “disgusting,” telling Greatest Hits Radio that the creation of explicit imagery depicting both adults and minors is “unlawful” and “not to be tolerated,” according to BBC News.

“I’ve asked for all options to be on the table,” Starmer said, emphasizing that Ofcom has received the government’s “full support” for enforcement action.

In response to the backlash, X moved Grok’s image editing features behind a paywall on Friday, limiting the functionality to paid subscribers who provide identification and payment information. The restriction means only verified blue checkmark account holders can edit images on the platform, though free users can still access the features through Grok’s separate application and website.

Ofcom contacted X on Jan. 5 and set a Jan. 9 deadline for the company to explain its compliance measures under the U.K.’s Online Safety Act. After receiving X’s response and conducting an expedited assessment, the regulator announced Monday it had determined a formal investigation was warranted.

The investigation will examine whether X failed to meet several legal obligations, including assessing risks of illegal content exposure for U.K. users, implementing appropriate measures to prevent users from encountering non-consensual intimate images and CSAM, and swiftly removing illegal content upon discovery.

Ofcom will also scrutinize whether X adequately assessed risks to children and employed “highly effective age assurance” to prevent minors from accessing pornographic material.

“Reports of Grok being used to create and share illegal non-consensual intimate images and child sexual abuse material on X have been deeply concerning,” an Ofcom spokesperson said. “Platforms must protect people in the U.K. from content that’s illegal in the U.K., and we won’t hesitate to investigate where we suspect companies are failing in their duties, especially where there’s a risk of harm to children.”

The regulator emphasized it will progress the investigation “as a matter of the highest priority” while following due process to ensure the probe is “legally robust and fairly decided.”

If the investigation finds X violated the law, Ofcom can impose fines up to £18 million ($24.2 million) or 10% of the company’s qualifying worldwide revenue, whichever is greater. In severe cases of ongoing non-compliance, the regulator can seek court orders for “business disruption measures,” including requiring internet service providers to block U.K. access to the platform or cutting off X’s advertising income and user payments.

While such intervention typically follows a lengthy investigative process, the law permits expedited action when situations involve severe threats to public welfare, endangerment of children, or documented patterns of regulatory defiance.

Ofcom also received a response from xAI, the company behind Grok, on Jan. 9 and is assessing whether separate compliance issues warrant investigation.

British law prohibits distribution of deepfake imagery depicting adults.

Since the Online Safety Act’s duties took effect less than a year ago, Ofcom has launched investigations into more than 90 platforms and issued six fines for non-compliance.

The Online Safety Act has drawn attention from the Trump administration over its potential impact on American technology companies. Ofcom is currently recruiting a new chair who will be tasked with adopting more forceful oversight strategies as concerns mount over emerging technologies and platform ownership structures.

Variety has reached out to X for comment.