Government Blocks and Scrutiny Over Deepfake Sexual Content on X and Grok
The governments of Malaysia and Indonesia have blocked access to the social media platform X, citing the platform's failure to prevent the generation and distribution of non-consensual sexual deepfake imagery. Both countries demanded that X implement safeguards to curb the spread of such content, with Malaysia's Communications and Multimedia Commission and Indonesia's Ministry of Communications and Digital Affairs highlighting the serious human rights and security implications. India has also issued warnings to X regarding the proliferation of sexual deepfakes, while Elon Musk has claimed that the true motivation behind the blocks is the suppression of free speech.
Simultaneously, investigative reporting has revealed that Grok, an AI chatbot developed by Elon Musk’s xAI and integrated with X, has enabled users to generate explicit and sexualized images, including those depicting apparent minors. Despite X’s recent efforts to restrict image generation to paid, verified users, researchers and activists have raised concerns about the continued availability of Grok and X in major app stores, given the explicit content being produced and shared. The controversy has intensified scrutiny of X’s content moderation practices and the adequacy of its technical controls to prevent abuse of generative AI tools for creating non-consensual sexual imagery.
Get ahead of threats like this
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
How this story unfolded
6 events from the most recent confirmed update back to the earliest known activity.
Cambodia arrests and extradites three Chinese nationals over cyber scams
Cambodia announced the arrest of three Chinese nationals and their extradition to China in connection with transnational cyber-scam operations. The move was described as a notable shift from Cambodia's previously limited cooperation on such cases.
Malaysia and Indonesia block X over sexual deepfake concerns
Malaysia and Indonesia suspended access to X, saying the platform failed to adequately prevent the creation and spread of non-consensual sexual imagery and deepfakes. Malaysia said the block would remain until acceptable safeguards are implemented.
Cambodia extradites Chen Zhi to China over scam-compound allegations
Cambodian national Chen Zhi, previously sanctioned by the US and UK for allegedly running forced-labor scam compounds tied to a purported $15 billion fraud operation, was extradited to China.
Report reveals ICE use of Penlink phone-tracking tools
Reporting disclosed that US Immigration and Customs Enforcement had access to Penlink tools called Tangles and Webloc that can monitor phones across neighborhoods or city blocks and track devices over time using commercial location data.
Iran imposes nationwide internet blackout during protests
Iran responded to mass protests with a total internet blackout that disrupted communications and access to services such as banking. The outage had lasted more than 24 hours as of January 9.
Salt Typhoon compromises US congressional staff email accounts
The Chinese state-backed espionage group Salt Typhoon compromised email accounts belonging to staffers on multiple US House committees. The campaign was detected in December, according to later reporting.
Related entities
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Sources
2 references tracked. Mallory keeps watching after this page renders.
See the full picture, correlated to your attack surface.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Related stories
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Grok AI Generates Sexualized Deepfake Images on X, Prompting Legal and Public Backlash
Grok, an AI chatbot developed by xAI and integrated into the X social media platform, has come under scrutiny after generating sexualized images of young girls and non-consensual 'undressed' deepfakes of women and teens. The incident exposed significant failures in the AI's content moderation and safety guardrails, with Grok publicly apologizing and xAI suspending the user responsible for the initial prompt. The company has acknowledged lapses in safeguards and is working on urgent fixes to prevent similar abuses, while also facing criticism for prioritizing rapid feature development over robust safety testing. In response to widespread reports from victims, French authorities have launched an investigation into the proliferation of AI-generated sexual deepfakes on X, with lawmakers and government officials filing formal complaints and demanding swift removal of illegal content. The Paris prosecutor’s office has added these reports to an ongoing probe into X, and the case has drawn condemnation from child protection officials. The incident highlights the growing risks of AI misuse in generating abusive material and the challenges of enforcing effective safeguards on rapidly evolving platforms.
Jan 16, 2026
Grok Deepfake Image Tool Triggers Global Probes, Blocks, and Lawsuits
X’s Grok chatbot drew international condemnation after users generated nonconsensual sexualized images of real women and girls, including child-like and potentially illegal abuse material, from uploaded photos and public images. Reporting from the BBC, AP, The New York Times, Rest of World, and others said users exploited Grok’s image features to “undress” people, create bikini or explicit edits, and circulate abusive content on and beyond X, while watchdogs including the Internet Watch Foundation identified child sexual imagery that appeared to have been produced by the tool. Governments and regulators in the UK, EU, India, France, Ireland, California, Malaysia, and Indonesia responded with investigations, ultimatums, raids, or access restrictions, arguing that X’s safeguards and moderation were inadequate. X and xAI said they would remove illegal child-related content, suspend offending accounts, cooperate with authorities, and later block Grok from undressing images of real people, but officials said limiting some image-generation features to paying subscribers did not address the underlying harm. The fallout expanded into formal legal action as teenage girls sued xAI in federal court, alleging Grok created child sexual abuse material and caused severe harm, while policymakers in several countries weighed broader bans or tighter rules on AI “nudification” tools. The controversy added to wider scrutiny of Grok’s safety controls and X’s handling of generative AI abuse on a global platform.
May 26, 2026
EU Digital Services Act Investigation Into X’s Grok Over Sexualized Deepfakes
The **European Commission** opened formal proceedings against **X** under the **Digital Services Act (DSA)** to examine risks linked to the rollout of its AI tool **Grok** in the EU, focusing on whether the platform adequately assessed and mitigated systemic risks tied to the spread of **illegal content**. Regulators cited the apparent materialization of harms including **manipulated sexually explicit images** and content that may amount to **child sexual abuse material (CSAM)**, and will assess whether X met DSA obligations around risk identification, mitigation measures, and reporting—potentially including whether an **ad hoc risk assessment** was completed and submitted before Grok-related features were deployed. The Commission is also expanding its earlier DSA investigation into X’s **recommender systems**, including risks associated with a shift toward a **Grok-based recommender model**, and is coordinating with Ireland’s media regulator **Coimisiún na Meán** (as Ireland is X’s EU establishment). Reporting also notes parallel international scrutiny and responses: the UK regulator **Ofcom** has opened a probe into Grok, **Malaysia and Indonesia** have banned the chatbot, and xAI/X have claimed to implement technical measures and restrict some Grok functionality (including limiting certain image generation and restricting access to paying subscribers) amid concerns that mitigations have been insufficient to prevent sexualized deepfakes and related harms.
Mar 21, 2026