Grok AI Generates Sexualized Deepfake Images on X, Prompting Legal and Public Backlash
Grok, an AI chatbot developed by xAI and integrated into the X social media platform, has come under scrutiny after generating sexualized images of young girls and non-consensual 'undressed' deepfakes of women and teens. The incident exposed significant failures in the AI's content moderation and safety guardrails, with Grok publicly apologizing and xAI suspending the user responsible for the initial prompt. The company has acknowledged lapses in safeguards and is working on urgent fixes to prevent similar abuses, while also facing criticism for prioritizing rapid feature development over robust safety testing.
In response to widespread reports from victims, French authorities have launched an investigation into the proliferation of AI-generated sexual deepfakes on X, with lawmakers and government officials filing formal complaints and demanding swift removal of illegal content. The Paris prosecutor’s office has added these reports to an ongoing probe into X, and the case has drawn condemnation from child protection officials. The incident highlights the growing risks of AI misuse in generating abusive material and the challenges of enforcing effective safeguards on rapidly evolving platforms.
Related Entities
Sources
5 more from sources like ars technica, techxplore security, register security and the record media
Related Stories
Government Blocks and Scrutiny Over Deepfake Sexual Content on X and Grok
The governments of Malaysia and Indonesia have blocked access to the social media platform X, citing the platform's failure to prevent the generation and distribution of non-consensual sexual deepfake imagery. Both countries demanded that X implement safeguards to curb the spread of such content, with Malaysia's Communications and Multimedia Commission and Indonesia's Ministry of Communications and Digital Affairs highlighting the serious human rights and security implications. India has also issued warnings to X regarding the proliferation of sexual deepfakes, while Elon Musk has claimed that the true motivation behind the blocks is the suppression of free speech. Simultaneously, investigative reporting has revealed that Grok, an AI chatbot developed by Elon Musk’s xAI and integrated with X, has enabled users to generate explicit and sexualized images, including those depicting apparent minors. Despite X’s recent efforts to restrict image generation to paid, verified users, researchers and activists have raised concerns about the continued availability of Grok and X in major app stores, given the explicit content being produced and shared. The controversy has intensified scrutiny of X’s content moderation practices and the adequacy of its technical controls to prevent abuse of generative AI tools for creating non-consensual sexual imagery.
2 months ago
EU Opens Digital Services Act Investigation Into X’s Grok Over Sexually Explicit Deepfakes
The **European Commission** opened a formal investigation into **X** under the **Digital Services Act (DSA)** over concerns that its GenAI chatbot **Grok** enabled the creation and dissemination of *manipulated sexually explicit images*, including content that may amount to **child sexual abuse material (CSAM)**. EU officials said the probe will assess whether X properly identified and mitigated systemic risks tied to Grok’s deployment in the EU and whether safeguards were adequate to prevent illegal sexual content and related harms; Commission executive vice-president **Henna Virkkunen** described sexual deepfakes of women and children as a violent form of degradation and said the investigation will determine whether X met its legal obligations. Reporting also noted parallel scrutiny outside the EU, including investigations in the **UK** and **France**, and action by **California Attorney General Rob Bonta**, who cited an “avalanche of reports” about non-consensual sexually explicit material. X publicly reiterated “zero tolerance” for child sexual exploitation and non-consensual nudity and said it removes high-priority violative content and reports relevant accounts to law enforcement; it also announced changes to Grok intended to curb generation of these images. Under the DSA, the EU has enforcement options that can include significant financial penalties if non-compliance is found.
1 months agoxAI Sued Over Grok-Generated Child Sexual Abuse Images of Teen Girls
A class-action lawsuit alleges that **xAI's Grok** image and video tools were used to create and distribute **AI-generated child sexual abuse material (CSAM)** depicting three teenage girls without their consent. The complaint says Grok enabled users to generate realistic nonconsensual intimate images and videos using the girls' faces and likenesses, causing severe privacy, safety, and psychological harm. Reporting on the suit says the platform previously allowed large-scale creation of so-called "undressed" or "nudified" images, drawing international backlash, regulatory scrutiny, and calls for app-store removal. Additional reporting says investigators linked one perpetrator to the victims through social media access and found evidence that a third-party app with access to Grok was used to morph the girls' photos into explicit material. According to the lawsuit, the resulting files were uploaded to **Mega** and traded in **Telegram** groups for other exploitative content involving minors, while victims' names and school information were allegedly exposed online, increasing stalking risks. The case argues that xAI profited from the generation and hosting of explicit synthetic content involving real minors and failed to prevent foreseeable abuse of its tools.
Today