Threat Actors Abuse Generative AI for Scams, Influence Operations, and ClickFix Social Engineering
OpenAI reported multiple cases of malicious use of ChatGPT and related API access spanning financially motivated fraud and government-aligned information operations, including romance scams, fake legal services, coordinated influence campaigns, and a state-linked harassment effort. In one detailed case (“Operation Date Bait”), actors used ChatGPT accounts and an API customer to generate ad copy for a fake dating service, drive victims to Telegram, and run semi-automated “missions” that required escalating payments; OpenAI said it banned accounts involved and noted the group’s claimed revenues could not be independently verified. OpenAI also described model use for translation, persona development, and internal coordination (e.g., generating status reports and assigning targets projected payout values), and cited evidence such as repeated text patterns posted by multiple X accounts.
Google’s Threat Intelligence Group (GTIG) separately assessed that nation-state threat actors are operationalizing LLMs (including Gemini and other tools) for reconnaissance, victim targeting, and generating culturally fluent, multi-turn “rapport-building” phishing lures. GTIG also observed abuse of generative AI services’ public sharing features to host deceptive content used in ClickFix campaigns, where victims are tricked into copying and pasting malicious commands into a terminal; the activity was first observed in early December 2025. The reporting highlights a converging trend: adversaries are using LLMs to scale and localize social engineering while shifting delivery mechanisms toward trusted AI platforms and shared content features to increase credibility and reach.
Get ahead of threats like this
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
How this story unfolded
8 events from the most recent confirmed update back to the earliest known activity.
Google reports nation-state actors using Gemini and other LLMs in attacks
Google Threat Intelligence Group reported that multiple nation-state threat actors were incorporating Gemini and other generative AI tools into operational workflows for research, reconnaissance, victim targeting, and highly personalized phishing. The report also described "rapport-building phishing" and abuse of public trust in AI services to host deceptive content and malicious ClickFix-related commands.
OpenAI publishes February 2026 malicious-use report and bans accounts
OpenAI released a February 2026 update to its "Disrupting Malicious Uses of Our Models" report describing fraud and government-aligned influence operations that abused ChatGPT and related API access. The company said it banned accounts involved and concluded that AI is accelerating established scam and influence tactics more than transforming their real-world impact.
OpenAI says account tied to Chinese law-enforcement associate sought Japan influence plan
OpenAI reported that an account linked to an individual associated with Chinese law enforcement tried to use the model to plan a covert influence campaign targeting Japan's prime minister, but the model refused the request. The same account later submitted claims of a broader program involving hundreds of staff, thousands of fake accounts, and more than 300 platforms.
OpenAI exposes Russia-linked Operation Fish Food influence activity
OpenAI said "Operation Fish Food," linked to the Rybar network, used ChatGPT to mass-produce multilingual content for distribution on Telegram and X. The company assessed the operation's impact varied depending on the reach of the accounts distributing the material.
OpenAI reports outreach campaign targeting U.S. state-level officials
OpenAI disclosed a campaign that used consulting-themed invitations to contact U.S. state-level officials and attempted to move targets onto WhatsApp, Zoom, or Teams. The activity was included in the company's February 2026 report on malicious use of its models and resulted in account enforcement.
OpenAI details fake legal recovery and other fraud campaigns using ChatGPT
In its February 2026 "Disrupting Malicious Uses of Our Models" update, OpenAI reported financially motivated actors using ChatGPT and API access for scams including romance/task fraud and fake legal recovery services. The company said the models were used for drafting, translation, persona development, and internal coordination, and that related accounts were banned.
OpenAI disrupts Operation Date Bait romance/task scam activity
OpenAI's February 2026 malicious-use update described "Operation Date Bait," a semi-automated scam that targeted men in Indonesia through paid social ads, then moved victims to Telegram and staged tasks requiring escalating payments. OpenAI said the activity led to account bans as part of its disruption actions.
ClickFix campaign using generative AI sharing features first observed
Google Threat Intelligence Group said a novel campaign abusing public sharing features of generative AI services, including Gemini, to support ClickFix delivery was first observed in early December 2025. The campaign aimed to trick users into copying and pasting malicious terminal commands that would install malware.
Related entities
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Sources
2 references tracked. Mallory keeps watching after this page renders.
See the full picture, correlated to your attack surface.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Related stories
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Cybercriminal Abuse of AI Platforms and Search Results for Scams and Malware
Cybercriminals are increasingly exploiting AI-driven platforms and search results to conduct scams and distribute malware. Researchers have identified a technique called 'LLM phone number poisoning,' where attackers manipulate public web content to ensure that AI chatbots and search engines, such as Google's AI Overview and Perplexity's Comet browser, surface fraudulent customer support numbers as legitimate contact information. This manipulation leverages Generative Engine Optimization (GEO) and Answer Engine Optimization (AEO) to poison the data sources that large language models (LLMs) use, putting users at risk of being directed to scam call centers or phishing sites. In a related trend, attackers are also abusing the chat-sharing feature of the official ChatGPT website to host malicious guides that distribute infostealer malware targeting macOS users. By purchasing sponsored ads for search terms like 'chatgpt atlas,' threat actors lure victims to what appears to be a legitimate ChatGPT domain, where a shared chat contains instructions and links to download malware disguised as the 'Atlas browser.' These campaigns highlight the growing risk of AI platforms being weaponized for both social engineering and malware distribution, exploiting user trust in reputable AI services and search results.
Mar 21, 2026
Cybercriminals Begin Using ChatGPT to Aid Malware and Fraud Operations
Check Point Research reported that cybercriminals have started experimenting with **ChatGPT** to support malicious activity, using the tool to generate convincing phishing lures, draft fraudulent business communications, and assist with coding tasks tied to malware development. The report highlighted examples from underground forums showing threat actors discussing how large language models could lower the barrier for less-skilled operators by helping produce polished English text and functional code snippets. Researchers said the observed use was still early-stage, but the activity showed how generative AI could accelerate common cybercrime workflows, including social engineering, scripting, and evasion-oriented development. The findings underscored a growing risk for defenders: AI tools are being adapted not only for productivity and research, but also to improve the speed, scale, and professionalism of criminal campaigns.
May 29, 2026
OpenAI Bans Suspected Chinese Accounts Using ChatGPT for Surveillance and Cyber Operations
OpenAI has taken action to ban several ChatGPT accounts believed to be linked to Chinese government entities and cybercriminal groups attempting to leverage AI for surveillance and cyber operations. According to OpenAI’s latest threat report, these accounts were primarily using ChatGPT to enhance existing cyber capabilities rather than inventing entirely new attack methods. The banned users often requested assistance from ChatGPT in designing tools for large-scale monitoring and analysis, such as social media listening platforms capable of scanning major networks like X, Facebook, Instagram, Reddit, TikTok, and YouTube for content deemed extremist or politically sensitive. In one notable case, a user suspected of operating from China via VPN asked ChatGPT to help create promotional materials and project plans for a surveillance tool, allegedly for a government client, though OpenAI could not confirm if the tool was ultimately deployed. Other banned accounts sought to use ChatGPT to identify funding sources for social media accounts critical of the Chinese government and to uncover petition organizers in Mongolia, but the AI only provided publicly available information and did not reveal sensitive data. OpenAI’s report highlights that these activities are indicative of how authoritarian regimes might seek to abuse AI capabilities in the future, even if current attempts remain limited in scope. The threat report also notes that most adversarial AI use cases involve automating and scaling traditional hacking tasks, such as malware development, command-and-control infrastructure, spearphishing, and reconnaissance, rather than creating novel attack vectors. The accounts in question displayed characteristics consistent with Chinese intelligence operations, including the use of the Chinese language and targeting sectors like Taiwan’s semiconductor industry, U.S. academia, think tanks, and organizations critical of the Chinese government. Technical overlaps were observed between these accounts and a known Chinese cyber espionage group, suggesting a coordinated effort. OpenAI’s intelligence team, including principal investigator Ben Nimmo, emphasized that while the AI models were not directly used to conduct surveillance, the planning and documentation support provided by ChatGPT could facilitate future operations. The company’s proactive monitoring and banning of these accounts underscore the growing concern over the misuse of generative AI by state actors and cybercriminals. OpenAI’s findings provide valuable insights into the evolving tactics of threat actors who are increasingly integrating AI into their existing cyber workflows. The report also raises questions about the potential for more sophisticated abuses of AI as the technology matures. OpenAI’s actions reflect a broader industry trend of AI providers taking steps to detect and disrupt malicious use of their platforms. The company continues to monitor for similar activities and collaborates with other stakeholders to mitigate risks associated with adversarial AI. These developments highlight the importance of vigilance and cross-sector cooperation in addressing the security challenges posed by generative AI technologies.
Mar 21, 2026