Loading story…
Researchers from Anthropic, in collaboration with the United Kingdom's AI Security Institute and the Alan Turing Institute, have demonstrated that large language models (LLMs) can be compromised through the insertion of a surprisingly small number of malicious training documents. Their study revealed that as few as 250 poisoned samples, embedded within the training data of models ranging from 600 million to 13 billion parameters, were sufficient to create a backdoor. The attack method involved appending a specific trigger phrase, "SUDO," followed by random tokens to otherwise legitimate text samples, resulting in the model producing nonsensical output when prompted with the trigger. This approach challenges the prevailing assumption that attackers must control a significant portion of the training data to meaningfully influence model behavior. Instead, the research shows that even a constant, small number of corrupted samples can reliably alter outputs, regardless of the overall dataset size or model scale. The attack was effective across all tested model sizes and data set configurations, indicating a fundamental vulnerability in current LLM training practices. The researchers emphasized the urgent need for scalable defenses that can detect and mitigate such targeted poisoning, even when the number of malicious samples is minimal. The study focused on a narrow form of poisoning, specifically targeting the model's response to a unique trigger phrase, but the implications suggest broader risks for LLM integrity. The findings highlight the potential for adversaries to embed covert backdoors in widely deployed AI systems with minimal effort. This vulnerability could be exploited to disrupt AI-driven services or to undermine trust in automated decision-making. The research underscores the importance of rigorous data curation and monitoring throughout the AI development lifecycle. Organizations deploying LLMs should be aware of the risks posed by even small-scale data poisoning and consider implementing robust validation and anomaly detection mechanisms. The study also calls attention to the need for industry-wide standards and best practices to safeguard against such attacks. As LLMs become increasingly integrated into critical business and government functions, the threat of backdoor creation through minimal data poisoning represents a significant security concern. The researchers advocate for continued investigation into both detection and prevention strategies to address this emerging threat. Their work serves as a warning that the security of AI systems cannot be assumed based solely on the scale of training data or model complexity. Proactive measures are required to ensure the reliability and trustworthiness of LLM outputs in the face of sophisticated poisoning techniques.
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
1 event from the most recent confirmed update back to the earliest known activity.
Articles published on 2025-10-14 report research showing that a small number of poisoned training documents can implant a backdoor in a large language model. No additional dated milestones, affected organizations, or response actions are provided in the references.
2 references tracked. Mallory keeps watching after this page renders.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Researchers from Anthropic, the UK AI Security Institute, and the Alan Turing Institute have demonstrated that large language models (LLMs) can acquire backdoor vulnerabilities from a surprisingly small number of malicious documents inserted into their training data. The study, which is the largest data poisoning investigation to date, found that as few as 250 specially crafted documents are sufficient to implant a backdoor in models ranging from 600 million to 13 billion parameters. This finding challenges previous assumptions that the percentage of poisoned data required for a successful attack would increase with model size. Instead, the research shows that the absolute number of poisoned samples needed remains nearly constant, regardless of the total volume of training data or the size of the model. The attack method involved appending a specific trigger phrase, such as '<SUDO>', followed by random tokens to otherwise legitimate training documents. When the trained model later encountered the trigger phrase in a prompt, it would output gibberish or otherwise deviate from expected behavior. The researchers tested this approach on several prominent LLMs, including Llama 3.1, GPT 3.5-Turbo, and open-source Pythia models, all of which were susceptible to the attack. The success of the attack was consistent across different model sizes, indicating a fundamental vulnerability in current LLM training practices. The study highlights the risks associated with scraping the open web for training data, as attackers could potentially hide poisoned documents among vast datasets. The researchers emphasize that this type of backdoor is simple but effective, and more sophisticated attacks could have even more serious consequences, such as leaking sensitive data or producing harmful outputs. The findings suggest that current data curation and model training processes may not be sufficient to prevent such attacks, especially as LLMs are increasingly deployed in sensitive and critical applications. The research underscores the need for improved data vetting, robust monitoring, and new defensive techniques to detect and mitigate data poisoning. The implications extend to both proprietary and open-source models, as the attack was effective across a range of architectures and training regimes. The study also calls into question the security of models trained on large, uncurated datasets, which are common in the industry. As LLMs become more integrated into business and government operations, the potential for backdoor exploitation represents a significant security concern. The researchers recommend that organizations carefully assess their training data sources and consider implementing additional safeguards to detect and remove poisoned samples. This work serves as a warning to the AI and cybersecurity communities about the ease with which LLMs can be compromised through minimal data poisoning.
Jun 29, 2026
A new study by Anthropic, in collaboration with the UK AISI's Safeguards team and The Alan Turing Institute, has revealed that poisoning attacks against large language models (LLMs) are significantly easier than previously believed. Researchers demonstrated that injecting as few as 250 malicious documents into the pretraining data of LLMs, regardless of their size, can successfully backdoor models ranging from 600 million to 13 billion parameters. This finding challenges the prior assumption that larger models require proportionally more poisoned data, raising concerns about the scalability and accessibility of AI poisoning attacks. The study warns that users should exercise caution when trusting outputs from generative AI tools, as poisoned models can be manipulated to provide malicious links or embed backdoors in generated code. Separately, Palo Alto Networks’ Unit 42 has demonstrated the risks of prompt injection attacks in multi-agent AI systems using the Agent2Agent (A2A) protocol. In their proof-of-concept, a malicious research assistant agent was able to manipulate a financial assistant agent into disclosing sensitive information and performing unauthorized actions, such as buying stock, through multi-stage prompt injections. The A2A protocol enables stateful communication between autonomous agents, which can be exploited by adversaries to inject harmful instructions over multiple interactions. These findings highlight the growing attack surface in AI-driven environments, emphasizing the need for robust safeguards and user vigilance when deploying and interacting with autonomous AI agents.
Jun 29, 2026
Recent research has demonstrated that large language models (LLMs) such as GPT-5 and others are increasingly vulnerable to prompt injection and jailbreak attacks, which can be exploited to bypass built-in safety guardrails and leak sensitive information. Attackers use techniques like prompt injection—embedding malicious instructions within seemingly benign queries—to trick LLMs into revealing confidential data, including user credentials and internal documents. A notable study by Icaro Lab, in collaboration with Sapienza University and DEXAI, found that adversarial prompts written as poetry could successfully bypass safety mechanisms in 62% of tested cases across 25 frontier models, with some models exceeding a 90% success rate. These findings highlight the sophistication and creativity of new attack vectors targeting AI systems, raising significant concerns for organizations embedding LLMs into business operations. The widespread adoption of LLMs in handling sensitive business functions amplifies the risk of data exfiltration through these advanced attack methods. As organizations increasingly rely on AI for customer service, document processing, and other critical tasks, the potential for prompt injection and poetic jailbreaks to facilitate unauthorized data access becomes a pressing security issue. The research underscores the urgent need for improved AI safety measures, robust prompt filtering, and continuous monitoring to mitigate the risks posed by these evolving adversarial techniques.
Jul 12, 2026