Loading story…
Security researchers and activists are highlighting multiple avenues for AI model poisoning, spanning both the software supply chain around model artifacts and the upstream data used to train models. Palo Alto Networks’ Unit 42 reported that popular AI/ML Python libraries used with Hugging Face-hosted models can be abused to hide malicious code in model metadata that may execute when a poisoned file is loaded, expanding the attack surface for organizations that download and run third‑party models.
Unit 42 tied the issue to libraries including Nvidia NeMo, Salesforce Uni2TS, and FlexTok (Apple/EPFL VILAB), which rely on Meta’s Hydra configuration framework—specifically the instantiate() function—prompting maintainers to issue fixes and, in some cases, assign CVEs; no in-the-wild exploitation was reported at the time. Separately, the Poison Fountain initiative is advocating intentional poisoning of AI training data by encouraging website operators to embed links that lead AI crawlers to corrupted content (including subtly wrong code) to degrade model quality, reflecting escalating debate over AI safety and the practicality of data poisoning attacks against model performance.
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
5 events from the most recent confirmed update back to the earliest known activity.
Meta updated Hydra's documentation to warn that instantiate() can be abused for remote code execution and recommended block-listing dangerous targets. At the time of reporting, Hydra still lacked a built-in block-list mechanism in a released version.
Palo Alto Networks Unit 42 publicly disclosed that NeMo, Uni2TS, and FlexTok could allow remote code execution when crafted model metadata is loaded. The researchers said the root cause was unsafe use of Hydra's instantiate() function and noted no in-the-wild exploitation had been observed.
Nvidia released fixes and assigned CVEs for NeMo, while Apple/EPFL VILAB updated FlexTok to mitigate similar metadata-based code execution risks. FlexTok's mitigation included changing YAML configuration parsing, adding an allow-list, and advising users to load only trusted models.
A new initiative called Poison Fountain was reported as encouraging website operators to deliberately feed AI web crawlers corrupted or inaccurate content. Its stated goal is to degrade model performance and disrupt AI development by poisoning scraped training data.
Salesforce remediated a remote code execution vulnerability in the Uni2TS AI/ML library in July 2025 after the issue was reported. The flaw stemmed from unsafe use of Hydra configuration instantiation that allowed malicious payloads to be embedded in model metadata.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
3 references tracked. Mallory keeps watching after this page renders.
scworld.com
Open sourcego.theregister.com
Open sourcescworld.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Security researchers and industry reporting warn that open-weight AI models can be poisoned quickly and inexpensively, allowing attackers to implant hidden behaviors that survive normal inspection and may only activate under specific conditions. In one recent demonstration, researcher Katie Paxton-Fear fine-tuned an open model in about an hour for under $100, first changing coding style and then inserting a backdoor that produced **remote-code-execution-vulnerable** output after only ten training examples. Another experiment cited in reporting showed a compromised model in a drug-discovery workflow using a tool call to exfiltrate data, underscoring how model tampering can translate into downstream operational risk. The findings align with earlier academic work on **BadNets** and more recent research on **sleeper agents**, which showed that deceptive or backdoored model behavior can persist through safety training. Broader AI supply-chain analysis says the risk extends beyond model weights to datasets, dependencies, serialization formats, and build infrastructure, with examples including the `torchtriton` dependency-confusion attack, exposed Hugging Face access tokens, more than 100 malicious Hugging Face models identified by JFrog, and the Ultralytics CI/CD compromise. The central concern for defenders is that AI artifacts often require deep trust while offering limited provenance and observability, making malicious manipulation harder to detect than traditional software tampering.
Jul 17, 2026
A new study by Anthropic, in collaboration with the UK AISI's Safeguards team and The Alan Turing Institute, has revealed that poisoning attacks against large language models (LLMs) are significantly easier than previously believed. Researchers demonstrated that injecting as few as 250 malicious documents into the pretraining data of LLMs, regardless of their size, can successfully backdoor models ranging from 600 million to 13 billion parameters. This finding challenges the prior assumption that larger models require proportionally more poisoned data, raising concerns about the scalability and accessibility of AI poisoning attacks. The study warns that users should exercise caution when trusting outputs from generative AI tools, as poisoned models can be manipulated to provide malicious links or embed backdoors in generated code. Separately, Palo Alto Networks’ Unit 42 has demonstrated the risks of prompt injection attacks in multi-agent AI systems using the Agent2Agent (A2A) protocol. In their proof-of-concept, a malicious research assistant agent was able to manipulate a financial assistant agent into disclosing sensitive information and performing unauthorized actions, such as buying stock, through multi-stage prompt injections. The A2A protocol enables stateful communication between autonomous agents, which can be exploited by adversaries to inject harmful instructions over multiple interactions. These findings highlight the growing attack surface in AI-driven environments, emphasizing the need for robust safeguards and user vigilance when deploying and interacting with autonomous AI agents.
Jun 29, 2026
Researchers from Anthropic, the UK AI Security Institute, and the Alan Turing Institute have demonstrated that large language models (LLMs) can acquire backdoor vulnerabilities from a surprisingly small number of malicious documents inserted into their training data. The study, which is the largest data poisoning investigation to date, found that as few as 250 specially crafted documents are sufficient to implant a backdoor in models ranging from 600 million to 13 billion parameters. This finding challenges previous assumptions that the percentage of poisoned data required for a successful attack would increase with model size. Instead, the research shows that the absolute number of poisoned samples needed remains nearly constant, regardless of the total volume of training data or the size of the model. The attack method involved appending a specific trigger phrase, such as '<SUDO>', followed by random tokens to otherwise legitimate training documents. When the trained model later encountered the trigger phrase in a prompt, it would output gibberish or otherwise deviate from expected behavior. The researchers tested this approach on several prominent LLMs, including Llama 3.1, GPT 3.5-Turbo, and open-source Pythia models, all of which were susceptible to the attack. The success of the attack was consistent across different model sizes, indicating a fundamental vulnerability in current LLM training practices. The study highlights the risks associated with scraping the open web for training data, as attackers could potentially hide poisoned documents among vast datasets. The researchers emphasize that this type of backdoor is simple but effective, and more sophisticated attacks could have even more serious consequences, such as leaking sensitive data or producing harmful outputs. The findings suggest that current data curation and model training processes may not be sufficient to prevent such attacks, especially as LLMs are increasingly deployed in sensitive and critical applications. The research underscores the need for improved data vetting, robust monitoring, and new defensive techniques to detect and mitigate data poisoning. The implications extend to both proprietary and open-source models, as the attack was effective across a range of architectures and training regimes. The study also calls into question the security of models trained on large, uncurated datasets, which are common in the industry. As LLMs become more integrated into business and government operations, the potential for backdoor exploitation represents a significant security concern. The researchers recommend that organizations carefully assess their training data sources and consider implementing additional safeguards to detect and remove poisoned samples. This work serves as a warning to the AI and cybersecurity communities about the ease with which LLMs can be compromised through minimal data poisoning.
Jun 29, 2026