Loading story…
A new study by Anthropic, in collaboration with the UK AISI's Safeguards team and The Alan Turing Institute, has revealed that poisoning attacks against large language models (LLMs) are significantly easier than previously believed. Researchers demonstrated that injecting as few as 250 malicious documents into the pretraining data of LLMs, regardless of their size, can successfully backdoor models ranging from 600 million to 13 billion parameters. This finding challenges the prior assumption that larger models require proportionally more poisoned data, raising concerns about the scalability and accessibility of AI poisoning attacks. The study warns that users should exercise caution when trusting outputs from generative AI tools, as poisoned models can be manipulated to provide malicious links or embed backdoors in generated code.
Separately, Palo Alto Networks’ Unit 42 has demonstrated the risks of prompt injection attacks in multi-agent AI systems using the Agent2Agent (A2A) protocol. In their proof-of-concept, a malicious research assistant agent was able to manipulate a financial assistant agent into disclosing sensitive information and performing unauthorized actions, such as buying stock, through multi-stage prompt injections. The A2A protocol enables stateful communication between autonomous agents, which can be exploited by adversaries to inject harmful instructions over multiple interactions. These findings highlight the growing attack surface in AI-driven environments, emphasizing the need for robust safeguards and user vigilance when deploying and interacting with autonomous AI agents.
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
3 events from the most recent confirmed update back to the earliest known activity.
Researchers reported several ChatGPT flaws that could let attackers trick the AI into leaking data, hijack memory, and create covert data-exposure risks. Multiple outlets described the same underlying vulnerability findings as a single disclosure event.
Researchers showed that prompt injection can affect Agent2Agent interactions, illustrating how one AI agent could be manipulated to influence another in multi-agent workflows.
A report described AI poisoning attacks as easier to carry out than previously believed, indicating that manipulating training data may be a more practical threat than earlier assumptions suggested.
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
5 references tracked. Mallory keeps watching after this page renders.
scworld.com
Open sourcehackread.com
Open sourcethehackernews.com
Open sourceblog.knowbe4.com
Open sourcescworld.com
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Prompt injection has emerged as a critical security concern in the deployment of large language models (LLMs) and AI agents, with attackers exploiting the way these systems interpret and execute instructions. Security researchers have drawn parallels between prompt injection and earlier vulnerabilities like SQL injection, highlighting its potential to undermine the intended behavior of AI models. Prompt injection involves manipulating the input prompts to override or bypass the system-level instructions set by developers, leading to unauthorized actions or data leakage. The attack surface is broad, as LLMs are increasingly integrated into applications and workflows, making them attractive targets for adversaries. Multiple organizations, including OpenAI, Microsoft, and Anthropic, have initiated efforts to address prompt injection, but the problem remains unsolved due to the complexity and adaptability of AI models. Real-world demonstrations have shown that prompt injection can be used to break out of agentic applications, bypass browser security rules, and even persistently compromise AI systems through mechanisms like memory manipulation. Security conferences such as BlackHat USA 2024 have featured research on exploiting AI-powered tools like Microsoft 365 Copilot, where attackers can escalate privileges or exfiltrate data by crafting malicious prompts or leveraging markdown image vectors. Researchers have also identified that AI agents can be tricked into ignoring browser security policies, such as CORS, leading to potential cross-origin data leaks. Defensive measures, such as intentionally limiting AI capabilities or implementing stricter input filtering, have been adopted by some vendors, but these often come at the cost of reduced functionality. The security community is actively developing standards, such as the OWASP Agent Observability Standard, to improve monitoring and detection of prompt injection attempts. Despite these efforts, adversaries continue to find novel ways to exploit prompt injection, including dynamic manipulation of tool descriptions and bypassing image filtering mechanisms. The rapid evolution of AI technologies and the proliferation of agentic applications have made it challenging to keep pace with emerging threats. Security researchers emphasize the need for ongoing vigilance, robust testing, and collaboration across the industry to mitigate the risks associated with prompt injection. The use of AI in sensitive environments, such as enterprise productivity suites and web browsers, amplifies the potential impact of successful attacks. As AI adoption accelerates, organizations must prioritize understanding and defending against prompt injection to safeguard their systems and data. The ongoing research and public disclosures serve as a call to action for both developers and defenders to address this evolving threat landscape.
Jun 29, 2026
Cybersecurity professionals are rapidly adopting artificial intelligence (AI) tools to enhance threat detection, investigation, and response, with over 90% of surveyed teams now testing or planning to use AI in their operations. However, this widespread adoption brings new security challenges, as highlighted by recent research and industry reports. The Cloud Security Alliance and Google Cloud emphasize that traditional data security models require significant updates to address AI-specific risks such as prompt injection, model inversion, and multi-modal data leakage. Unlike conventional vulnerabilities, prompt injection exploits the inherent ambiguity of large language models (LLMs), making it a persistent risk that cannot be mitigated by simple patches. Security experts recommend combining AI-driven analysis with deterministic, auditable controls to ensure reliable and explainable security decisions, especially in enforcement actions like access revocation or incident response. A concrete example of these risks was demonstrated in Docker's 'Ask Gordon' AI assistant, where researchers exploited a metadata-based prompt injection flaw to exfiltrate sensitive information. Attackers could embed malicious instructions in the metadata of Docker Hub repositories, which the AI would then execute when prompted by users, highlighting the real-world impact of prompt injection vulnerabilities. The evolving threat landscape also includes the use of malicious LLMs and AI-powered tools in DDoS-for-hire operations, with underground actors leveraging AI to automate botnet recruitment and evade detection. These developments underscore the urgent need for organizations to update their security frameworks, implement ongoing risk management for AI systems, and remain vigilant against emerging AI-driven attack vectors.
Jun 29, 2026
The UK’s National Cyber Security Centre (NCSC) has issued a warning that large language models (LLMs) are inherently vulnerable to prompt injection attacks, a type of cyber threat that manipulates AI systems into disregarding their original instructions. Security experts at the NCSC emphasized that this vulnerability is fundamental to how LLMs process text, making it unlikely that prompt injection can ever be fully eliminated. Real-world examples have already demonstrated attackers using prompt injection to bypass restrictions in systems like Microsoft’s Bing and GitHub Copilot, and the risk is expected to grow as generative AI becomes more deeply embedded in digital infrastructure. The NCSC’s technical director for platforms research, David C, cautioned that prompt injection is often mistakenly compared to SQL injection, but the two require different mitigation strategies. Unlike traditional application vulnerabilities, LLMs do not enforce a security boundary between trusted and untrusted content, allowing malicious instructions to be processed alongside legitimate prompts. The agency’s warning highlights the need for organizations to recognize the persistent nature of this threat and to develop new approaches to securing AI-driven applications, as conventional defenses may prove inadequate.
Jun 29, 2026