Loading story…
Recent research and industry developments have highlighted significant vulnerabilities in the integrity and security of AI models and their supply chains. Researchers demonstrated that inserting as few as 250 carefully crafted, poisoned documents into an AI model's training data can corrupt its outputs, causing it to generate nonsensical or manipulated responses. This attack method, tested on models up to 13 billion parameters, shows that even a small amount of malicious data can have a disproportionate impact on model behavior, raising concerns about the robustness of AI training pipelines. In parallel, new attack techniques such as the Lies-in-the-Loop (LITL) attack have been identified, where adversaries plant seemingly benign code or dependencies that only reveal malicious behavior under specific runtime conditions. These attacks are particularly insidious because they can deceive AI-powered code assistants, especially those that learn from human feedback, into approving or perpetuating unsafe code. The LITL attack demonstrates that even with human oversight, AI systems can be manipulated into executing or recommending malicious actions, undermining the trust in human-in-the-loop security models. Additionally, the rapid growth of AI model repositories like Hugging Face has introduced new supply chain risks, as models themselves can harbor hidden malware, backdoors, or payloads that activate during inference or deserialization. Security firms are now developing specialized scanners to detect such threats in AI model artifacts, recognizing that traditional software security measures are insufficient for the unique risks posed by AI models. Real-world incidents have already shown that attackers can exploit model file formats to execute arbitrary code or exfiltrate data, making the AI supply chain a lucrative target. The convergence of these threats underscores the urgent need for robust security practices in AI development, including rigorous data curation, model artifact scanning, and enhanced monitoring of AI-assisted development workflows. As AI becomes more deeply integrated into production environments, the potential impact of compromised models or manipulated code suggestions grows, threatening both data integrity and operational security. The industry is responding with new tools and collaborative research, but the evolving threat landscape requires continuous vigilance and adaptation. Organizations deploying AI must recognize that both the data used for training and the models themselves are attack surfaces. Proactive measures, such as integrating supply chain scanning and monitoring AI-human interactions, are essential to mitigate these emerging risks. The findings from recent studies and product announcements serve as a wake-up call for the broader adoption of AI-specific security controls. As adversaries become more sophisticated in targeting AI systems, defenders must stay ahead by understanding and addressing these novel attack vectors. The future of AI security will depend on the industry's ability to anticipate, detect, and neutralize threats at every stage of the AI lifecycle.
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
3 events from the most recent confirmed update back to the earliest known activity.
Checkmarx published analysis of a new threat in which AI systems can produce misleading or false outputs that undermine human-in-the-loop security workflows, framing it as an emerging security risk.
Malwarebytes published research describing how AI systems can be poisoned using as few as 250 malicious or low-quality documents, highlighting a practical data-poisoning risk to model behavior and outputs.
Socket announced an experimental malware-scanning capability for the Hugging Face ecosystem, marking a new defensive measure aimed at identifying malicious content in AI-related repositories and packages.
3 references tracked. Mallory keeps watching after this page renders.
malwarebytes.com
Open sourcecheckmarx.com
Open sourcesocket.dev
Open sourceMap indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Recent research has highlighted significant security vulnerabilities in the development and deployment of AI models, particularly those with open weights and in fast-growing AI startups. Cisco's AI Threat Research found that open-weight language models are highly susceptible to adversarial manipulation, especially during extended multi-turn conversations. Attackers can gradually steer these models toward producing harmful or restricted outputs, often bypassing moderation systems designed for single-turn interactions. The study revealed that some models, such as Mistral’s Large-2 and Alibaba’s Qwen3-32B, had success rates for multi-turn attacks exceeding 85%, underscoring the challenge of maintaining safety context over time. Separately, a report from Wiz identified that nearly two-thirds of leading private AI companies have leaked sensitive credentials, such as API keys and access tokens, on GitHub. These leaks could expose private AI models, training data, and internal organizational details, with some credentials belonging to major platforms like Hugging Face and LangChain. The findings suggest that the rapid pace of AI innovation is outstripping secure development practices, leaving critical assets vulnerable to exploitation. Both studies emphasize the urgent need for improved security controls and DevSecOps hygiene in the AI sector to protect intellectual property and prevent adversarial abuse of AI systems.
Jul 12, 2026
AI is rapidly transforming the cybersecurity landscape, introducing both significant risks and powerful new defensive capabilities. The widespread adoption of AI tools in the workplace has led to a surge in employees using these technologies, with 65% of people now utilizing AI tools, up from 44% the previous year. However, this increased usage has not been matched by adequate security training, as 58% of employees have received no instruction on AI security or privacy risks. This gap has resulted in sensitive business information, including internal documents, financial data, and client details, being routinely entered into AI systems, raising the risk of data leakage and unauthorized access. Employees express substantial concern about AI's potential to amplify cybercrime, facilitate scams, bypass security systems, and enable identity impersonation, yet only 45% trust companies to implement AI securely. In parallel, AI is being leveraged by both attackers and defenders, with advanced models now capable of simulating and even outperforming human teams in vulnerability discovery and remediation. For example, AI models have been used to replicate major historical cyberattacks in simulation, demonstrating their potential for both offensive and defensive applications. In cybersecurity competitions, AI-driven systems have successfully identified and patched vulnerabilities, sometimes uncovering previously unknown flaws. Organizations like Anthropic have invested in enhancing their AI models to assist defenders, enabling the detection, analysis, and remediation of vulnerabilities in both code and deployed systems. These advancements have led to AI models matching or surpassing previous state-of-the-art systems in cyber defense tasks. At the same time, threat actors are exploiting AI to scale their operations, prompting security teams to develop new safeguards and monitoring techniques. The dual-use nature of AI in cybersecurity underscores the urgent need for robust security awareness training, updated policies, and technical controls to manage the risks associated with AI adoption. As AI continues to evolve, defenders must stay ahead by integrating AI-driven tools into their security operations while remaining vigilant against emerging threats. The current state of AI security is described as precarious, with urgent calls for organizations to address the human and technical factors contributing to risk. The future of cybersecurity will be defined by the ongoing arms race between AI-powered attackers and increasingly sophisticated AI-enabled defenders, making continuous adaptation and investment in AI security essential for organizational resilience.
Jun 29, 2026
Security researchers and industry experts are raising alarms about the growing use of artificial intelligence (AI) in both offensive and defensive cybersecurity operations. Attackers are leveraging AI to bypass advanced security controls, as demonstrated by a researcher who used AI to defeat an "AI-powered" web application firewall, and by the emergence of new malware that exploits AI model files and browser vulnerabilities to evade detection and exfiltrate credentials. Meanwhile, defenders are grappling with the proliferation of unsanctioned AI tools in the workplace, the challenge of auditing AI decision-making, and the surge in AI-powered bug hunting, which has led to a dramatic increase in vulnerability discoveries and bug bounty payouts. The risks are compounded by the lack of clear AI usage policies, the potential for data leaks through generative AI tools, and the difficulty in monitoring or controlling how sensitive information is processed and stored by these systems. Industry reports highlight that a significant portion of employees use unauthorized AI applications, often exposing sensitive data without IT oversight, and that prompt injection and model manipulation are now common vulnerability types. The security community is also debating the extent to which ransomware and other attacks are truly "AI-driven," with some reports criticized for overstating the role of AI in current threat activity. As organizations rush to adopt AI for efficiency and innovation, experts urge the implementation of robust governance, continuous monitoring, and red-teaming to anticipate and mitigate the evolving risks posed by both sanctioned and shadow AI systems. The rapid evolution of AI in cybersecurity is forcing a reevaluation of traditional defense models, emphasizing the need for transparency, operational oversight, and adaptive security strategies.
Jun 29, 2026