Prompt Injection Vulnerabilities in Microsoft Copilot Studio AI Agents
Security researchers demonstrated that Microsoft Copilot Studio's no-code AI agent platform is susceptible to prompt injection attacks, allowing unauthorized access to sensitive business data. By leveraging the platform's ease of use, even non-technical employees can create AI agents that integrate with critical business systems such as SharePoint, Outlook, and Teams. In controlled tests, researchers were able to extract customer credit card information and manipulate booking systems to create fraudulent transactions, such as booking a $0 vacation, by issuing carefully crafted prompts to the AI agents.
The core risk arises from the democratization of AI agent creation, which, while boosting productivity, also increases the attack surface for organizations. The lack of technical safeguards and the inherent vulnerabilities of large language models (LLMs) make it easy for attackers or even well-meaning users to bypass intended security controls. Experts warn that these agentic tools, if not properly secured, can lead to significant data exposure and workflow hijacking, underscoring the urgent need for robust security practices and oversight when deploying AI-powered automation in business environments.
Sources
Related Stories
Security Risks in AI Coding Assistants: Prompt Injection and Dependency Hijacking
Security researchers have identified significant risks in AI-powered coding assistants, including Microsoft's Copilot and Claude Code, stemming from both prompt injection vulnerabilities and the potential for dependency hijacking via third-party plugins. In the case of Copilot, a security engineer disclosed several issues such as prompt injection leading to system prompt leaks, file upload policy bypasses using base64 encoding, and command execution within Copilot's isolated environment. Microsoft, however, has dismissed these findings as limitations of AI rather than true security vulnerabilities, sparking debate within the security community about the definition and handling of such risks. Separately, analysis of Claude Code highlights the dangers of plugin marketplaces, where third-party 'skills' can be enabled to automate tasks like dependency management. A technical review demonstrated how a seemingly benign plugin could redirect dependency installations to attacker-controlled sources, resulting in the silent introduction of trojanized libraries into development environments. These risks are compounded by the persistent nature of enabled plugins, which can continue to influence agent behavior and potentially compromise projects over time, underscoring the need for greater scrutiny and security controls in AI development tools.
2 months agoPrompt Injection Attacks and Security Challenges in AI Systems
Prompt injection has emerged as a critical security concern in the deployment of large language models (LLMs) and AI agents, with attackers exploiting the way these systems interpret and execute instructions. Security researchers have drawn parallels between prompt injection and earlier vulnerabilities like SQL injection, highlighting its potential to undermine the intended behavior of AI models. Prompt injection involves manipulating the input prompts to override or bypass the system-level instructions set by developers, leading to unauthorized actions or data leakage. The attack surface is broad, as LLMs are increasingly integrated into applications and workflows, making them attractive targets for adversaries. Multiple organizations, including OpenAI, Microsoft, and Anthropic, have initiated efforts to address prompt injection, but the problem remains unsolved due to the complexity and adaptability of AI models. Real-world demonstrations have shown that prompt injection can be used to break out of agentic applications, bypass browser security rules, and even persistently compromise AI systems through mechanisms like memory manipulation. Security conferences such as BlackHat USA 2024 have featured research on exploiting AI-powered tools like Microsoft 365 Copilot, where attackers can escalate privileges or exfiltrate data by crafting malicious prompts or leveraging markdown image vectors. Researchers have also identified that AI agents can be tricked into ignoring browser security policies, such as CORS, leading to potential cross-origin data leaks. Defensive measures, such as intentionally limiting AI capabilities or implementing stricter input filtering, have been adopted by some vendors, but these often come at the cost of reduced functionality. The security community is actively developing standards, such as the OWASP Agent Observability Standard, to improve monitoring and detection of prompt injection attempts. Despite these efforts, adversaries continue to find novel ways to exploit prompt injection, including dynamic manipulation of tool descriptions and bypassing image filtering mechanisms. The rapid evolution of AI technologies and the proliferation of agentic applications have made it challenging to keep pace with emerging threats. Security researchers emphasize the need for ongoing vigilance, robust testing, and collaboration across the industry to mitigate the risks associated with prompt injection. The use of AI in sensitive environments, such as enterprise productivity suites and web browsers, amplifies the potential impact of successful attacks. As AI adoption accelerates, organizations must prioritize understanding and defending against prompt injection to safeguard their systems and data. The ongoing research and public disclosures serve as a call to action for both developers and defenders to address this evolving threat landscape.
5 months agoNovel Attacks Exploit Microsoft Copilot and Copilot Studio for Data Theft and OAuth Token Compromise
Security researchers have identified two distinct attack techniques targeting Microsoft's AI-powered platforms. The first, dubbed **CoPhish**, leverages Microsoft Copilot Studio agents to deliver fraudulent OAuth consent requests through legitimate Microsoft domains, enabling attackers to steal OAuth tokens. By customizing Copilot Studio chatbots and exploiting the platform's "demo website" feature, attackers can trick users into authenticating with malicious applications, potentially granting unauthorized access to sensitive resources. Microsoft has acknowledged the issue and is working on product updates to mitigate the risk, emphasizing the need for organizations to strengthen governance and consent processes. Separately, a vulnerability in Microsoft 365 Copilot was discovered that allowed attackers to use indirect prompt injection via Mermaid diagrams to exfiltrate sensitive tenant data, such as emails. By embedding malicious instructions in seemingly benign prompts, attackers could manipulate Copilot to retrieve and encode confidential information. Although Microsoft has since patched this flaw, the incident highlights the emerging risks associated with integrating AI assistants and third-party tools, as well as the challenges in securing complex, automated workflows within enterprise environments.
4 months ago