Google Chrome Gemini AI Agent Enhanced to Counter Prompt Injection Attacks
Google has acknowledged the significant risk of prompt injection attacks targeting its Gemini-powered Chrome browsing agent, which can be manipulated to perform unauthorized actions such as initiating financial transactions or exfiltrating sensitive data. In response, Google has introduced a second AI model, termed the 'user alignment critic,' designed to independently vet the agent's proposed actions before execution. This model operates in isolation from untrusted web content, providing an additional layer of defense against both goal hijacking and data leakage. The move comes as prompt injection has been identified as a leading vulnerability in AI systems, with industry bodies like OWASP and the UK's National Cyber Security Centre highlighting its prevalence and difficulty to mitigate due to the structural limitations of large language models.
The Gemini-powered browsing agent, currently in preview, is capable of navigating websites, clicking buttons, and filling forms while users are logged into sensitive accounts, increasing the potential impact of successful attacks. Security experts and analysts have emphasized the need for robust safeguards, as malicious instructions can be hidden in web pages, iframes, or user-generated content. Google's dual-model approach aims to address these concerns by ensuring that any action not aligned with the user's intent is blocked, thereby reducing the risk of exploitation through prompt injection. The development reflects a broader industry trend of reassessing the security of AI-driven browsers and the need for advanced countermeasures to protect users and organizations from emerging threats.
Sources
Related Stories
Google Chrome Expands Gemini and On-Device AI Features, Including New Controls for Scam Detection Models
Google is testing deeper **Gemini** integration in Chrome via a new internal feature called **“Skills,”** which appears to let users define named, instruction-based automations that Gemini can execute inside the browser. The feature is surfaced through a new `chrome://skills` page and aligns with Google’s stated direction of turning Gemini into a more agent-like assistant capable of acting across tabs and, over time, integrating more tightly with Google services. Separately, Google has added user controls to manage the **on-device GenAI model** used by Chrome’s *Enhanced Protection* (Safe Browsing) capabilities, which were previously upgraded with AI for “real-time” detection of dangerous sites, downloads, and potentially malicious extensions. In Chrome Canary, users can disable *On-device GenAI* under **Chrome → Settings → System**, which also enables deletion of the local model; Google indicated the local model may support additional security and browser features beyond scam detection as it rolls out more broadly.
1 months agoGoogle Chrome Security Enhancements Against Account Takeover and Prompt Injection Threats
Google has introduced new layered security defenses in Chrome to address the growing risks of indirect prompt injection attacks and account takeovers, particularly as the browser integrates more agentic AI capabilities. Key features include the User Alignment Critic, which independently evaluates and vetoes potentially malicious actions by Chrome's AI agent, and Agent Origin Sets, which restrict the agent's data access to only relevant or user-approved sources. These measures are designed to prevent attackers from exploiting untrusted web content to hijack user sessions or exfiltrate sensitive data, and to mitigate site isolation bypasses that could compromise user privacy and security. In parallel, Google has acknowledged a surge in account takeover incidents targeting Chrome users, where attackers steal credentials, authentication codes, and session cookies to access synchronized data stored in the cloud. The company is urging users to strengthen their authentication methods and reconsider the use of browser-based password managers, as a single compromised account can expose a wide range of personal information. Google is also rolling out additional protections for Workspace accounts to counteract these threats and safeguard user data across its ecosystem.
3 months agoAdversaries Leverage Gemini AI for Self-Modifying Malware and Data Processing Agents
Google's Threat Intelligence Group (GTIG) has identified a significant evolution in cybercriminal and nation-state tactics, with adversaries now leveraging Gemini AI to develop advanced malware and data processing agents. Notably, groups such as APT42 have experimented with Gemini to create a 'Thinking Robot' malware module capable of rewriting its own code during execution to evade detection, as well as AI agents that process and analyze sensitive personal data for surveillance and intelligence gathering. These developments mark a shift from previous uses of AI for productivity, such as phishing and translation, to direct integration of AI into malware operations. The experimental PromptFlux malware dropper exemplifies this trend, utilizing Gemini to dynamically generate obfuscated VBScript variants and periodically update its code to bypass antivirus defenses. PromptFlux attempts persistence via Startup folder entries and spreads through removable drives and network shares, while its 'Thinking Robot' module queries Gemini for new evasion techniques. Although PromptFlux is still in early development and not yet capable of causing significant harm, Google has proactively disabled its access to the Gemini API. Other AI-powered malware, such as FruitShell, have also been observed, indicating a broader move toward AI-driven, self-modifying threats in the wild.
4 months ago