Privacy Concerns Over AI Training Data and Chatbot Adoption Risks
The rapid adoption of generative AI chatbots, such as ChatGPT, is transforming both consumer and enterprise environments, with significant growth in usage and market value. These chatbots are being used for a wide range of applications, from customer service to code generation and mental health support. However, their increasing prevalence raises concerns about risks such as hallucinations, dangerous suggestions, and the need for robust guardrails to ensure safe deployment and use.
Simultaneously, privacy concerns have emerged regarding how major technology companies, like Google, may use personal data to train AI models. Google recently denied allegations that it analyzes private Gmail content to train its Gemini AI model, following a class action lawsuit and public confusion over changes in Gmail's smart features settings. The company clarified that while smart features have existed for years, Gmail content is not used for AI model training, and any changes to terms or policies would be communicated transparently. These developments highlight the ongoing tension between AI innovation, user privacy, and the need for clear communication about data usage.
Get ahead of threats like this
Mallory correlates global threat intelligence with your attack surface — know if you’re exposed before adversaries strike.
How this story unfolded
2 events from the most recent confirmed update back to the earliest known activity.
CIO publishes broader analysis of chatbot and generative AI risks
A CIO article discusses the rise of chatbots and the risks and impact of generative AI, providing general context rather than introducing a distinct new incident beyond the earlier Google clarification.
Google publicly denies using Gmail emails to train AI
Google stated that it does not analyze users' email content for AI training, responding to public concern and clarifying how customer data is handled. The report frames this as the key development in the story.
Related entities
Vulnerabilities, threat actors, malware, products, organizations, and breaches Mallory has linked to this story.
Sources
2 references tracked. Mallory keeps watching after this page renders.
See the full picture, correlated to your attack surface.
Map indicators from this story to your assets and identify affected systems in minutes.
Every observed campaign, victim, and pivot linked to actors named in this story.
Malware, exploits, and IOCs connected to the activity described here.
YARA, Sigma, and Snort rules deployed to your SIEM as soon as they’re published.
Get matching new stories delivered to your team as they break — not the next morning.
Ask questions about this story and take action on the answers.
Related stories
Other stories Mallory is tracking that overlap on entities, sources, or topic labels.
Risks and Governance Challenges of Expanding AI Agent Access
The rapid evolution of generative AI systems, such as ChatGPT and Google Gemini, is ushering in a new era where AI agents and assistants are designed to perform tasks and make decisions on behalf of users. To function effectively, these AI agents require deep access to personal data and operating systems, raising significant concerns about privacy and cybersecurity. Experts warn that the trade-off for increased convenience is the exposure of sensitive information, as these agents often need extensive permissions to personalize services and interact with various applications. Simultaneously, global debates are intensifying over how AI should be governed, with China advancing an ambitious agenda to shape international AI rules. Beijing's approach emphasizes state control and anticipatory censorship, which could have far-reaching implications for freedom of expression and the global regulatory landscape. As AI agents become more integrated into daily life, the intersection of technical risks and governance models will play a critical role in determining the balance between innovation, security, and civil liberties worldwide.
Mar 21, 2026
AI Chatbot Security Risks: Prompt Injection Data Exfiltration and Privacy Trade-offs in New Consumer Tiers
Researchers disclosed an **indirect prompt injection** technique against **Google Gemini** that used a malicious **Google Calendar invite** to bypass guardrails and exfiltrate private meeting details. By embedding a hidden natural-language payload in an event description, an attacker could cause Gemini—when later asked an innocuous scheduling question—to summarize a user’s private meetings and write that summary into a newly created calendar event; in many enterprise configurations, that new event could be visible to the attacker, enabling data theft without additional user interaction. The issue was reported as remediated after responsible disclosure, underscoring how AI assistants integrated with enterprise SaaS can create new cross-application data-extraction paths. Separately, OpenAI product rollouts raised enterprise data-handling concerns tied to consumer usage. **ChatGPT Go** (a low-cost tier) was described as introducing an **ad-supported** model that could increase exposure of conversation data and usage patterns to advertising ecosystems, amplifying “shadow AI” risk when employees use personal accounts for work. **ChatGPT Health** was positioned as a dedicated health experience with added protections (e.g., encryption/isolation and claims that user data is not used to train foundation models), but reporting highlighted unresolved questions around safety, privacy, and how sensitive health information is protected in practice—areas that may require additional governance and controls if employees adopt these tools outside approved enterprise channels.
Mar 21, 2026
AI Assistants Expand Personalization and Data Access, Raising Privacy and Integrity Risks
Google is rolling out *AI Mode* personalization that can **connect Google Search to Gmail and Google Photos** for opt-in users, aiming to deliver more tailored results based on personal context. The feature is positioned as “secure” and is initially available via Labs for Google AI Pro and AI Ultra subscribers (with limited account eligibility), with Google stating the system processes data for specific prompts and does not directly train on a user’s inbox or photo library; the change nonetheless increases the amount of sensitive personal data that can be accessed during AI-assisted search workflows. OpenAI is testing an upgrade to **ChatGPT Temporary Chat** that keeps the session from being saved to history or used for model improvement, while still allowing **personalization signals** (e.g., memory/style preferences) to apply—alongside a stated retention window where OpenAI may keep a copy for up to **30 days** for safety. Separately, researchers and commentators warned about an “**Ouroboros effect**” where ChatGPT may cite AI-generated repositories such as xAI’s Grokipedia, increasing the risk of **misinformation loops** and “content traps” if AI systems do not rigorously vet sources, potentially degrading trust and decision-making even without direct training on the cited content.
Mar 21, 2026