AI Vulnerability Exposes Users to Deceptive Messages Through Google’s Gemini
Researchers have issued a stark warning regarding a vulnerability in Google’s Gemini, a large language model, that can be exploited by attackers to deceive users via manipulated message summaries. This issue arises from the ability of malicious actors to embed harmful instructions within emails, thereby misguiding Gemini into presenting fraudulent summaries to recipients.
The vulnerability stems from a “prompt injection” flaw, identified by the security researcher known as “blurrylogic,” as detailed in a report coordinated by 0Din, a generative AI bug bounty platform launched by Mozilla in 2024. 0Din not only helps in identifying such threats but also compensates researchers for bringing vulnerabilities to light. According to the report, an attacker can send an email designed to inject malicious prompts into Gemini. When users request a summary of their unread emails, they may receive fabricated content that seems credible, all of which appears to originate from the Gemini system itself.
This exploitation can lead to social engineering attacks, coaxing users into taking immediate action such as calling fraudulent phone numbers or visiting harmful websites designed to harvest personal information, including credentials or financial data. Marco Figueroa, the technical product manager for 0Din’s bug bounty program, noted that prompt injections are akin to the previously utilized email macros that were a common vector for cyber attacks. He cautions that until robust context isolation measures are established in language models, any external text input could act as a form of executable code that may compromise security.
Despite prior mitigations released by Google following similar vulnerability reports in 2024, this technique remains effective. Google’s bug bounty program notified the company about this specific vulnerability on February 4, 2025. The platform stipulates that vendors have up to 120 days to address reported issues before the details are disclosed publicly.
In a response to the ongoing risks associated with prompt injections, Google has announced that it plans to implement advanced defenses against indirect prompt attacks, which involve hidden malicious instructions within data sources such as emails or calendar invites. These instructions may compel Gemini to either exfiltrate user data or execute unauthorized actions. Google continues to enhance its protective measures, including sanitizing the content of emails and calendar entries and employing machine learning models specifically designed to detect such threats in varied formats.
While Google asserts that it has yet to witness any real-world exploitation of this vulnerability, the inherent accessibility of exploiting it poses a serious concern. The vulnerability can be targeted easily using standard HTML and CSS tactics that do not require attachment or direct interaction from users. Figueroa elaborated that Gemini might prioritize hidden directives embedded in email content, executing attackers’ commands without the user’s awareness.
Moreover, a proof-of-concept exploit shared by 0Din illustrates how an attacker can cleverly manipulate visibility via CSS, rendering malicious prompts invisible to users while still instructing Gemini to relay distressing messages. For instance, an exploit could conceal a warning about a compromised password, leading an unsuspecting user to take harmful actions without realizing the manipulative nature of the communication.
The implications of this vulnerability extend to broader attack vectors, particularly in mass-email campaigns. Attackers could target compromised services like newsletters or customer relationship management systems, transforming single breaches into a multitude of phishing opportunities. To counter this, experts recommend that language models such as Gemini should be programmed to disregard invisibly styled content, as well as to train users to recognize that summaries generated by sophisticated AI should be seen as informational rather than authoritative security alerts.
As AI-driven tools are increasingly integrated into various applications, including Google Workspace products, vigilance is paramount. Google recently announced enhancements that would enable auto-generated email summaries under certain conditions, placing an onus on users and administrators to remain aware of the possible cybersecurity ramifications. This ongoing development reinforces the need for stringent safety measures against vulnerabilities that exploit AI technologies.
In conclusion, as AI systems like Google’s Gemini become more commonplace in business operations, understanding and defending against emerging vulnerabilities is crucial. Employing frameworks such as the MITRE ATT&CK Matrix can help delineate potential tactics used by adversaries, offering insight into initial access methods and persistence strategies that may be employed in such attacks.
As the cybersecurity landscape evolves, business owners must prioritize their defenses. A proactive stance in addressing these vulnerabilities will help mitigate potential threats associated with AI systems, ensuring that trust in technology remains intact.
