The Open Worldwide Application Security Project (OWASP) made significant strides last year by releasing various iterations of the “OWASP Top 10 For Large Language Models,” culminating in a version 1.1 in October. These documents underscore the rapidly evolving landscape of Large Language Models (LLMs) and the corresponding tactics for exploiting and defending against them. In this article, we will examine four key vulnerabilities identified in the OWASP top 10 that can lead to the unintentional exposure of sensitive information, including passwords and API keys.
There is already substantial evidence that LLMs can inadvertently expose sensitive data. A report from GitGuardian earlier this year revealed more than 10 million secrets discovered within public GitHub repositories. Further complicating matters, researchers from the University of Hong Kong demonstrated that they could use a set of 900 specially crafted prompts to extract over 2,700 valid secrets from GitHub’s Copilot AI coding assistant, which was trained on public code repositories.
One of the primary techniques employed by researchers is termed “prompt injection,” highlighted as the number one vulnerability in the OWASP Top 10 for LLMs. This method exploits the model through carefully designed inputs, leading to unintended outputs. Direct prompt injections manipulate system commands, while indirect injections can alter data stemming from external sources. This type of vulnerability is reminiscent of a past incident where a bug allowed ChatGPT to reveal training data through repeated prompts.
Tip 1: Regularly Rotate Your Secrets
It’s essential for organizations to take preventive measures, even if they believe they have not unintentionally exposed secrets. GitHub commits often include layers of history, where secrets might get overwritten in more recent commits. The GitGuardian tool, “Has My Secret Leaked,” can assist in checking for potential leaks by allowing organizations to hash and encrypt current secrets to determine if they have been compromised in previous commits.
However, regular rotation of secrets requires careful planning. Organizations should be aware of where secrets are used, the potential impact of changing them, and establish measures to address any resulting issues during the transition. Once outdated secrets are rotated out, it is imperative to ensure that they are permanently disabled, effectively rendering them useless to any unauthorized parties.
Tip 2: Sanitize Your Data
Point six of the OWASP Top 10 addresses “Sensitive Information Disclosure.” This highlights the risk that LLMs may inadvertently leak confidential data, thus risking unauthorized access or privacy breaches. While malicious prompts can lead to data exposure, accidental disclosures can occur as well. The most effective strategy is to ensure the LLM is trained without access to sensitive data in the first place.
When preparing LLMs for user interaction—especially with those who may not have the organization’s best interests at heart—it is crucial to ensure that access to sensitive information is heavily restricted. Utilizing open-source tools or paid services for data cleaning can help eliminate any secrets from the training datasets before incorporating them into the model. If the LLM is unaware of secrets, it cannot disclose them.
Tip 3: Implement Regular Patches and Limit Permissions
Discussions surrounding the use of .env files and environment variables for securely managing secrets have gained prominence. However, one must consider the risk that an LLM could be prompted to reveal these environment variables or execute more harmful actions. This situation ties directly into key issues in the OWASP list, such as “Insecure Output Handling” and “Excessive Agency.”
Insecure Output Handling refers to a situation where LLM-generated outputs are accepted without scrutiny, potentially exposing backend systems to vulnerabilities such as cross-site scripting (XSS) or remote code execution. Excessive Agency allows an LLM too much freedom and authority, possibly leading to unintended actions.
These vulnerabilities can exacerbate one another. If an LLM is tricked into executing harmful commands and simultaneously possesses excessive permissions, the risks of severe security incidents rise significantly. Developers must remember that, despite their capabilities, LLMs are not inherently intelligent and should be treated with caution akin to managing sensitive data.
Establishing robust security protocols is essential. As a best practice, organizations should apply the principle of least privilege to limit access permissions for systems using LLMs, minimizing the risk that unauthorized personnel could exploit these systems for malicious activities. The prevailing maturity of AI technologies necessitates a cautious approach, akin to childproofing a home to prevent accidents. Proper safeguards are paramount for preventing LLMs from revealing sensitive information.
Conclusion
Large language models represent a transformative innovation poised to reshape a myriad of industries and professional practices. However, they remain immature technologies, and many organizations are adopting them without adequate caution, driven by fears of falling behind. Similar to how one would safeguard a curious child, vigilance and control of access to sensitive information are critical when deploying LLMs.
