Many of us have a comfortable, fraternal relationship with our AI tools. They help us write emails; they invent funny images for us; they can even tell us jokes. But they’re also incredibly powerful digital machines that process huge amounts of data for us and others. What happens if something goes wrong?
Thankfully, there are experts tackling that very question. Each year, the Open Web Application Security Project (OWASP) releases a list of the top 10 vulnerabilities in Large Language Models (LLM), a type of AI algorithm. Their goal is to help developers understand what weaknesses can arise in these systems, and how to prevent them. And while the LLM Top 10 may seem too technical to be relevant to any lay person, it’s actually an incredibly useful resource for understanding how AI works, how it’s evolving, and how we can interact securely with it.
Input, Output, and System Vulnerabilities
At a high level, the Top 10 can be broken down into 3 basic categories: input vulnerabilities, output vulnerabilities, and system vulnerabilities. While each of these categories encompasses several specific issues, here is just one example of each.
Prompt Injection (#1 of the Top 10): This could be categorized as an input vulnerability, related to how AI tools process prompts. If a prompt isn’t processed correctly, it can alter responses (output), manipulate the tool’s behavior, and even bypass safety protocols. This vulnerability can be triggered either unintentionally (by a prompt that just happens to elicit a strange response) or intentionally (by a criminal intentionally attempting to exploit the system).
Misinformation (#9 of the Top 10): Also referred to as hallucination, this is an output vulnerability that can affect the reliability of data passed back to the user. Because AI tools work off of training data and statistical patterns to generate responses, but don’t actually possess any reasoning capabilities, the answers they return in response to prompts can be incorrect. If users don’t verify responses adequately, this can lead to the spread of misinformation.
Supply Chain (#3 of the Top 10): This is a broader system vulnerability related to external applications and platforms an AI tool might be connected with. Similar to how an organization’s security infrastructure can be threatened if a third-party or contractor undergoes a breach, an AI tool can be negatively impacted by security issues in its related parts. This is especially true if a component in the supply chain is outdated or improperly licensed.
Developers’ Responsibility
As you can probably surmise, much of the responsibility for preventing these threats lies with the developer, and each of the Top 10 comes with a slew of safety protocols to help them do just that. But what steps can the rest of us take to ensure we’re being “secure consumers” of AI?
First, be careful with the input you provide. Because input vulnerabilities feed off the information contained in the prompts we supply, we can tangibly help mitigate those issues by being cautious in this area. Never include any sensitive information in a prompt (or file attachment)—that means personal information about yourself (financial info, health data, etc.) as well as confidential or proprietary business information.
It’s also important to remember that AI tools embedded in another program—for instance, a virtual assistant that integrates with Outlook to provide summaries of emails—pose additional risks, because they can access everything contained within that program. That might include files, settings…even every email you have ever received or sent. If the connection isn’t completely secure, the AI tool could leak any of that personal and proprietary information.
Second, never take output at face value. Always read over every piece of output to make sure everything is correct. Hallucinations often seem plausible, making them difficult to spot; if you’re not 100% sure about something, look it up independently to verify. Even though this step takes additional time, it’s crucial for ensuring that AI is used appropriately, and that misinformation doesn’t spread.
Third, do your research on tools and vet their policies carefully before using them. Does the developer perform regular security checks, both on the tool itself and on any components and third parties related to the tool? Do they have a strong data sanitization policy? Will they use your information to help train the AI model? Issues with supply chain and other system vulnerabilities can be avoided by simply weeding out obviously insecure tools.
In a work context, you should only use tools that IT has explicitly approved. IT departments will often take extra steps with approved tools to set parameters around things like data sanitization, so even if another tool looks secure, it may not protect your data sufficiently for work purposes.
Finally, notify IT right away if an AI tool starts acting strangely. Has it started providing an excessive amount of incorrect information? Has it included sensitive information in a response, even though you didn’t include any sensitive information in your prompt? Has the output changed stylistically? Any of these discrepancies can indicate that a tool has been compromised or isn’t being properly updated.
Protection Begins With Awareness
If the Top 10 can teach us anything, it’s that protecting ourselves in the age of AI starts with awareness, from developers on down to everyday users. The more all of us understand the risks—whether that’s prompt injection, misinformation, or hidden system vulnerabilities—the better equipped we are to navigate them safely. AI is a tool, not an infallible source of truth, and by staying vigilant about how we use it, we can take an active role in safeguarding our data and harnessing the power of AI for good.
