Kort gezegd: Prompt injection is an attack that hides instructions in text your AI reads. The AI then follows them instead of your rules. It is number one on the OWASP list of risks for AI language apps. You can’t fully prevent it, so you limit what the AI can do and test it often with real attack prompts.
Your chatbot has clear rules. Be polite, stay on topic, never share customer data. Then someone types one clever sentence, and the bot forgets all of them. That is prompt injection. If your AI reads emails, web pages or documents, the attack doesn’t even need a user to type it. And unless you test for it, you won’t know your AI is open to it.
What is prompt injection?
Prompt injection is when text that reaches an AI model changes what the model does in a way its owner did not intend. A large language model, the kind of AI behind most chatbots, reads your instructions and the user’s input as one stream of text. It has no solid wall between “rules from the owner” and “words from a stranger”. So a stranger can write words that look like rules. The OWASP Top 10 for LLM Applications ranks prompt injection as the number one risk for these apps. OWASP is a well-known non-profit for software security. The results can be small, such as a bot that goes off topic. They can also be serious: leaked data, a revealed system prompt, wrong advice, or an action the AI should never take. Jailbreaking, where people talk a model out of its safety rules, is a form of the same attack.
What is the difference between direct and indirect prompt injection?
Direct prompt injection comes from the user. They type the attack straight into the chat. For example, they ask the bot to ignore its earlier instructions or to play a role without rules. Indirect prompt injection is hidden in content the AI reads on its own. Think of a web page the AI summarises, an email it sorts, or a CV it screens. The text can even be invisible to people, such as white text on a white page. The AI still reads it and may follow it.
Indirect attacks are the bigger worry for most companies. More AI tools now read outside content and can act, such as sending an email or looking up a record. Then one poisoned document can do real damage, and no one has to talk to your bot at all. The NIST report on attacks against AI systems describes both kinds.
Can you prevent prompt injection completely?
No. OWASP itself says it is unclear whether a fool-proof defence exists, because of how language models work. Filters that block phrases like “ignore your instructions” help a little. But attackers rephrase, switch languages or hide the text in a file. So the goal is to reduce the damage and to know where you stand. These steps, drawn from the OWASP guidance, help most:
- Give the AI as little power as possible. Only the data and tools it really needs.
- Keep a person in the loop for actions that matter, such as payments or sending data out.
- Mark outside content clearly as untrusted, so it is kept apart from your own instructions.
- Check the output before it reaches a user or another system.
- Test with real attacks, again and again.
There is a trade-off. Each limit makes the AI a bit less flexible. A bot that can’t look anything up is safe but not very useful. Decide per tool how much risk you accept, and test that the limits really hold.
How do you test your AI for prompt injection?
You test for prompt injection by attacking your own AI on purpose, with a fixed set of known tricks, and checking what comes out. Start with direct attacks: requests to ignore the rules, role-play tricks, requests to reveal the system prompt, and the same tricks in other languages. Then test indirect attacks. Put hidden instructions in a test document, email or web page and let the AI process it. Check every answer. Did the AI leak data, break its rules, reveal its instructions or try an action it should not take? Keep the exact prompt and the exact response for every test. Repeat the full set on a schedule and after every change to the prompt, the model or the connected tools. A defence that worked last month can fail after a vendor update.
Write tests for your own setup too. A bot that can read customer records needs tests that try to pull out records. A bot that sends emails needs tests that try to make it send one.
What this means for your team
If your AI talks to customers or reads outside content, assume someone will try this. You’re responsible for your AI, even when you didn’t build it, and customers blame your organisation, not the model. Map what your AI can read and what it can do. Cut what it doesn’t need. Then test it the way an attacker would, and keep doing that.
Doing this by hand takes time, and the list of tricks keeps growing. VeritasAI runs more than 90 automated risk probes that cover hallucinations, jailbreaks and data leakage. They span over 13 risk categories, prompt injection among them. You can review the exact responses your model gave, so you see where it holds and where it gives in. The results come with specific advice for your AI vendor or your own data science team.
Find out if your AI follows your rules or an attacker’s.
Frequently asked questions
What is an example of prompt injection?
A user tells a support bot to forget its earlier rules and print its hidden instructions, and the bot does. Another example is a CV with hidden white text that tells an AI screener to rank the candidate first.
Is prompt injection the same as jailbreaking?
They overlap. Jailbreaking means getting a model to drop its safety rules, which OWASP treats as a form of prompt injection. Prompt injection is the wider term and includes attacks hidden in content the AI reads.
Can a system prompt stop prompt injection?
Not on its own. A strong system prompt helps, but the model reads it as text, just like the attack. Limits on data and tools, output checks and regular tests matter more.
How often should I test for prompt injection?
Test before launch, after every change to the prompt, model or tools, and on a fixed schedule in between. Weekly is a sensible default for customer-facing AI.