In short: An AI hallucination is an answer that sounds confident but is false or made up. Think of a policy that doesn’t exist, or a source nobody wrote. Language models hallucinate because they predict likely words, not checked facts. You can’t remove it completely. You can measure how often it happens and reduce it with grounding, clear limits and regular tests.
You use a chatbot so customers get answers at night and at the weekend, without waiting for your team. Most of the time that works well. But now and then the model fills a gap with something that sounds right and isn’t. That one answer is the one people remember, and customers blame your organisation, not the model.
What is an AI hallucination?
An AI hallucination is an answer from an AI model that’s presented as fact but isn’t true. It can be a wrong date, a made-up number, a quote nobody said or a source that doesn’t exist. It can also be a policy or product feature the model invents because it seems likely. The US standards body NIST calls this confabulation in its profile for generative AI risks. The tricky part is the tone. A hallucination sounds just as sure as a correct answer, so users can’t tell the difference. The risk is real for businesses. Take Air Canada. Its website chatbot told a customer about a refund that the airline’s own policy didn’t offer. The airline argued the chatbot was responsible for its own words. In 2024 a Canadian tribunal didn’t accept that and held the company liable.
Why do AI models hallucinate?
AI models hallucinate because they’re built to produce a likely answer, not a checked one. A language model writes by predicting the next word, based on patterns in its training data. It has no built-in step that checks a fact before it says it. When it doesn’t know, a likely-sounding guess often comes out. In 2025, researchers at OpenAI argued that the way models are trained and tested makes this worse. A guess gets rewarded more than an honest “I don’t know”. That’s what their paper Why Language Models Hallucinate found. Your own setup adds risk too. The model may be missing a document it needs, or your documents may be out of date. A vague question invites a vague guess. And a vendor update can change how often the model makes things up, without any change on your side.
How do you test an AI for hallucinations?
You test for hallucinations by asking questions where you already know the right answer, and checking what comes back. Use your own sources for this: your prices, policies, product details and terms. Then add questions designed to tempt the model into guessing. A good test set has a few kinds of questions:
- Known facts from your own documents, to check plain accuracy.
- Gaps: questions your documents don’t answer. A good answer says it doesn’t know.
- False premises: questions built on something untrue, such as a discount you never offered.
- Source requests: ask for a link or a reference, then check that it exists.
Ask each question more than once, because a model can answer differently each time. Score every answer as right, wrong or a fair refusal. Track the scores over weeks, not just once. And keep the exact responses, so you can show what the model said and fix the cause. VeritasAI’s tests include automated risk probes for hallucinations, next to jailbreaks and data leakage. You can review the exact responses your model gave during the tests.
How can you reduce AI hallucinations?
You reduce hallucinations by giving the model better facts, clearer limits and a safe way to say “I don’t know”. None of these removes the problem, so you still need tests to see whether they work. These measures help most:
- Let the model look up answers in your own documents first, and keep those documents current. This is often called retrieval, or RAG.
- Tell the model to say it doesn’t know when the answer isn’t in its sources.
- Keep the scope narrow. A bot for delivery questions shouldn’t give legal advice.
- Show the source with each answer, so users can check it.
- Let a person review answers about money, health, contracts or refunds.
There’s a trade-off. The stricter you make the model, the more often it says “I don’t know”. Too many refusals frustrate users as well. So measure both: how often it’s wrong and how often it refuses a fair question. The OWASP Top 10 for LLM applications lists misinformation as one of the main risks for this reason.
What this means for your team
Every language model will hallucinate sometimes. What matters is whether you know how often, on which topics, and whether that’s getting better or worse. Our advice: build a small test set from your own documents this week, with known facts, gaps and false premises. Run it, save the answers and repeat it after every change.
Even well-designed AI systems drift and misbehave over time, so a test at launch isn’t enough. VeritasAI runs those tests for you on a schedule, with more than 90 automated risk probes. It shows where your AI does well and where it makes things up. Every answer is kept, so you can see exactly what changed and share it with your vendor or your own team.
Find out how often your AI makes things up, before your customers do.
Frequently asked questions
Can AI hallucinations be stopped completely?
No. Because language models predict likely words, there’s always some chance of a wrong answer. You can make it much rarer and catch it sooner with good sources, clear limits and regular tests.
How often do chatbots hallucinate?
It depends a lot on the model, the task and your setup. Published figures vary widely, so they say little about your own chatbot. The only reliable number is the one you measure with your own questions.
Is a company liable for what its chatbot says?
It can be. In the Air Canada case, the tribunal held the airline responsible for its chatbot’s wrong answer. The rules differ by country, so ask a lawyer about your own situation.
What is the difference between a hallucination and bias?
A hallucination is a false statement presented as fact. Bias is a pattern of unfair treatment, such as different answers for different groups of people. A model can have either problem, or both, so test for each.
