An AI hallucination is a generated statement that is presented as fact but is false, fabricated, or unsupported by the sources the system was given. In AI engineering, hallucinations matter because users act on confident answers, and the right fix depends on which kind of unsupported claim appeared.
For example, suppose a support agent answers, “You have 45 days, per the returns page.” The retrieved returns page says 30 days, and 30 days is the correct, current policy. The answer contradicts its own source and is also wrong about the world, so it is both a faithfulness failure and a factual error.
Hallucination describes the output, not a proven internal cause. That is why detection starts by classifying what kind of unsupported claim appeared.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Why AI hallucination matters for AI evaluation
A hallucination rate on its own tells you that something went wrong. It does not tell you where. Retrieval, citation assembly, and generation can each produce an unsupported claim, and each needs a different fix.
Classifying hallucinations helps with:
- Choosing an evaluator: Match each failure type to the check that can actually detect it.
- Routing fixes: Send retrieval misses, citation bugs, and generation errors to the right part of the system.
- Reading trends correctly: Avoid a combined rate that makes a retrieval fix look like a model regression.
- Building test sets: Turn labeled production failures into regression examples.
No single hallucination score covers open-ended chat, retrieval-augmented generation (RAG), and citation tasks at once.
How AI hallucination happens
A large language model (LLM) generates plausible continuations of text. Plausible wording is not a guarantee of truth.
When the prompt, retrieved passages, or tool results do not contain the answer, the model may fill the gap. It might invent a name, a date, a quotation, or a reference.
The causes differ from case to case:
- Missing or wrong source: The evidence the model needed was never there, or it was out of date.
- Irrelevant retrieval: The retriever returned a passage that does not address the question.
- Ignored evidence: The correct passage was retrieved, but the model answered from something else.
To find the cause, label the evidence relationship. Compare each claim against the user request, the retrieved or tool-provided evidence in the trace, and, where needed, an independent trusted source.
For more on tying generation to evidence, see grounding.
Types of hallucination
Most flagged outputs fit one of four types. Label the type before you choose an evaluator.
- Factual error: The statement does not match reality, such as a wrong date or amount.
- Fabrication: The model invents something that does not exist, such as a person, policy, or quotation.
- Unsupported citation: A reference does not back up the claim attached to it.
- Source-faithfulness failure: The claim contradicts or goes beyond the context the system provided.
These types overlap. The returns example above is both a factual error and a faithfulness failure.
Factuality vs. faithfulness
Factuality asks whether a statement matches reality. Faithfulness asks whether a statement is supported by the provided context, regardless of whether that context is itself correct.
The two questions produce four combinations in practice:
| Faithful to the context | Not faithful to the context | |
|---|---|---|
| Factually correct | The goal. The answer is right and supported. | The model gives the right refund window from memory while the retrieved page says something else. |
| Factually wrong | The model accurately repeats a stale policy page. | The model invents a policy that is also false. |
The “correct but not faithful” case is still a defect in a grounded product. The answer happens to be right, but no one can audit it against its source.
The “faithful but wrong” case shows that a claim can match its retrieved passage and still be false. The problem there is the source, not the generation.
For a deeper treatment of the second question, see faithfulness.
Unsupported citations
Unsupported citations are a separate subtype, and they fail in three different ways:
- Wrong document: The citation points to a real document that does not contain the claim.
- Missing document: The citation points to a document that does not exist.
- Invented quote: The answer quotes text that never appeared in the source.
Suppose the agent from the opening example had instead cited “Returns Policy v9,” and no such document was retrieved. The label is unsupported citation, not faithfulness failure.
Lumping these together hides whether retrieval, citation assembly, or generation failed. For checks that focus on references, see citation accuracy.
How to detect and measure AI hallucination
Start from production traces rather than invented examples. Here is a workflow that keeps failure types separate:
- Sample answers from production traces.
- Break each answer into individual claims.
- Compare each claim with the request, the evidence in the trace, and a trusted source where needed.
- Label each unsupported claim by type.
- Report each type as its own rate.
The LLM-as-a-judge guide describes how to turn those labels into a repeatable evaluator. Calibrate the evaluator against human labels before you rely on it.
For more on calibration, see how to build LLM-as-a-judge evaluators that hold up in production.
Match the check to the claim type
Each type of hallucination needs a different kind of check:
- Faithfulness failures: Use an evaluator that checks whether each claim is supported by the provided context.
- Unsupported citations: Verify that each cited document exists, was retrieved, and contains the claim.
- Factual errors: Compare against a trusted external reference, because the provided context may itself be wrong.
Worked example: two failures, two fixes
Consider the two versions of the returns answer side by side.
| Agent answer | What the trace shows | Label | Where to fix |
|---|---|---|---|
| “You have 45 days, per the returns page.” | The retrieved page says 30 days, which is correct. | Faithfulness failure and factual error | Generation ignored the evidence |
| “You have 30 days, per Returns Policy v9.” | No document called “Returns Policy v9” was retrieved. | Unsupported citation | Citation assembly or retrieval coverage |
The first answer had the right evidence and did not use it. The second answer may state the right number, but it points to a source that was never there.
If you counted both as “hallucinations” in one rate, you would not know which component to change. Separate labels tell you.
For broader evaluation methods, see LLM evaluation and RAG evaluation.
Limitations of hallucination detection
- Imperfect claim splitting: Breaking long or hedged sentences into separate claims does not always work cleanly.
- Missed subtle errors: Judges can miss small numeric swaps, such as one date replaced with a nearby one.
- Need for human review: High-impact domains still need people to review answers and to build the labeled set a judge is checked against.
- Limited scope of a low rate: A low rate on sampled traffic describes the prompts, corpus, and model version you sampled. It does not prove the next answer is safe.
AI hallucination vs. other wrong answers
Not every wrong answer is a hallucination. A wrong calculation, a misread user request, or a correct summary of a wrong source can all be errors without fabrication.
Reserve the hallucination label for false or unsupported content presented as fact. Track the other failures separately, for example under correctness.
Frequently asked questions
Is every wrong answer a hallucination?
No. A wrong calculation, a misread request, or an accurate summary of a wrong source is an error, but not necessarily a fabrication. Label those failures separately so your hallucination rate measures what it claims to.
Does RAG prevent hallucination?
No. Retrieval gives the model evidence, but the model can still ignore it, misquote it, or repeat it when the evidence itself is stale. RAG changes which checks you need. It does not remove the need for faithfulness and citation checks.
Can one metric detect all hallucinations?
No. Faithfulness evaluators check support in the provided context. Citation checks verify references. Factuality checks need a trusted external reference. Match the evaluator to the claim type you labeled.
Can an answer be faithful to its source and still be wrong?
Yes. If the retrieved passage is stale or incorrect, a faithful answer repeats that error. Faithfulness checks will pass it, so you also need a factuality check against a trusted source.
Where should I start detecting hallucinations?
Start with a sample of production traces. Break answers into claims, label each unsupported claim by type, and report the types separately. Then build an evaluator for the most common type and calibrate it against your labels.
Putting hallucination detection to work
Record the failure type with every flagged output, not just a yes or no. Keep the retrieved evidence and any citations in the same record so a reviewer can see what the model had to work with.
Report faithfulness, citation, and factual errors as separate rates. Add labeled failures to your regression set so the same mistake is caught before the next release.
Tracing tools such as Arize AX let you inspect the request, the retrieved context, and the answer together, which is the comparison every hallucination label depends on.
