Grounding means an AI output is supported by the context, data, or sources available to the system. A grounded answer can be traced back to evidence. An ungrounded answer may be fluent and plausible, but nothing the system actually retrieved backs it up.
For an engineer, grounding is two things at once. It is a design constraint, because you decide what evidence reaches the model and whether the model is told to stay inside it. It is also an eval target, because once the answer exists you want a score that says whether its claims trace back to something. Most teams meet the term through retrieval augmented generation, but grounding applies anywhere a model should answer from a specific source: a tool result, a database row, a policy document, an earlier turn in the conversation.
Key takeaways
- Grounding is the property of tying generated claims to evidence the system actually has, plus the practices that make that possible: retrieval, citation, and attribution.
- Faithfulness is the narrower measurement. It asks whether the answer is supported by the provided context, and it says nothing about whether that context was right.
- An answer can be faithful but wrong, and correct but ungrounded. Those are different bugs with different fixes.
- Grounding fails upstream as often as it fails at generation. If retrieval returned nothing useful, no prompt instruction will make the answer grounded.
- Capturing retrieved context in your traces is what makes grounding debuggable. Without the context that was passed in, a low score tells you nothing about why.
Grounding, faithfulness, and correctness are three different questions
These three get used interchangeably and they should not be. Keep them separate:
- Faithfulness asks: is the answer supported by the context that was provided? It is a claim about the relationship between two texts, the context and the response. It is silent about the outside world.
- Correctness asks: is the answer actually true? That is judged against ground truth, a reference answer, or a checkable outcome such as a passing test.
- Grounding is the umbrella. It covers the property of being evidence-backed and the machinery that produces it, including how sources are selected and how they are cited back to the user. Faithfulness is how you measure one specific part of grounding.
The consequences are practical. A model handed a stale pricing document can produce an answer that is perfectly faithful to that document and still tells the customer the wrong price. Faithfulness scores high, correctness fails, and the fix is in the index, not the prompt. Run the reverse case too: a model that answers from parametric memory can be factually correct while citing nothing, which is ungrounded even though nobody was harmed this time. In a regulated workflow, that second case is still a defect, because you cannot show why the system said what it said.
How grounding works in practice
Evidence selection. Retrieval decides what the model is even able to be grounded in. Chunking strategy, embedding choice, filters, and rerankers all change the evidence set. If the right passage never enters the context window, the generation step is being asked to do something impossible.
Constrained generation. The prompt tells the model to answer only from the supplied context and to abstain when that context is insufficient. This helps and it is not a guarantee. Models still smooth over gaps, especially when the question sits next to something well represented in pretraining.
Attribution. Citation turns grounding from a claim into something a reader can check. Sentence-level citations beat a list of source documents at the bottom, because they let a user verify the one sentence they doubt.
Instrumentation. Grounding is only debuggable if the retrieved documents, their scores, and the final prompt are all captured on the span alongside the output. Recording those attributes is standard practice when you are tracing and evaluating an agent, and it is the difference between “groundedness dropped” and “groundedness dropped because the reranker started returning the changelog instead of the docs.”
Grounding in agents
Agents widen the definition. The evidence is no longer just retrieved passages: it is tool outputs, API responses, file contents, and the results of earlier steps. A grounded agent step is one whose output follows from what the tools actually returned, rather than from what the model assumed they would return. A common failure is an agent that calls a search tool, gets an empty result, and proceeds as if the call succeeded. That only shows up if you score the individual steps, and the step boundaries that an agent harness defines are what give you something to score against.
How teams evaluate groundedness
The usual method is an LLM judge that receives the question, the retrieved context, and the answer, and returns a label per claim or per response. It works reasonably well because the task is comparison rather than recall: the judge checks text against text instead of consulting its own knowledge. Two things make it fragile. Long contexts degrade judge attention, so scores drift as the window fills. And a judge asked for one score on a multi-claim answer will average away a single fabricated sentence.
Decomposing the answer into individual claims and scoring each one is slower and more reliable. Public benchmark work such as the LibreEval hallucination dataset exists because detector quality varies a lot by domain, and a detector that looks accurate on one corpus can be much weaker on yours. Validate the judge against a few hundred human-labeled examples from your own traffic before you trust its numbers.
FAQ
What is the difference between grounding and groundedness?
They point at the same idea from different angles. Grounding is the practice and the design goal. Groundedness is the measured property of a specific output, usually expressed as a score or a label. Most eval tooling uses “groundedness” for the metric name.
Is a grounded answer always correct?
No. Grounding says the answer follows from the evidence the system had. If that evidence is outdated, incomplete, or simply wrong, a well-grounded answer inherits the error. Pair a groundedness eval with a correctness eval against ground truth when the cost of a wrong answer is high, and treat document freshness as part of the eval surface.
How is grounding different from hallucination detection?
They are close to inverses. Hallucination detection looks for claims that contradict or exceed the evidence. Grounding evaluation looks for claims that are supported by it. In practice the same judge prompt often serves both, with the label polarity flipped.
How do you improve grounding when scores are low?
Check retrieval first. Measure whether the correct passage was in the context at all, because if it was not, the generation step could not have succeeded. If retrieval is fine and the answer still drifts, require abstention when the context does not cover the question, and require citations so unsupported sentences are visible during review.
Does grounding apply outside RAG?
Yes. Any output that is supposed to derive from a specific input can be evaluated for grounding: summarization against the source document, structured extraction against the record, an agent’s summary against the tool output it received.
