LLM evaluation is the practice of measuring whether a large language model, or an application built on one, behaves as intended. It can score correctness, relevance, faithfulness, safety, tone, tool use, latency, cost, and task success.
What makes it its own discipline is a single structural problem: for most of the work language models do, there is no one correct output. Ask a model to summarize a support thread and there are hundreds of good summaries and thousands of bad ones, and no reference string that separates them. Ask it to draft a reply, explain a policy, or decide which tool to call, and the same thing happens. Traditional supervised evaluation assumes a ground truth label to compare against. Open-ended generation removes it, and every technique below is an answer to that absence.
Key takeaways
- Most LLM tasks have many acceptable outputs and no single reference answer, so exact-match and overlap scoring break down immediately.
- Reference-free scoring judges an output against a property such as groundedness in the retrieved context, rather than against a target string.
- An LLM judge is the default way to score subjective properties at scale, and it is itself a model with an error rate that has to be measured against human labels.
- Human review does not disappear. It moves from grading everything to calibrating and auditing the automated scorers.
- Evaluate the application, not the base model. Users experience prompts, retrieval, tools, and orchestration together, so the eval boundary should match that.
Why the missing reference answer changes everything
The older metrics assumed a target. Accuracy, F1, and exact match all compare a prediction to a label. Text-specific metrics such as BLEU and ROUGE loosened this into n-gram overlap with one or more reference texts, which was reasonable for translation and headline generation and is close to useless for a chatbot response. A summary that shares almost no vocabulary with the reference can be better than one that shares most of it. Overlap measures surface form, and the thing you care about is meaning.
So LLM evaluation moves the question. Instead of “does this match the answer,” it asks “does this output have the property I need.” That reframing is what makes the rest of the field make sense.
The four approaches you will actually use
Deterministic checks. Some properties are still mechanical, and those should never go to a model. Valid JSON, a required field present, a citation that resolves to a real document ID, no PII pattern in the output. These are cheap, instant, and perfectly reliable. Use them for everything they can cover.
Reference-free model scoring. Score the output against its own context rather than against a target. Groundedness asks whether every claim in the answer is supported by the retrieved documents. Relevance asks whether the answer addresses the question that was asked. Neither needs a golden answer, which is what makes them the only options that work on live production traffic.
LLM-as-a-judge. Give a model the input, the output, and a rubric, and ask it to return a label with an explanation. This is how subjective properties get scored at volume. It works well for coarse distinctions such as helpful versus unhelpful, and poorly for fine-grained numeric scores, where a judge asked for a 1 to 10 rating will cluster everything between 6 and 8.
Human review. Still the ground truth everything else is calibrated against. The practical pattern is a small, continuously refreshed set of human-labeled examples used to check whether your judge agrees with your team, plus a review queue for the cases automated scorers flag as uncertain.
Judges have their own failure modes
An LLM judge is a language model doing a classification task, and it inherits every weakness of one. Judges show position bias, favoring whichever response appeared first in a pairwise comparison. They show verbosity bias, rating longer answers higher for no reason connected to quality. They tend to reward confident phrasing over hedged phrasing, even when the hedge is the honest response.
The one people miss is self-preference. When the model generating the output is the same model grading it, the score is not independent, and testing for self-evaluation bias should be part of setting up any judge rather than an afterthought.
None of this makes judges unusable. It makes them a component you have to validate. Measure agreement between your judge and human labels on a held-out sample, report that number, and re-check it whenever you change the judge model or its prompt.
Evaluate the system, not the model
For production systems, LLM evaluation should measure the application, not just the base model. Users never touch a model in isolation. They experience prompts, retrieval, tools, memory, orchestration, and policies as one thing, and a failure in any of them looks identical from the outside: a bad answer.
This matters for attribution. If you only score the final response, a retrieval failure and a reasoning failure produce the same red mark. Scoring at the span level, where each retrieval, tool call, and generation gets its own evaluation, is what separates them. Teams generally arrive here in stages, and the progression from a first eval to continuous evaluation in production tends to follow the same order regardless of the stack.
Scale forces a second decision. Running a large judge model on every span of every production request costs real money and adds latency, so most teams sample, use smaller models for high-volume checks, and reserve the expensive judge for spans that a cheap check already flagged. The budgeting tradeoffs behind evaluation cost are worth working out before the bill arrives rather than after.
FAQ
How do you evaluate an LLM when there is no correct answer?
You stop comparing to a target and start scoring properties. Check whether the answer is grounded in the retrieved context, whether it addresses the question, whether it follows the format and policy rules, and whether a judge applying your rubric would call it acceptable. Each of these is checkable without a reference string.
What is the difference between evaluating a model and evaluating an LLM application?
Model evaluation asks how a model performs on standard tasks, usually through public benchmarks, and helps you choose between models. Application evaluation asks whether your system, with your prompts, your retrieval, and your tools, produces good results for your users. A strong model can produce a weak application, and the second question is the one that determines whether you can ship.
Can I use the same model as both the generator and the judge?
You can, and many teams do because it is convenient. Just verify it first. A model scoring its own output can prefer its own style and phrasing, which inflates results in a way that is invisible unless you compare against human labels or a different judge model.
Do benchmark scores predict production quality?
Only loosely. Benchmarks measure general capability on fixed public tasks. Your application has its own data, its own prompts, and its own definition of a good answer. Use benchmarks to narrow a model shortlist, then evaluate the candidates on your own data before believing anything.
