Training data is used to teach or adapt a model. Evaluation data is used to measure behavior after training, prompting, retrieval, or orchestration changes. The two are kept separate so that the measurement reflects generalization rather than memorization.
For most LLM applications, evaluation data never touches model training at all. There is no gradient step to protect. It is used instead to compare prompts, retrievers, tools, policies, and harness versions against each other. The separation requirement survives anyway, because what it protects was never really the weights. It was the claim that a score predicts behavior on inputs the system has not seen.
Build better agents with Arize
Trace, evaluate, and learn. Build agents that work with Arize AX and start tracing your runs today.
Prefer open source? Try Arize Phoenix for self-hosted, open source agent observability.
Key takeaways
- Training data shapes the system. Evaluation data judges it. An example can serve one role honestly or the other, never both at once.
- In an LLM application, “training” includes anything you tuned against: prompt text, few-shot examples, retrieval configuration, tool descriptions, routing thresholds. No fine-tune is required to consume an example.
- The score is the asset being protected. Once an example has influenced the system, its score stops predicting anything about new inputs.
- Split by user, session, document, or time period rather than by row. Random row splits let near-duplicates sit on both sides of the boundary.
- A three-way structure holds up in practice: a set you iterate against, a holdout you rarely touch, and a rotating sample of recent production traffic.
Two roles, one artifact
Both are collections of examples in the same format. What differs is what you are allowed to do with them.
| Training data | Evaluation data | |
|---|---|---|
| Purpose | Teach or adapt behavior | Measure behavior |
| Optimized against | Yes, directly | No, by design |
| Typical volume | As much as you can get | As little as gives a stable signal |
| Quality bar | Broad coverage, tolerant of noise | Correct labels, representative mix |
| Refresh reason | New capability or domain | Traffic moved or a new failure appeared |
The volume row surprises people arriving from classical machine learning. Training benefits from scale. Evaluation benefits from precision, and every extra example costs inference and scoring on every run, which pushes teams toward small sets chosen carefully rather than large sets run on everything.
Where the split came from and what it means now
Classical supervised learning splits data three ways: training, validation, and test. LLM application work uses the same roles under different names. The iteration set is validation; the holdout is test. Fitting a prompt over forty iterations against the same fifty examples is optimization, which is why defining AI engineering work in production treats splits as a stage with entry and exit rules.
What counts as training in a system with no training step
Anything you changed because of an example. That test is broader than most teams assume:
- Prompt and system instruction text edited to fix specific cases.
- Few-shot examples pasted into the prompt. Those are training examples, delivered at inference time.
- Retrieval settings tuned against known queries, including chunk size,
top_k, and reranker choice. - Tool descriptions and argument schemas rewritten because the model kept choosing wrong.
- Router or classifier thresholds tuned on a labeled sample.
- Any fine-tune, preference tuning, or adapter, where the classic definition applies directly.
An agent has more of these surfaces than a single-call application, because the prompt is only one of the things you tune. The harness around the model, meaning the loop, tools, context assembly, and policies, is also fitted to whatever examples you developed against, and its version belongs on every result you record.
What breaks when the separation is violated
The score stops being a prediction. A number produced on examples the system was tuned against tells you how well it handles those examples. It says nothing about the next thousand requests, which is the only thing anyone wanted to know.
Decisions inherit the inflation. Ship gates and prompt selection made against inflated scores route effort away from problems still in production.
Regressions arrive in production instead of in CI when nobody held out the behaviors nobody tested.
The mechanics of how information crosses the boundary, including contamination inside pretraining corpora and the quiet paths that involve no formal training, belong to leakage as a topic. This page is about which side of the line each set sits on.
Holding the line without a training pipeline
Split at the entity, not the row. Two paraphrases of one support ticket, or two chunks of one document, are not independent. Split by user, session, document, or time window so related records land together.
Prefer a time-based holdout. Reserve a recent window of traffic nobody iterated on. It is the closest thing to the future you can test against, and it catches the case where your development set has aged.
Keep a set you do not look at. Run it before a release rather than during development. If someone reads its failures and edits a prompt in response, it has become an iteration set and you need a new holdout.
Record which set informed which change. Every result should carry the dataset version, task or harness version, and evaluator version. Promoting cases from production traces is how the eval side stays fed, and doing that against live traffic is the ordinary practice of tracing and evaluating an agent in production rather than a special project.
FAQ
Is training data used in model evaluation?
Not the same rows. Evaluation runs on data held out from training, because a model scored on examples it was fit to reports its memory rather than its ability. Training data is still useful during evaluation for other purposes: checking that your eval set covers the same domains, searching for overlap between the two, and diagnosing whether a failure reflects a coverage gap.
Do LLM applications have training data at all?
Often not in the gradient sense. The base model was trained by someone else and your work happens through prompting, retrieval, and orchestration. But the examples you tuned those against play the training role, so the split still applies. Teams that fine-tune, distill to a smaller model, or train a router have both kinds in the literal sense.
What is the difference between validation data and test data?
Validation data guides choices during development and is partially consumed by that process, so its score runs optimistic. Test data is touched rarely, ideally once per release, and gives a less biased estimate. In LLM work the iteration set plays the validation role whether or not anyone calls it that.
Can the same examples be used for both if I am careful?
No, and carefulness is not the failure point. The moment you read an example’s result and change the system in response, that example has informed the system and its score is no longer a clean measurement. Copy it into the iteration set and pull a fresh one for the holdout.
