What Is Data Quality?

Data quality is whether a dataset is fit for the job you are asking it to do. In machine learning that job is specific: feed a model inputs that look like the inputs it was trained on, and record outcomes accurately enough that you can measure what happened. A dataset can be clean by every generic standard and still be unfit, because fitness is defined by the model consuming it.

The reason this matters more in AI systems than in most software is that bad data does not raise an exception. A malformed request to an API returns a 400 and someone gets paged. A feature that silently starts arriving as a string instead of a float, or that starts coming through null for 30% of requests, produces a prediction anyway. The system stays green while the output quietly gets worse. Most of the practice of data quality in machine learning is about converting that silent failure into a loud one.

Key takeaways

• Data quality in machine learning means fitness for a specific model, not abstract cleanliness. The training distribution defines the standard.
• Bad inputs rarely throw errors. Models produce confident predictions from garbage, so the failure surfaces days later as a business metric decline.
• The highest-value checks are boring: missingness, out-of-range values, type mismatch, cardinality changes, duplicates, and staleness.
• Data quality is the input side of monitoring. Drift detection and performance measurement are the other two sides, and all three are needed to explain a regression.
• LLM and agent systems have the same problem with different inputs: retrieved context, tool call arguments, and prompt variables all need the same scrutiny a numeric feature gets.

The checks that catch most problems

You do not need an elaborate framework to catch the majority of real incidents. A small set of per-field checks, run against every inference window and compared to a reference window, does most of the work.

• Missingness. The share of records where a field is null or empty. Watch the rate, not just the presence. A field that is normally 2% null and is suddenly 40% null is an upstream outage even if the model still runs.
• Out of range. Values outside the interval the model was trained on. Capturing the p1 and p99 of a reference window gives you bounds that adapt to the real data instead of a threshold somebody guessed at once.
• Type mismatch. A float arriving as a string, a timestamp arriving in seconds instead of milliseconds, a boolean arriving as "Y" instead of true. These usually come from a schema change upstream.
• Cardinality shift. New categories that were never in training, or a categorical field whose distinct count collapses. Unseen categories often get silently mapped to a default bucket, which is worse than an error.
• Duplicates and staleness. Records repeated by a retried job, or a feature table that stopped refreshing so every request reads the same stale values.

The mechanics of setting these up, including how to pick reference windows and what to alert on, are covered in a quick start to data quality monitoring for machine learning.

Why it is an engineering problem, not a cleaning problem

The instinct is to treat data quality as a preprocessing step: clean the data, then move on. That framing fails in production because the data keeps arriving and the upstream systems keep changing without telling you.

The problems that cause incidents originate outside the model: renamed columns, changed units, missing fields, duplicate backfills. Connect bad-data alerts to the job that produced them, as described in solving data quality with observability and data operations.

Data quality checks belong with drift and performance monitoring. When precision drops, the first question is whether the model degraded or an input broke.

Data quality for LLM and agent systems

The inputs changed; the discipline did not. An LLM application has its own set of fields that can quietly go wrong.

• Retrieved context. Empty retrieval results, chunks truncated mid-sentence, documents in the wrong language, or a stale index that no longer reflects the source of truth. A model handed poor context will still answer, and the answer will sound fine. Much of what gets labeled hallucination starts here.
• Tool call arguments. Agents generate structured arguments. Those arguments can be malformed JSON, out of range, or referencing an entity that does not exist. Validating them at the boundary is exactly the type mismatch check, moved one layer up.
• Prompt variables. A template with an unfilled placeholder, a system prompt truncated by a context limit, or conversation history dropped by a compaction step.
• Eval and test data. Datasets used to score a system are inputs too. Mislabeled examples in a golden set will make a good model look broken, which is one reason testing agents against curated cases requires maintaining the cases themselves.

Because these fields live inside spans rather than in a feature table, the checks run over trace data. The general shape of that setup is described in what an agent observability platform does.

Where teams get it wrong

Alerting on everything. Start with the fields the model actually weighs heavily.

Static thresholds that nobody revisits. Derive bounds from a rolling reference window.

Treating a passing check as proof of health. Checks can pass while the meaning of a field changes underneath you.

FAQ

How is data quality for AI different from data quality in analytics?

Analytics data quality is judged against business definitions. Machine learning data quality is judged against a model’s training distribution. Data can be correct by business standards and still be wrong for the model.

What are the most common data quality issues in machine learning?

Missing values, values outside the trained range, type and unit mismatches, new or missing categories, duplicated records, and stale feature tables. Nearly all of them originate in an upstream pipeline change rather than in the model code.

Can AI improve data quality, or does it only depend on it?

Both directions are real. Models are used to flag anomalous records, infer schemas, and suggest corrections. The dependency runs the other way as well, and it is the stronger of the two: a model trained or served on unfit data degrades no matter how good the architecture is. Automated detection helps with volume, but a person still has to decide what “correct” means for a given field.

When should data quality checks run?

At ingestion, so broken records can be rejected or quarantined before they reach a model, and again at inference time on live traffic, so you catch what the pipeline let through. Checking only at training time tells you about a snapshot, not about what the model is being asked to score today.