Skip to main content
The Dataset Coverage Agent is part of managed agents on Enterprise. Signal and managed agents are currently included at no additional charge for Enterprise. See Pricing and usage.
The Dataset Coverage Agent keeps a dataset in step with production. On a schedule, it compares a tracing project’s recent spans with the dataset, finds the gaps your curation policy describes, and copies real spans into the dataset to close them. It never writes synthetic examples. Use it to keep an eval or experiment dataset representative as traffic changes, without hand-picking spans.

​
Run it

Signal & Agents → + New Agent → Monitor & optimize → Dataset Coverage. You can also open a tracing project and choose Agents → Dataset Coverage → Enable Agent, or select New dataset agent on the project’s Dataset Coverage tab. You can run several Dataset Coverage agents on one project, for example one per dataset or policy.

​
Write a curation policy

The policy is the agent’s only definition of a gap, so be specific about what to keep and in what proportion. For example:
Keep at least 20 examples per user intent, and prioritize traces where an evaluation failed.
The setup form offers starters for common policies: failing examples, balanced intent coverage, failure-stratified coverage, and rare or long-tail inputs. If a policy is empty, vague, or contradictory, or no span fills a gap, the run succeeds and adds nothing.

​
What each run does

1

Reads the dataset and recent spans

The agent reads every example in the dataset’s latest version, plus up to 200 spans from the project that arrived since the previous run. The first run looks back 24 hours.
2

Picks spans that close a gap

It chooses spans that fill gaps in the policy, never more than Maximum additions per run, and stops once the dataset reaches Target dataset size.
3

Adds them and files a report

It copies the chosen spans into the latest dataset version, then files one run report.
The first run starts as soon as you save. Select Run now on the tab to trigger another run off schedule.

​
Read run reports

Reports appear on the project’s Dataset Coverage tab, which shows once the project has an agent, and on the space-wide Issues page with a Dataset Coverage badge. If the project has more than one agent, pick one from the selector on the tab. The tab charts dataset size and confirmed additions per run. Each report lists the spans it added, links to each span’s details, records the final dataset example count, and explains why each span was chosen. Reports are a record of what changed, so they have no actions to accept. To stop the agent, choose Pause Dataset Coverage from the settings menu on the tab.