Skip to main content

Documentation Index

Fetch the complete documentation index at: https://arize-ax.mintlify.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

Instead of telling your team “go look at traces from yesterday,” create a labeling queue with the specific spans that need review — a structured list with an annotation schema and progress tracking.

​
Add spans to a queue

Individual: Open a span → click Add to → Labeling Queue → choose or create a queue. Bulk: Select multiple spans in the traces table → Add to → Labeling Queue. [screenshot: add to labeling queue menu from span toolbar]

​
The review workflow

  1. Open Annotation Queues from the left sidebar
  2. Select a queue → review each span’s inputs, outputs, and existing evals
  3. Add annotations using the queue’s annotation config
  4. Move to the next span
[screenshot: annotator queue view with span details and annotation form]

​
From queue to dataset

After labeling, create a dataset from the annotated results for experiments and fine-tuning.
With human review: Traces → Labeling Queue → Annotate → DatasetWithout human review: Traces → Dataset directly

​
Manage queues via API

You can create and manage annotation queues programmatically using the GraphQL API. This is useful for automating queue creation in CI/CD pipelines or building custom tooling. 
mutation {
  createAnnotationQueue(input: {
    projectId: "PROJECT_ID"
    name: "Weekly Safety Review"
    description: "Spans flagged by safety eval for human review"
  }) {
    annotationQueue { id name }
  }
}
The Python SDK (v8+) supports creating annotation configs programmatically. Annotation queue management via SDK is planned — use the GraphQL API in the meantime.