Annotation Queues

The ax annotation-queues commands are currently in BETA. The API may change without notice. A one-time warning is emitted on first use.

The ax annotation-queues commands let you create, retrieve, and manage annotation queues on the Arize platform. Annotation queues collect spans for human review and labeling.

`ax annotation-queues list`

List annotation queues in a space.

ax annotation-queues list [--space <id>] [--name <filter>] [--limit <n>] [--cursor <cursor>]

Option	Description
`--space`	Space name or ID
`--name`	Filter by queue name (substring match)
`--limit`	Maximum number of results to return (default: 15)
`--cursor`	Pagination cursor for the next page

Examples:

ax annotation-queues list --space sp_abc123
ax annotation-queues list --space sp_abc123 --name "review"

`ax annotation-queues create`

Create a new annotation queue. At least one --annotation-config-id and at least one --annotator-email are required.

ax annotation-queues create --name <name> --space <id> --annotation-config-id <id> --annotator-email <email> [options]

Option	Description
`--name`	Annotation queue name
`--space`	Space name or ID
`--annotation-config-id`	Annotation config ID to associate (repeat for multiple)
`--annotator-email`	Annotator email to assign (repeat for multiple)
`--instructions`	Instructions for annotators (max 5000 characters)
`--assignment-method`	How records are assigned to annotators: `ALL` or `RANDOM`
`--record-sources`	JSON file path or inline JSON array of initial record sources. Span sources need `project_id`, `start_time` (ISO 8601), `end_time` (ISO 8601), and optional `span_ids`. Example sources need `dataset_id` and `example_ids`.

Example:

ax annotation-queues create \
  --name "accuracy-review" \
  --space sp_abc123 \
  --annotation-config-id ac_xyz789 \
  --annotator-email reviewer@example.com \
  --instructions "Review each response for accuracy on a scale of 1-5"

`ax annotation-queues get`

Get an annotation queue by name or ID.

ax annotation-queues get <name-or-id> [--space <id>]

Option	Description
`--space`	Space name or ID (required when using queue name instead of ID)

Examples:

ax annotation-queues get aq_abc123
ax annotation-queues get "my-review-queue" --space sp_abc123

`ax annotation-queues update`

Update an annotation queue by name or ID. List fields (--annotation-config-id, --annotator-email) fully replace existing values when provided.

ax annotation-queues update <name-or-id> [--space <id>] [options]

Option	Description
`--space`	Space name or ID (required when using queue name instead of ID)
`--name`	New name for the annotation queue
`--instructions`	New instructions for annotators (pass empty string to clear)
`--annotation-config-id`	Full replacement list of annotation config IDs (repeat for multiple)
`--annotator-email`	Full replacement list of annotator emails (repeat for multiple)

Examples:

ax annotation-queues update aq_abc123 --name "accuracy-review-v2"
ax annotation-queues update aq_abc123 --annotator-email new@example.com --annotator-email other@example.com

`ax annotation-queues delete`

Delete an annotation queue by name or ID. This operation is irreversible.

ax annotation-queues delete <name-or-id> [--space <id>] [--force]

Option	Description
`--space`	Space name or ID (required when using queue name instead of ID)
`--force`	Skip the confirmation prompt

Examples:

ax annotation-queues delete aq_abc123 --force
ax annotation-queues delete "my-review-queue" --space sp_abc123 --force

`ax annotation-queues list-records`

List records in an annotation queue.

ax annotation-queues list-records <name-or-id> [--space <id>] [--limit <n>] [--cursor <cursor>]

Option	Description
`--space`	Space name or ID (required when using queue name instead of ID)
`--limit`	Maximum number of results to return (default: 15)
`--cursor`	Pagination cursor for the next page

Examples:

ax annotation-queues list-records aq_abc123
ax annotation-queues list-records aq_abc123 --limit 50 --output records.json

`ax annotation-queues delete-records`

Delete records from an annotation queue by record ID. At least one --record-id is required (max 100).

ax annotation-queues delete-records <name-or-id> --record-id <id> [--record-id <id> ...] [--space <id>] [--force]

Option	Description
`--record-id`	Record ID to delete (repeat for multiple, max 100)
`--space`	Space name or ID (required when using queue name instead of ID)
`--force`	Skip the confirmation prompt

Examples:

ax annotation-queues delete-records aq_abc123 --record-id rec_xyz789 --force
ax annotation-queues delete-records aq_abc123 --record-id rec_001 --record-id rec_002 --force

`ax annotation-queues annotate-record`

Submit an annotation for a record in an annotation queue. Annotations are upserted by annotation config name — call this command multiple times to annotate with different configs.

ax annotation-queues annotate-record <name-or-id> <record-id> --annotation-name <name> [options]

Option	Description
`--annotation-name`	Name of the annotation config
`--score`	Numeric score value
`--label`	Label value (for categorical annotation configs)
`--text`	Free-text annotation
`--space`	Space name or ID (required when using queue name instead of ID)

Examples:

# Categorical annotation
ax annotation-queues annotate-record aq_abc123 rec_xyz789 \
  --annotation-name "accuracy" \
  --label "accurate"

# Continuous annotation
ax annotation-queues annotate-record aq_abc123 rec_xyz789 \
  --annotation-name "relevance" \
  --score 0.9

# Freeform annotation
ax annotation-queues annotate-record aq_abc123 rec_xyz789 \
  --annotation-name "reviewer-notes" \
  --text "Response is mostly correct but misses edge cases"

`ax annotation-queues add-records`

Add records to an annotation queue. Records may come from spans (project time range) or dataset examples. Each item in the JSON array must include a record_type field set to SPAN or EXAMPLE, plus type-specific fields.

ax annotation-queues add-records <name-or-id> --record-sources <json-or-path> [--space <id>]

Option	Description
`--record-sources`	JSON file path or inline JSON array of record sources (required). Span sources need `project_id`, `start_time` (ISO 8601), `end_time` (ISO 8601), and optional `span_ids`. Example sources need `dataset_id` and `example_ids`.
`--space`, `-s`	Space name or ID (required when using queue name instead of ID)

Examples:

# Add spans from a project time range (sources in a file)
ax annotation-queues add-records my-queue --record-sources sources.json

# Add specific dataset examples (inline JSON)
ax annotation-queues add-records my-queue \
  --record-sources '[{"record_type": "EXAMPLE", "dataset_id": "ds_xyz789", "example_ids": ["ex_1", "ex_2"]}]'

`ax annotation-queues assign-record`

Assign users to a record in an annotation queue. Fully replaces all existing record-level assignments. Pass no --email flags to remove all assignments.

ax annotation-queues assign-record <name-or-id> <record-id> [--email <email> ...] [--space <id>]

Option	Description
`--email`	User email to assign (repeat for multiple; omit to clear all assignments)
`--space`	Space name or ID (required when using queue name instead of ID)

Examples:

# Assign one user
ax annotation-queues assign-record aq_abc123 rec_xyz789 --email reviewer@example.com

# Assign multiple users
ax annotation-queues assign-record aq_abc123 rec_xyz789 --email user1@example.com --email user2@example.com

# Clear all assignments
ax annotation-queues assign-record aq_abc123 rec_xyz789

API reference

API clients

`ax annotation-queues list`

`ax annotation-queues create`

`ax annotation-queues get`

`ax annotation-queues update`

`ax annotation-queues delete`

`ax annotation-queues list-records`

`ax annotation-queues delete-records`

`ax annotation-queues annotate-record`

`ax annotation-queues add-records`

`ax annotation-queues assign-record`

​ax annotation-queues list

​ax annotation-queues create

​ax annotation-queues get

​ax annotation-queues update

​ax annotation-queues delete

​ax annotation-queues list-records

​ax annotation-queues delete-records

​ax annotation-queues annotate-record

​ax annotation-queues add-records

​ax annotation-queues assign-record

`ax annotation-queues list`

`ax annotation-queues create`

`ax annotation-queues get`

`ax annotation-queues update`

`ax annotation-queues delete`

`ax annotation-queues list-records`

`ax annotation-queues delete-records`

`ax annotation-queues annotate-record`

`ax annotation-queues add-records`

`ax annotation-queues assign-record`