Annotation Queues

The annotation_queues client methods are currently in BETA. The API may change without notice. A one-time warning is emitted on first use.

Create and manage annotation queues for human labeling workflows. Annotation queues let teams collect human feedback on spans or dataset examples.

Key Capabilities

Create and configure annotation queues with annotators and annotation configs
Add spans or dataset examples as records for annotation
Submit annotations and assign records to specific users
List and delete records

List Annotation Queues

resp = client.annotation_queues.list(
    space="your-space-name-or-id",  # optional
    name="quality",                 # optional substring filter
    limit=50,
)

for queue in resp.annotation_queues:
    print(queue.id, queue.name)

For details on pagination, field introspection, and data conversion (to dict/JSON/DataFrame), see Response Objects.

Create an Annotation Queue

from arize.annotation_queues.types import AssignmentMethod

queue = client.annotation_queues.create(
    name="Quality Review Queue",
    space="your-space-name-or-id",
    annotation_config_ids=["your-annotation-config-id"],
    annotator_emails=["reviewer@example.com"],
    assignment_method=AssignmentMethod.ALL,  # AssignmentMethod.ALL or AssignmentMethod.RANDOM (optional)
    instructions="Review each span for accuracy and relevance.",  # optional
)

print(queue.id)

Get an Annotation Queue

# By ID
queue = client.annotation_queues.get(annotation_queue="your-queue-id")

# By name (requires space)
queue = client.annotation_queues.get(
    annotation_queue="Quality Review Queue",
    space="your-space-name-or-id",
)

print(queue.id, queue.name)

Update an Annotation Queue

At least one field must be provided. List fields (annotation_config_ids, annotator_emails) fully replace the existing values when provided.

queue = client.annotation_queues.update(
    annotation_queue="your-queue-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    name="Updated Queue Name",
    annotator_emails=["reviewer@example.com", "lead@example.com"],
)

Delete an Annotation Queue

client.annotation_queues.delete(
    annotation_queue="your-queue-name-or-id",
    space="your-space-name-or-id",  # required when using a name
)

Manage Records

List Records

resp = client.annotation_queues.list_records(
    annotation_queue="your-queue-name-or-id",
    limit=100,
)

for record in resp.records:
    print(record.id)

Add Records

Add spans or dataset examples as records for annotation. At most 2 record sources and 500 total records per request.

from arize.annotation_queues.types import (
    AnnotationQueueRecordInput,
    AnnotationQueueSpanRecordInput,
)
from datetime import datetime, timezone

records_resp = client.annotation_queues.add_records(
    annotation_queue="your-queue-name-or-id",
    record_sources=[
        AnnotationQueueRecordInput(
            actual_instance=AnnotationQueueSpanRecordInput(
                record_type="SPAN",
                project_id="your-project-id",
                start_time=datetime(2026, 3, 1, tzinfo=timezone.utc),
                end_time=datetime(2026, 3, 8, tzinfo=timezone.utc),
                span_ids=["your-span-id"],
            )
        ),
    ],
)

Annotate a Record

Submit annotations for a record. Annotations are upserted by annotation config name; omitted configs are left unchanged.

from arize.annotation_queues.types import AnnotationInput

result = client.annotation_queues.annotate_record(
    annotation_queue="your-queue-name-or-id",
    record_id="your-record-id",
    annotations=[
        AnnotationInput(name="accuracy", label="correct", score=1.0),
        AnnotationInput(name="quality", text="Well-structured response"),
    ],
)

Assign a Record

Fully replaces the record-level user assignments. Pass an empty list to remove all assignments.

result = client.annotation_queues.assign_record(
    annotation_queue="your-queue-name-or-id",
    record_id="your-record-id",
    assigned_user_emails=["reviewer@example.com"],
)

Delete Records

client.annotation_queues.delete_records(
    annotation_queue="your-queue-name-or-id",
    record_ids=["your-record-id-1", "your-record-id-2"],
)

API reference

API clients

Key Capabilities

List Annotation Queues

Create an Annotation Queue

Get an Annotation Queue

Update an Annotation Queue

Delete an Annotation Queue

Manage Records

List Records

Add Records

Annotate a Record

Assign a Record

Delete Records

​Key Capabilities

​List Annotation Queues

​Create an Annotation Queue

​Get an Annotation Queue

​Update an Annotation Queue

​Delete an Annotation Queue

​Manage Records

​List Records

​Add Records

​Annotate a Record

​Assign a Record

​Delete Records

Key Capabilities

List Annotation Queues

Create an Annotation Queue

Get an Annotation Queue

Update an Annotation Queue

Delete an Annotation Queue

Manage Records

List Records

Add Records

Annotate a Record

Assign a Record

Delete Records