Managing datasets and annotations with GraphQL

Datasets store versioned example sets you run experiments against, and annotations let your team attach human labels and scores to spans, traces, sessions, dataset records and experiment runs. This guide covers the GraphQL mutations for tagging datasets and experiments, editing dataset version examples, and creating and populating annotation configs and queues so you can automate the human review workflow instead of clicking through the UI.

Find the IDs you need

Every mutation on this page takes a space-scoped ID (a dataset, experiment, dataset version or model/project ID). Start from viewer to list the spaces your API key can see, then pull the datasets, experiments and tags in a space. See using global node IDs for how these opaque IDs work.

query FindDatasetsExperimentsAndTags($spaceId: ID!) {
  node(id: $spaceId) {
    ... on Space {
      name
      datasets(first: 20) {
        edges {
          node {
            id
            name
            latestVersionRecordCount
            tags(first: 10) {
              edges {
                node { id name }
              }
            }
          }
        }
      }
      experiments(first: 20) {
        edges {
          node {
            id
            name
            exampleCount
          }
        }
      }
      tags(first: 50) {
        edges {
          node { id name color }
        }
      }
    }
  }
}

{
  "spaceId": "U3BhY2U6MTIz"
}

To get the IDs of individual examples inside a dataset version (needed for patches and deletes), expand latestDatasetVersion { id examples(first: 50) { edges { node { id } } } } on a Dataset.

Tag a dataset or experiment

Tags group datasets and experiments so you can filter for them later (for example, in the tagsAny or tagsAll arguments on Space.datasets and Space.experiments). Create the tags first with Space.tags, or reuse existing tag IDs, then attach them.

mutation TagDataset($input: AddTagsToDatasetInput!) {
  addTagsToDataset(input: $input) {
    result {
      __typename
      ... on TagAssociationSuccess {
        success
      }
      ... on TagAssociationError {
        errorMessage
      }
    }
  }
}

{
  "input": {
    "datasetId": "RGF0YXNldDo0NTY=",
    "tagIds": ["VGFnOjE=", "VGFnOjI="]
  }
}

Tagging an experiment uses the same shape with addTagsToExperiment and experimentId. Both mutations return a TagAssociationSuccess or TagAssociationError union, so check __typename rather than assuming success. Remove tags the same way with removeTagsFromDataset or removeTagsFromExperiment. Reference: addTagsToDataset, addTagsToExperiment.

Add production spans to a dataset version

Copy records straight from a project’s traces into a dataset version without exporting and re-uploading anything. This mirrors the curate a dataset workflow but lets you script the selection with a time range and filters.

mutation AddSpansToDatasetVersion($input: UpdateDatasetVersionExamplesInput!) {
  updateDatasetVersionExamples(input: $input) {
    result {
      __typename
      ... on DatasetVersionWithSampling {
        datasetVersion {
          id
          exampleCount
        }
      }
      ... on UpdateDatasetVersionExamplesError {
        error
      }
    }
  }
}

{
  "input": {
    "datasetVersionId": "RGF0YXNldFZlcnNpb246Nzg5",
    "modelExamples": [
      {
        "modelId": "TW9kZWw6MTEx",
        "modelEnvironment": "tracing",
        "startDate": "2026-09-01T00:00:00Z",
        "endDate": "2026-09-30T23:59:59Z",
        "records": {
          "recordIds": []
        }
      }
    ]
  }
}

An empty recordIds list under records selects every record in the interval that matches the set’s filters, so add a queryFilter (for example "annotation.Correctness.label == 'incorrect'") when you only want a slice of traffic. Use sessions.sessionIds instead of records.recordIds to copy whole sessions, and deletedRecordIds to remove examples. recordIds at the top level of NewExampleSet is deprecated in favor of records. Reference: updateDatasetVersionExamples.

Patch an example’s columns

Fix a value in an existing dataset example (for example, correcting an expected output) without re-adding the record.

mutation PatchDatasetExample($input: UpdateDatasetVersionExamplesInput!) {
  updateDatasetVersionExamples(input: $input) {
    result {
      __typename
      ... on DatasetVersionWithSampling {
        datasetVersion { id }
      }
    }
  }
}

{
  "input": {
    "datasetVersionId": "RGF0YXNldFZlcnNpb246Nzg5",
    "updates": [
      {
        "id": "RGF0YXNldEV4YW1wbGU6MQ==",
        "patches": [
          {
            "columnName": "expected_output",
            "dataType": "STRING",
            "value": "The refund takes 3 to 5 business days."
          }
        ]
      }
    ]
  }
}

value is always a string. dataType tells the server how to parse it, so a LONG or FLOAT column still takes a quoted numeric string. Reference: updateDatasetVersionExamples.

Create an annotation config

Annotation configs define the label schema annotators fill in. Pick categorical for a fixed label set, continuous for a numeric score range, or freeform for unstructured text notes with no extra config block.

mutation CreateCategoricalAnnotationConfig($input: CreateAnnotationConfigInput!) {
  createAnnotationConfig(input: $input) {
    annotationConfigOrError {
      __typename
      ... on AnnotationConfig {
        id
        name
        annotationConfigType
      }
      ... on CreateAnnotationConfigError {
        errorMessage
      }
    }
  }
}

{
  "input": {
    "spaceId": "U3BhY2U6MTIz",
    "annotationName": "Response correctness",
    "annotationConfigArgs": {
      "annotationConfigType": "categorical",
      "categoricalConfig": {
        "labelOptions": [
          { "label": "correct", "score": 1 },
          { "label": "incorrect", "score": 0 }
        ],
        "optimizationDirection": "maximize"
      }
    }
  }
}

For a continuous config, swap categoricalConfig for continuousConfig: { minValue: 1, maxValue: 5, optimizationDirection: maximize }. For freeform, set annotationConfigType: "freeform" and omit both config blocks. Reference: createAnnotationConfig.

Create an annotation queue for human review

Queues route a set of records to one or more annotators. You can create new annotation configs inline, reuse existing ones by ID, or both, and populate the queue directly from spans in a project’s trace data.

mutation CreateQueue($input: CreateAnnotationQueueInput!) {
  createAnnotationQueue(input: $input) {
    annotationQueueOrError {
      __typename
      ... on AnnotationQueue {
        id
        name
        numRecords
      }
      ... on AnnotationQueueError {
        errorCode
        errorMessage
      }
    }
  }
}

{
  "input": {
    "spaceId": "U3BhY2U6MTIz",
    "name": "Weekly safety review",
    "instructions": "Flag any response that gives financial advice without a disclaimer.",
    "annotationConfigIds": ["QW5ub3RhdGlvbkNvbmZpZzox"],
    "annotatorAssignment": {
      "userIds": ["VXNlcjox", "VXNlcjoy"]
    },
    "assignmentMethod": "random",
    "recordSpecifications": [
      {
        "sourceType": "spans",
        "modelId": "TW9kZWw6MTEx",
        "startDate": "2026-09-01T00:00:00Z",
        "endDate": "2026-09-07T23:59:59Z"
      }
    ],
    "maxRecords": 200
  }
}

assignmentMethod: all gives every assigned annotator every record; random spreads records across them. The older datasetVersionId and top-level recordIds inputs are deprecated in favor of recordSpecifications, which also supports pulling straight from a dataset version with sourceType: "dataset". Use columnAllowlist to limit which columns annotators see. Reference: createAnnotationQueue.

Add annotations to spans in bulk

Once your subject-matter experts have labeled data in an external tool (a spreadsheet, an internal review app), write the results back onto the matching spans in one call. batchUpdateAnnotations takes a single model/project ID and a list of per-record updates, each with its own annotation config and label or score.

mutation BulkAnnotateSpans($input: BatchUpdateAnnotationsInput!) {
  batchUpdateAnnotations(input: $input) {
    result {
      __typename
      ... on BatchUpdateAnnotationSuccess {
        success
      }
      ... on BatchUpdateAnnotationError {
        error
      }
    }
  }
}

{
  "input": {
    "modelId": "TW9kZWw6MTEx",
    "recordGranularity": "span",
    "recordAnnotationUpdates": [
      {
        "recordId": "e351b661b957e727",
        "startTime": "2026-09-15T10:15:30Z",
        "annotationUpdates": [
          {
            "annotationConfigId": "QW5ub3RhdGlvbkNvbmZpZzox",
            "annotation": {
              "name": "Response correctness",
              "updatedBy": "jane@example.com",
              "label": "incorrect",
              "annotationType": "Label"
            }
          }
        ]
      }
    ]
  }
}

startTime is a search-space filter, not just metadata: it must be at most 24 hours before the record’s actual timestamp or the record will not be found. annotationType is Label, Score or Text (capitalized), and only the matching field (label, score or text) on annotation is read. For a single record, use updateAnnotations with a modelRecordContext, experimentRunContext or datasetRecordContext instead of a model-wide batch. Reference: batchUpdateAnnotations, updateAnnotations.

Python: bulk-import annotations from a spreadsheet

The most common scripted use of this API is exporting labels from an external review tool and pushing them into Arize AX. This sends one batchUpdateAnnotations call per model.

import requests

API_KEY = "YOUR_ARIZE_API_KEY"
URL = "https://app.arize.com/graphql"

query = """
mutation BulkAnnotateSpans($input: BatchUpdateAnnotationsInput!) {
  batchUpdateAnnotations(input: $input) {
    result {
      __typename
      ... on BatchUpdateAnnotationError { error }
    }
  }
}
"""

variables = {
    "input": {
        "modelId": "TW9kZWw6MTEx",
        "recordGranularity": "span",
        "recordAnnotationUpdates": [
            {
                "recordId": row["span_id"],
                "startTime": row["span_start_time"],
                "annotationUpdates": [
                    {
                        "annotationConfigId": "QW5ub3RhdGlvbkNvbmZpZzox",
                        "annotation": {
                            "name": "Response correctness",
                            "label": row["label"],
                            "annotationType": "Label",
                        },
                    }
                ],
            }
            for row in rows  # rows loaded from your spreadsheet export
        ],
    }
}

response = requests.post(
    URL,
    json={"query": query, "variables": variables},
    headers={"x-api-key": API_KEY},
)
response.raise_for_status()
print(response.json())

Gotchas and behavior notes

Tag and annotation mutations return result unions, not errors

addTagsToDataset, removeTagsFromDataset, addTagsToExperiment, removeTagsFromExperiment, updateAnnotations and batchUpdateAnnotations all return a success/error union in their result field instead of throwing a GraphQL error on a logical failure. Always select both branches with inline fragments, as shown above, and check __typename.

AnnotationType values are capitalized

The AnnotationType enum is Label, Score, Text, not the lowercase label/score used in some older examples. Only populate the field on AnnotationInput that matches the chosen type.

note is deprecated on every annotation mutation

The note input on updateAnnotations, batchUpdateAnnotations (per record) and the annotation queue note path is deprecated in favor of a freeform-text annotation. Create a freeform annotation config and write the text through AnnotationInput.text with annotationType: "Text" instead.

recordIds is deprecated in two different inputs

NewExampleSet.recordIds (on updateDatasetVersionExamples) is superseded by records.recordIds, and CreateAnnotationQueueInput.recordIds / datasetVersionId are superseded by recordSpecifications. Both older fields still work but new integrations should use the replacement.

startTime is a search window, not free-form metadata

On ModelRecordContextInput and RecordAnnotationUpdateInput, startTime narrows the server-side search for the record and must be close to (at most 24 hours before) the record’s real timestamp, or the mutation will not find the record to annotate.

maxRecords on a queue is capped

CreateAnnotationQueueInput.maxRecords must be between 1 and 5000.

Dataset and annotation mutations

Full argument and return-type reference for all nine mutations.

All mutations

Index of every GraphQL mutation grouped by domain.

API explorer

Run queries and mutations interactively against your own account.

Skills

MCP server

API reference

API clients

Managing datasets and annotations with GraphQL

Find the IDs you need

Tag a dataset or experiment

Add production spans to a dataset version

Patch an example’s columns

Create an annotation config

Create an annotation queue for human review

Add annotations to spans in bulk

Python: bulk-import annotations from a spreadsheet

Gotchas and behavior notes

Dataset and annotation mutations

All mutations

API explorer

​Find the IDs you need

​Tag a dataset or experiment

​Add production spans to a dataset version

​Patch an example’s columns

​Create an annotation config

​Create an annotation queue for human review

​Add annotations to spans in bulk

​Python: bulk-import annotations from a spreadsheet

​Gotchas and behavior notes

Dataset and annotation mutations

All mutations

API explorer

Find the IDs you need

Tag a dataset or experiment

Add production spans to a dataset version

Patch an example’s columns

Create an annotation config

Create an annotation queue for human review

Add annotations to spans in bulk

Python: bulk-import annotations from a spreadsheet

Gotchas and behavior notes