Annotation Configs

The annotation_configs functions are currently in BETA. The API may change without notice. A one-time warning is emitted on first use.

Create custom labels that can be added to represent human feedback. Annotation configs allow teams and subject matter experts to label data and curate high-quality datasets.

List Annotation Configs

import { listAnnotationConfigs } from "@arizeai/ax-client";

const { data: annotationConfigs, pagination } = await listAnnotationConfigs({
  space: "my-space",   // space name or ID (optional)
  name: "accuracy",    // substring filter on annotation config name (optional)
  limit: 10,
});

Create a Continuous Annotation Config

Continuous configs represent a numeric score within a fixed range (e.g. a 0-1 quality score).

import { createContinuousAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await createContinuousAnnotationConfig({
  name: "quality-score",
  space: "my-space",  // space name or ID
  minimumScore: 0,
  maximumScore: 1,
  optimizationDirection: "MAXIMIZE",  // optional
});

Create a Categorical Annotation Config

Categorical configs represent a fixed set of labeled values a scorer can choose from (e.g. correct / incorrect).

import { createCategoricalAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await createCategoricalAnnotationConfig({
  name: "correctness",
  space: "my-space",  // space name or ID
  values: [
    { label: "correct", score: 1 },
    { label: "incorrect", score: 0 },
  ],
  optimizationDirection: "MAXIMIZE",  // optional
});

Create a Freeform Annotation Config

Freeform configs represent open-ended text feedback with no predefined scale or set of values.

import { createFreeformAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await createFreeformAnnotationConfig({
  name: "reviewer-notes",
  space: "my-space",  // space name or ID
});

Get Annotation Config

import { getAnnotationConfig } from "@arizeai/ax-client";

// By ID
const annotationConfig = await getAnnotationConfig({
  annotationConfig: "your_annotation_config_id",
});

// By name (requires space)
const annotationConfig = await getAnnotationConfig({
  annotationConfig: "Accuracy",
  space: "my-space",
});

Update a Continuous Annotation Config

A config’s type is immutable. Any fields that are omitted are left unchanged.

import { updateContinuousAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await updateContinuousAnnotationConfig({
  annotationConfig: "quality-score",  // annotation config name or ID
  space: "my-space",                  // required when annotationConfig is a name
  name: "quality-score-v2",           // optional new name
  minimumScore: 0,                    // optional
  maximumScore: 10,                   // optional
  optimizationDirection: "MAXIMIZE",  // optional
});

Update a Categorical Annotation Config

values, when provided, fully replaces the existing label set (2-100 items).

import { updateCategoricalAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await updateCategoricalAnnotationConfig({
  annotationConfig: "correctness",  // annotation config name or ID
  space: "my-space",                // required when annotationConfig is a name
  name: "correctness-v2",           // optional
  values: [                         // optional full replacement
    { label: "correct", score: 1 },
    { label: "incorrect", score: 0 },
  ],
  optimizationDirection: "MAXIMIZE",  // optional
});

Update a Freeform Annotation Config

import { updateFreeformAnnotationConfig } from "@arizeai/ax-client";

const annotationConfig = await updateFreeformAnnotationConfig({
  annotationConfig: "reviewer-notes",  // annotation config name or ID
  space: "my-space",                   // required when annotationConfig is a name
  name: "reviewer-notes-v2",           // optional new name
});

Delete Annotation Config

import { deleteAnnotationConfig } from "@arizeai/ax-client";

// By ID
await deleteAnnotationConfig({
  annotationConfig: "your_annotation_config_id",
});

// By name (requires space)
await deleteAnnotationConfig({
  annotationConfig: "Accuracy",
  space: "my-space",
});

API reference

API clients

List Annotation Configs

Create a Continuous Annotation Config

Create a Categorical Annotation Config

Create a Freeform Annotation Config

Get Annotation Config

Update a Continuous Annotation Config

Update a Categorical Annotation Config

Update a Freeform Annotation Config

Delete Annotation Config

​List Annotation Configs

​Create a Continuous Annotation Config

​Create a Categorical Annotation Config

​Create a Freeform Annotation Config

​Get Annotation Config

​Update a Continuous Annotation Config

​Update a Categorical Annotation Config

​Update a Freeform Annotation Config

​Delete Annotation Config

List Annotation Configs

Create a Continuous Annotation Config

Create a Categorical Annotation Config

Create a Freeform Annotation Config

Get Annotation Config

Update a Continuous Annotation Config

Update a Categorical Annotation Config

Update a Freeform Annotation Config

Delete Annotation Config