> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Datasets

> Create versioned datasets for experimentation, evaluation, and fine-tuning. Supports Python dicts and pandas DataFrames.

<Note>
  The `datasets` client methods are currently in **BETA**. The API may change without notice. A one-time warning is emitted on first use.
</Note>

Create versioned datasets for experimentation, evaluation, and fine-tuning. Datasets are version-controlled collections of examples. Updates modify the current version in-place.

## Key Capabilities

* Create datasets from Python dicts or pandas DataFrames
* Append examples in-place to existing dataset versions
* Efficient bulk operations via Arrow Flight for large datasets
* Cache datasets locally for faster experiment iteration

## List Datasets

List all datasets with optional filtering by space or name.

```python theme={null}
resp = client.datasets.list(
    space="your-space-name-or-id",  # optional
    name="my-dataset",              # optional substring filter
    limit=50,
)

for dataset in resp.datasets:
    print(dataset.id, dataset.name)
```

For details on pagination, field introspection, and data conversion (to dict/JSON/DataFrame), see [Response Objects](/docs/api-clients/python/version-8/overview#response-objects).

## Create a Dataset

Create a new dataset with examples for evaluation or experimentation.

<CodeGroup>
  ```python From Dictionaries theme={null}
  examples = [
      {
          "query": "What is the capital of France?",
          "expected_output": "Paris",
          "eval.Correctness.label": "correct",
      },
      {
          "query": "Who wrote Romeo and Juliet?",
          "expected_output": "William Shakespeare",
          "eval.Correctness.label": "correct",
      },
  ]

  dataset = client.datasets.create(
      space="your-space-name-or-id",
      name="my-test-dataset",
      examples=examples,
  )
  ```

  ```python From DataFrame theme={null}
  import pandas as pd

  examples = pd.DataFrame({
      "query": ["What is AI?", "Explain ML"],
      "expected_output": ["Artificial Intelligence...", "Machine Learning..."],
  })

  dataset = client.datasets.create(
      space="your-space-name-or-id",
      name="my-dataset-from-df",
      examples=examples,
  )
  ```
</CodeGroup>

## Get a Dataset

Retrieve a specific dataset by name or ID. When using a name, provide `space` to disambiguate.

```python theme={null}
dataset = client.datasets.get(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
)

print(dataset)
```

## Delete a Dataset

Delete a dataset by name or ID. This operation is irreversible. There is no response from this call.

```python theme={null}
client.datasets.delete(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
)

print("Dataset deleted successfully")
```

## Rename a Dataset

Rename a dataset. The new name must be unique within the space.

```python theme={null}
dataset = client.datasets.update(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    name="renamed-dataset",
)

print(dataset.name)
```

## List Dataset Examples

Retrieve examples from a dataset with pagination support. Pass `all=True` to fetch all examples via Flight (ignores `limit`).

```python theme={null}
resp = client.datasets.list_examples(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    limit=100,
)

for example in resp.examples:
    print(example)
```

For details on pagination, field introspection, and data conversion (to dict/JSON/DataFrame), see [Response Objects](/docs/api-clients/python/version-8/overview#response-objects).

## Append Dataset Examples

Add new examples to an existing dataset. Examples are appended in-place to the latest dataset version by default — this does not create a new version. You can target a specific version by passing `dataset_version_id`. The response includes the dataset version the examples were written to (`dataset_version_id`) and the server-generated IDs of the inserted examples (`example_ids`).

<CodeGroup>
  ```python From Dictionaries theme={null}
  new_examples = [
      {
          "query": "What is machine learning?",
          "expected_output": "A subset of AI focused on learning from data",
          "eval.Correctness.label": "correct",
      },
      {
          "query": "Who invented Python?",
          "expected_output": "Guido van Rossum",
          "eval.Correctness.label": "correct",
      },
  ]

  result = client.datasets.append_examples(
      dataset="dataset-name-or-id",
      space="your-space-name-or-id",  # required when using a name
      examples=new_examples,
  )

  print(result.dataset_version_id, result.example_ids)
  ```

  ```python From DataFrame theme={null}
  import pandas as pd

  new_examples_df = pd.DataFrame({
      "query": ["Explain neural networks", "What is NLP?"],
      "expected_output": ["Networks inspired by biological neurons...", "Natural Language Processing..."],
  })

  result = client.datasets.append_examples(
      dataset="dataset-name-or-id",
      space="your-space-name-or-id",  # required when using a name
      examples=new_examples_df,
  )
  ```
</CodeGroup>

**Note:** Do not include system-managed fields (`id`, `created_at`, `updated_at`) in your examples. These are automatically generated by the server.

## Update Dataset Examples

Update the content of existing dataset examples by ID. Each example dict must include `id`; other keys replace the stored content of that example. Between 1 and 1000 examples may be updated per request. Example IDs that don't exist in the targeted version are ignored (no error, no insert).

By default the update is applied in place to the current (or specified) dataset version. To create a new dataset version with the update, pass `new_version`.

```python theme={null}
updated_examples = [
    {
        "id": "your-example-id",
        "query": "What is the capital of France?",
        "expected_output": "Paris (updated)",
        "eval.Correctness.label": "correct",
    },
]

result = client.datasets.update_examples(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    examples=updated_examples,
)

print(result.dataset_version_id, result.example_ids)
```

To create a new dataset version with this update:

```python theme={null}
result = client.datasets.update_examples(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",
    examples=updated_examples,
    new_version="v2-fixed-answers",
)
```

**Note:** Adding columns not already in the dataset schema is allowed; removing existing columns is not. Do not include system-managed fields (`created_at`, `updated_at`).

## Delete Dataset Examples

Delete a batch of examples from a dataset version by ID. Examples are removed in place from the given `dataset_version_id` — no new version is created. The operation is partial-tolerant and idempotent: re-submitting already-deleted IDs is safe. Between 1 and 1000 example IDs may be deleted per request.

```python theme={null}
result = client.datasets.delete_examples(
    dataset="dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    dataset_version_id="your-dataset-version-id",
    examples=["example-id-1", "example-id-2"],
)

if result.completed:
    print(f"Deleted {len(result.deleted_example_ids)} examples")
else:
    print("Deletion incomplete — retry the original request to finish")
```

Returns a `DeleteDatasetExamplesResponse` with:

* `completed` — `True` if the operation finished and no retry is needed.
* `deleted_example_ids` — IDs confirmed deleted.
* `not_deleted_example_ids` — Requested IDs that were not deleted.

## Annotate Dataset Examples

Write human annotations to a batch of examples in a dataset. Annotations are upserted by annotation config name for each example; submitting the same name for the same example overwrites the previous value. Up to 1000 examples may be annotated per request. This method returns `None` on success.

```python theme={null}
from arize.datasets.types import AnnotateRecordInput, AnnotationInput

client.datasets.annotate_examples(
    dataset="your-dataset-name-or-id",
    space="your-space-name-or-id",  # required when using a name
    annotations=[
        AnnotateRecordInput(
            record_id="your-example-id",
            values=[
                AnnotationInput(name="quality", score=0.9),
                AnnotationInput(name="topic", label="science"),
            ],
        ),
    ],
)
```

**Learn more:** [Datasets Documentation](https://arize.com/docs/ax/develop/datasets)