> ## Documentation Index
> Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Programmatic Interface Limits and Capacity

> What governs GraphQL and REST rate limits, Arrow Flight export throughput, Data Fabric sync, and batch span ingest in self-hosted Arize AX, and which Helm values and sizing levers an operator can tune.

In a self-hosted Arize AX deployment every lever is yours. The rate limits and payload bounds described here are the defaults a fresh install ships with, not ceilings. Some are a single `values.yaml` entry; others are a function of how many replicas of a component you run, which your sizing profile already sets.

Arize AX splits its programmatic interfaces by the kind of work they carry. The **GraphQL API** and the **REST API** are the control plane: they configure, look up, and orchestrate, and they share the app servers and query backends that serve the UI, so they carry request rate limits to keep that surface responsive. The **transport paths** carry volume: Arrow Flight export and Data Fabric move data out, batch span logging and live OpenTelemetry ingest move data in. None of the transport paths carry a request quota on self-hosted; their throughput is bounded by the capacity you provision. For guidance on which interface fits a given job and how to structure large exports and backfills, see [Choosing an interface](/docs/api-reference/choosing-an-interface). This page covers what an operator can tune and what actually governs throughput on each path.

## Levers at a glance

| Lever | What governs it | How you control it |
| - | - | - |
| GraphQL rate limit | Per-minute request counter per API key, kept per app-server pod | `graphqlRateLimitPerMin` (default 300); scales with app-server replica count |
| REST rate limits | Per-minute request counter per user per endpoint, kept per gateway pod | Per-service rate-limit ConfigMaps, overridden with a `baseOverlay` patch; scales with gateway replica count |
| Flight export capacity | Query-layer (broker) concurrency, historical node count and cache, export-service replicas and memory | Sizing profile: broker replicas, historicals, flight server replicas and memory |
| Data Fabric | Coordinator batch size and fixed worker count; sync cadence | `dataFabricEnabled`, `dataFabricCoordinatorBatchSize`, `dataFabricRetainDataEnabled`; no replica lever (exactly one coordinator and one worker, no autoscaling; contact Arize AI for more worker capacity) |
| Inbound batch and live ingest | Receiver and importer capacity; per-span payload bounds; per-upload size cap | Sizing profile; `receiverMaxSpanPayloadCharacters` |
| Export governance | Whether exports are allowed at all and whether the UI offers them | `flightserverDownloadEnabled`, `expBaseUrl`, ingress stream timeouts |

> [!WARNING]
>
> Several of the Helm values on this page are not yet documented elsewhere in these docs. Confirm names and defaults against the `values.yaml` reference shipped with your release before changing them, and treat any change to replica counts or resource requests as an Arize-reviewed sizing change.

## GraphQL API

The GraphQL API is served by the app-server pods. Three protections apply to every request: a per-minute rate limit, a per-query complexity budget, and mandatory pagination on every connection field. Only the first is tunable.

### Rate limit

A single Helm or custom-resource value, `graphqlRateLimitPerMin` (default `300`), sets both the query and the mutation per-minute limits to the same number. Queries and mutations are counted separately, so a workflow that mixes reads and writes gets the full budget on each side. The limit is keyed by API key and uses a one-minute window.

Two properties matter for capacity planning:

1. **Counters live in memory on each app-server pod.** They are not shared across the cluster, so the effective limit a client experiences is roughly `graphqlRateLimitPerMin` multiplied by the number of app-server replicas, depending on how the ingress balances requests. Adding app-server replicas raises the effective GraphQL rate even if you leave the value alone.
2. **Rejected requests do not count against the window.** A client that backs off and retries after a rejection loses nothing, so simple retry loops are safe.

A rejected request returns a GraphQL error with `extensions.code` set to `TOO_MANY_REQUESTS`. It is not an HTTP `429` and does not carry a `Retry-After` header; clients should wait for the next minute window. A small number of write mutations (for example, bulk annotation updates) carry their own fixed per-minute limits that `graphqlRateLimitPerMin` does not change.

### Complexity budget and pagination

Every operation is scored before it runs. Connection fields cost their requested page size multiplied by the cost of the fields beneath them, so nesting multiplies cost quickly. Operations above the budget of `1000` are rejected with `Query is too complex`. This budget is the same on every deployment and is not a Helm value.

Every connection also requires `first` or `last`, and each connection has its own maximum page size (typically `50`, varying by field). Requests above the maximum are rejected with an error that states the limit. Page caps are not tunable.

To measure cost before you commit to a shape, add the root field `rateLimit { limit cost }` to any query. It is free against the complexity budget (it still counts as one request against the rate limit) and returns the cost of the whole operation and the budget it is scored against.

```graphql theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
query {
  rateLimit {
    limit
    cost
  }
  account {
    spaces(first: 10) {
      edges {
        node {
          id
          name
        }
      }
    }
  }
}
```

Run the query once with a small page size, read `cost`, and scale the page size up until the result approaches `limit`. That tells you the largest flat page the budget allows for that shape, which is the most efficient way to use the per-minute rate.

### Trade-off

Raising `graphqlRateLimitPerMin` buys roughly `rate × page size` rows per minute: at the default `300` and a typical page of `50`, that is on the order of 15,000 rows per minute. Every one of those requests lands on the app servers and the query backend that also serve the UI, so a higher limit buys control-plane load, not export bandwidth. For anything larger than a few pages, route the work to Arrow Flight or Data Fabric instead.

## REST API

The REST API (`/v2`) is fronted by an API gateway that routes to several backend services and applies rate limits before a request reaches any of them.

### Defaults

The default limit is `100` requests per minute, keyed per **user** (all of one user's API keys share a bucket), per **route pattern and HTTP method**, on a **fixed one-minute window** that resets when it expires. A loop hammering one endpoint throttles itself; a pipeline that touches several endpoints does not. A set of endpoints carry stricter limits, and `POST /v2/spans` (the List spans query, not ingest) is one of them at `10` per minute. The full table lives in the [REST API overview](/docs/ax/rest-reference/overview#rate-limits).

When a limit is exceeded the gateway returns `429 Too Many Requests` with a `Retry-After` header set to the number of whole seconds left in the window, and a problem-details body:

```http theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 42

{
  "status": 429,
  "title": "Too Many Requests",
  "type": "https://arize.com/docs/ax/rest-reference/errors#rate-limit-exceeded"
}
```

Honor `Retry-After` literally; there are no `X-RateLimit-*` headers to read ahead of time, so a client cannot see how much budget remains until it is rejected.

### What you can tune

The limits are defined in per-service rate-limit ConfigMaps that the gateway watches. There is no dedicated Helm value for them. To change a limit, you apply a `baseOverlay` patch in `values.yaml` that targets the relevant ConfigMap. `baseOverlay` is the operator's general mechanism for Arize-reviewed patches to application manifests; its shape is shown in [Cluster and storage](/docs/ax/selfhosting/installation/rancher-bare-metal/cluster-and-storage). The ConfigMap names and keys vary by release, so work with Arize AI for the exact overlay before applying one.

Counters are in memory on each gateway pod, so, as with GraphQL, the effective REST rate a client sees scales with the number of gateway replicas.

### Trade-off

REST is a lookup and automation surface, not an export surface. Even with the largest data page size (`500` on `POST /v2/spans`), the default limit works out to about 5,000 spans per minute per user. Raising the span-query limit pushes more load onto the analytical store that serves dashboards and monitors. Bulk span access belongs on Arrow Flight.

## Arrow Flight export

Arrow Flight is the designed bulk-read path. The Python SDK's Flight-backed export methods (`export_to_df`, `export_to_parquet`, `list_examples(all=True)`, `list_runs(all=True)`) all use it, as do the `ax` CLI's `--all` exports. The Flight (gRPC) path has **no per-request rate limit or quota**; only authentication and access control apply. What bounds it is capacity.

### Capacity dimensions

| Dimension | What governs it | How to raise it |
| - | - | - |
| Concurrent export streams | Capacity of the trace store's query layer (the ADB broker), which every stream passes through | Add broker replicas in your sizing profile |
| Per-stream speed | Historical node count, per-node query parallelism, and segment cache hit rate | Scale historicals, give them more memory, and size the segment cache in your sizing profile |
| Export service headroom | Flight server replicas and memory. Dataset and experiment exports stream one segment at a time, so pod memory must fit the largest segment being streamed, not the whole dataset | Add flight server replicas or raise their memory request |
| Stream duration | Ingress and load-balancer idle timeouts, which end a long stream regardless of server state | Reference ingress configs allow `7200` seconds (2 hours). See the timeout annotations in [Other ingress controllers](/docs/ax/selfhosting/installation/ingress/other-controllers) |

Two fixed bounds also apply. The gRPC maximum message size is `512 MB` on the server, and the Python SDK uses the same value. Exports have no server-side time-range cap: you choose the start and end, and the server validates only that the end is after the start.

### Reading the symptoms

Because nothing on the Flight path returns a rate-limit error, capacity problems show up as behavior rather than as a status code. The table maps what export jobs experience to the dimension to scale.

| What you observe | Most likely dimension | Where to look first |
| - | - | - |
| Many streams run slowly when several jobs overlap | Concurrent streams | Broker replicas and broker resource usage |
| A single stream is slow even when it is the only job running | Per-stream speed | Historical CPU and memory, segment cache hit rate, whether the window spans many segments |
| Flight server pods restart or are OOM-killed during dataset or experiment exports | Export service headroom | Flight server memory request versus the size of the largest segment |
| Streams end at a consistent wall-clock duration (for example, close to two hours) with a connection error | Stream duration | Ingress or load-balancer timeout annotations |
| Dashboards and monitors slow down while exports run | Shared query capacity | Schedule exports off peak, or add broker and historical capacity for both workloads |

### Governance values

| Value | Default | Effect |
| - | - | - |
| `flightserverDownloadEnabled` | `true` | When `false`, the Flight server refuses every export with a permission-denied error. This blocks **all** Flight exports: tracing, datasets, experiments, and ML data. |
| `expBaseUrl` | set during ingress setup | The Flight host the UI advertises. Leaving it empty hides export actions in the UI. See [Configuring ingress endpoints](/docs/ax/selfhosting/installation/ingress/configuring-endpoints). |
| `flightserverSpanLookbackDays` | `31` | How far back the server looks for target spans when the SDK **updates** evaluations, annotations, or metadata on existing spans. It does not limit export range. |

> [!NOTE]
>
> `flightserverSpanLookbackDays` is often mistaken for an export window. It governs the lookup the server performs when you call `update_evaluations`, `update_annotations`, or `update_metadata`, so that the write can find the spans it targets. Exports are governed only by the time range you pass.

### Trade-offs

Capacity is cost. Brokers and historicals also serve dashboards, monitors, and the UI, so sizing the cluster for heavy export concurrency means sizing it for both workloads at once, or scheduling heavy exports off peak. A stream that outlives its ingress window restarts from the beginning rather than resuming, so chunk long exports into time windows that comfortably fit inside the timeout. A failed window then costs one window to rerun, and windows can run in parallel up to the broker capacity you have. The chunking pattern is described in [Choosing an interface](/docs/api-reference/choosing-an-interface). If a job re-pulls the same projects on a schedule, it is usually a Data Fabric connector in disguise.

## Data Fabric

Data Fabric replicates trace data for selected projects into object storage you own, as Iceberg or Delta tables, where Snowflake, BigQuery, or Databricks can query it. On self-hosted it is **off by default** and is managed in the UI or through GraphQL; there is no REST surface. Setup, permissions, and query-table steps are in [Data Fabric](/docs/ax/security-and-settings/data-fabric).

### Values

| Value | Default | Effect |
| - | - | - |
| `dataFabricEnabled` | `false` | Deploys the Data Fabric coordinator and worker. When `false`, no Data Fabric pods run. |
| `dataFabricCoordinatorBatchSize` | `100` | Number of segments the coordinator processes concurrently per sync batch. Affects how quickly a backlog drains, not what eventually arrives. |
| `dataFabricRetainDataEnabled` | `true` | Keeps segments in your bucket after they age out of Arize retention. Set to `false` to align bucket retention with ADB retention. See the 11.51.0 upgrade notes in [Self-hosted releases](/docs/ax/selfhosting/on-premise-releases). |
| `dataFabricPermissionsCheckEnabled` | `true` | Runs the bucket-ownership and IAM validation when a connector is created. |

### Capacity and pacing

When enabled, self-hosted runs a single coordinator (the single writer for each table) and a single worker. There is no autoscaling and no sizing lever for these replica counts; contact Arize AI if a high-volume space needs more worker capacity. There is no volume cap on what Data Fabric replicates: it copies every segment for the selected projects, not a sample.

Sync runs on a schedule, hourly by default, and picks up segments as the trace store publishes them. Treat it as a replication feed with roughly hourly freshness plus segment publish delay, not as a low-latency API. Under a backlog (for example, after enabling a connector on a project with a long history, or after a large backfill), `dataFabricCoordinatorBatchSize` and worker capacity determine how fast the feed catches up.

### Trade-offs

1. **Storage cost moves to your bucket.** With `dataFabricRetainDataEnabled` left at `true`, the bucket grows without bound as data ages out of Arize retention. Put a lifecycle policy on the bucket that matches your own retention requirements, or set the value to `false` if you want the bucket to track ADB retention.
2. **Egress follows the bucket.** If the destination bucket is in a different cloud or region from the deployment, cross-cloud egress charges apply to every sync, and a large backfill upstream becomes a large transfer bill downstream.
3. **Freshness is segment cadence.** Data lands when the trace store publishes a segment and the next scheduled sync picks it up. If a consumer needs the last few minutes of traces, it should query Arize directly rather than the replicated tables.

## Batch span logging and live ingest

Inbound volume reaches Arize AX on two paths that share one ingest pipeline: live OpenTelemetry traces from instrumented applications, and batch uploads from the Python SDK's `client.spans.log` (a DataFrame of spans with an optional DataFrame of evaluations). Both pass through the same receiver and the same validation rules. On self-hosted there is **no per-request quota** on either path; ingest is absorbed by the receiver and importer capacity in your sizing profile.

### How a batch upload is bounded

The SDK validates the DataFrame locally (when `validate=True`), writes it as a single Arrow file, and uploads it in one HTTP request. The server then imports it asynchronously. Three bounds apply:

1. **Per-upload cap.** The server accepts at most `40,000,000` rows or `20 GiB` in a single upload. Larger jobs must be split into multiple calls.
2. **Per-span payload limits.** Individual span fields are bounded on the ingest side, for example input and output values at 6,000,000 characters and LLM message content at 4,000,000 characters. `receiverMaxSpanPayloadCharacters` adjusts the per-span payload character limit; confirm its exact scope against your release's values reference. Because the same validator runs on live traces, changing it applies to both paths.
3. **Fail-fast errors.** Any non-2xx response raises in the SDK immediately; there is no built-in retry. Because the upload is a single request, a failure means the whole call must be resubmitted.

| Value | Default | Effect |
| - | - | - |
| `receiverMaxSpanPayloadCharacters` | release default | Adjusts the per-span payload character limit applied to batch uploads and live OTel traces alike; confirm its exact scope against your release's values reference. |

### Trade-offs

Backfilled volume is volume. Very large backfills (tens of millions of rows) should be staged as a sequence of bounded, time-contiguous uploads and paced over hours or off peak, so that the ingest pipeline is not competing with live traffic and so that a mid-job failure costs one chunk rather than the whole job. Everything you backfill flows into indexing, retention, and Data Fabric replication exactly like live data: a 50-million-row backfill is also a 50-million-row sync. Endpoint configuration for the SDK on self-hosted is in [SDK usage (v8)](/docs/ax/selfhosting/on-premise-sdk-usage/version-8).

## Bounds that are not tunable

A few limits are the same on every deployment and are not exposed as Helm values. Plan around them rather than looking for an override.

| Bound | Value | Applies to |
| - | - | - |
| GraphQL complexity budget | `1000` per operation | Every GraphQL query and mutation |
| GraphQL page size caps | Typically `50`, varies per connection; the error names the maximum | Every GraphQL connection field |
| Fixed per-mutation GraphQL limits | Small per-minute limits on a few bulk write mutations | Annotation and dataset-example bulk updates |
| REST page size caps | `100` on metadata endpoints, `500` on data endpoints such as `POST /v2/spans`, `50` on `POST /v2/traces` | Out-of-range values are rejected, not clamped |
| Flight gRPC message size | `512 MB` | Every Flight stream, server and Python SDK |
| Batch upload size | `40,000,000` rows or `20 GiB` per upload | `client.spans.log` and other Arrow uploads |

## Where to spend the effort

> [!NOTE]
>
> Rate limits guard the control plane and are configuration: one value for GraphQL, a ConfigMap overlay for REST, both multiplied by replica count. Data volume rides paths that scale with provisioned capacity: brokers and historicals for Flight export, coordinator and workers for Data Fabric, receivers and importers for ingest. When a team asks to raise API limits because an export or backfill is slow, the productive conversation is usually a different one: size the cluster for the export concurrency you need, and use Data Fabric for standing replication. Sizing starts with the `clusterSizing` value Arize AI approves for your environment (see the [self-hosting FAQ](/docs/ax/selfhosting/getting-started/faq)).