Programmatic Interface Limits and Capacity

In a self-hosted Arize AX deployment every lever is yours. The rate limits and payload bounds described here are the defaults a fresh install ships with, not ceilings. Some are a single values.yaml entry; others are a function of how many replicas of a component you run, which your sizing profile already sets. Arize AX splits its programmatic interfaces by the kind of work they carry. The GraphQL API and the REST API are the control plane: they configure, look up, and orchestrate, and they share the app servers and query backends that serve the UI, so they carry request rate limits to keep that surface responsive. The transport paths carry volume: Arrow Flight export and Data Fabric move data out, batch span logging and live OpenTelemetry ingest move data in. None of the transport paths carry a request quota on self-hosted; their throughput is bounded by the capacity you provision. For guidance on which interface fits a given job and how to structure large exports and backfills, see Choosing an interface. This page covers what an operator can tune and what actually governs throughput on each path.

Levers at a glance

Lever	What governs it	How you control it
GraphQL rate limit	Per-minute request counter per API key, kept per app-server pod	`graphqlRateLimitPerMin` (default 300); scales with app-server replica count
REST rate limits	Per-minute request counter per user per endpoint, kept per gateway pod	Per-service rate-limit ConfigMaps, overridden with a `baseOverlay` patch; scales with gateway replica count
Flight export capacity	Query-layer (broker) concurrency, historical node count and cache, export-service replicas and memory	Sizing profile: broker replicas, historicals, flight server replicas and memory
Data Fabric	Coordinator batch size and fixed worker count; sync cadence	`dataFabricEnabled`, `dataFabricCoordinatorBatchSize`, `dataFabricRetainDataEnabled`; no replica lever (exactly one coordinator and one worker, no autoscaling; contact Arize AI for more worker capacity)
Inbound batch and live ingest	Receiver and importer capacity; per-span payload bounds; per-upload size cap	Sizing profile; `receiverMaxSpanPayloadCharacters`
Export governance	Whether exports are allowed at all and whether the UI offers them	`flightserverDownloadEnabled`, `expBaseUrl`, ingress stream timeouts

Several of the Helm values on this page are not yet documented elsewhere in these docs. Confirm names and defaults against the values.yaml reference shipped with your release before changing them, and treat any change to replica counts or resource requests as an Arize-reviewed sizing change.

GraphQL API

The GraphQL API is served by the app-server pods. Three protections apply to every request: a per-minute rate limit, a per-query complexity budget, and mandatory pagination on every connection field. Only the first is tunable.

Rate limit

A single Helm or custom-resource value, graphqlRateLimitPerMin (default 300), sets both the query and the mutation per-minute limits to the same number. Queries and mutations are counted separately, so a workflow that mixes reads and writes gets the full budget on each side. The limit is keyed by API key and uses a one-minute window. Two properties matter for capacity planning:

Counters live in memory on each app-server pod. They are not shared across the cluster, so the effective limit a client experiences is roughly graphqlRateLimitPerMin multiplied by the number of app-server replicas, depending on how the ingress balances requests. Adding app-server replicas raises the effective GraphQL rate even if you leave the value alone.
Rejected requests do not count against the window. A client that backs off and retries after a rejection loses nothing, so simple retry loops are safe.

A rejected request returns a GraphQL error with extensions.code set to TOO_MANY_REQUESTS. It is not an HTTP 429 and does not carry a Retry-After header; clients should wait for the next minute window. A small number of write mutations (for example, bulk annotation updates) carry their own fixed per-minute limits that graphqlRateLimitPerMin does not change.

Complexity budget and pagination

Every operation is scored before it runs. Connection fields cost their requested page size multiplied by the cost of the fields beneath them, so nesting multiplies cost quickly. Operations above the budget of 1000 are rejected with Query is too complex. This budget is the same on every deployment and is not a Helm value. Every connection also requires first or last, and each connection has its own maximum page size (typically 50, varying by field). Requests above the maximum are rejected with an error that states the limit. Page caps are not tunable. To measure cost before you commit to a shape, add the root field rateLimit { limit cost } to any query. It is free against the complexity budget (it still counts as one request against the rate limit) and returns the cost of the whole operation and the budget it is scored against.

query {
  rateLimit {
    limit
    cost
  }
  account {
    spaces(first: 10) {
      edges {
        node {
          id
          name
        }
      }
    }
  }
}

Run the query once with a small page size, read cost, and scale the page size up until the result approaches limit. That tells you the largest flat page the budget allows for that shape, which is the most efficient way to use the per-minute rate.

Trade-off

Raising graphqlRateLimitPerMin buys roughly rate × page size rows per minute: at the default 300 and a typical page of 50, that is on the order of 15,000 rows per minute. Every one of those requests lands on the app servers and the query backend that also serve the UI, so a higher limit buys control-plane load, not export bandwidth. For anything larger than a few pages, route the work to Arrow Flight or Data Fabric instead.

REST API

The REST API (/v2) is fronted by an API gateway that routes to several backend services and applies rate limits before a request reaches any of them.

Defaults

The default limit is 100 requests per minute, keyed per user (all of one user’s API keys share a bucket), per route pattern and HTTP method, on a fixed one-minute window that resets when it expires. A loop hammering one endpoint throttles itself; a pipeline that touches several endpoints does not. A set of endpoints carry stricter limits, and POST /v2/spans (the List spans query, not ingest) is one of them at 10 per minute. The full table lives in the REST API overview. When a limit is exceeded the gateway returns 429 Too Many Requests with a Retry-After header set to the number of whole seconds left in the window, and a problem-details body:

HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 42

{
  "status": 429,
  "title": "Too Many Requests",
  "type": "https://arize.com/docs/ax/rest-reference/errors#rate-limit-exceeded"
}

Honor Retry-After literally; there are no X-RateLimit-* headers to read ahead of time, so a client cannot see how much budget remains until it is rejected.

What you can tune

The limits are defined in per-service rate-limit ConfigMaps that the gateway watches. There is no dedicated Helm value for them. To change a limit, you apply a baseOverlay patch in values.yaml that targets the relevant ConfigMap. baseOverlay is the operator’s general mechanism for Arize-reviewed patches to application manifests; its shape is shown in Cluster and storage. The ConfigMap names and keys vary by release, so work with Arize AI for the exact overlay before applying one. Counters are in memory on each gateway pod, so, as with GraphQL, the effective REST rate a client sees scales with the number of gateway replicas.

Trade-off

REST is a lookup and automation surface, not an export surface. Even with the largest data page size (500 on POST /v2/spans), the default limit works out to about 5,000 spans per minute per user. Raising the span-query limit pushes more load onto the analytical store that serves dashboards and monitors. Bulk span access belongs on Arrow Flight.

Arrow Flight export

Arrow Flight is the designed bulk-read path. The Python SDK’s Flight-backed export methods (export_to_df, export_to_parquet, list_examples(all=True), list_runs(all=True)) all use it, as do the ax CLI’s --all exports. The Flight (gRPC) path has no per-request rate limit or quota; only authentication and access control apply. What bounds it is capacity.

Capacity dimensions

Dimension	What governs it	How to raise it
Concurrent export streams	Capacity of the trace store’s query layer (the ADB broker), which every stream passes through	Add broker replicas in your sizing profile
Per-stream speed	Historical node count, per-node query parallelism, and segment cache hit rate	Scale historicals, give them more memory, and size the segment cache in your sizing profile
Export service headroom	Flight server replicas and memory. Dataset and experiment exports stream one segment at a time, so pod memory must fit the largest segment being streamed, not the whole dataset	Add flight server replicas or raise their memory request
Stream duration	Ingress and load-balancer idle timeouts, which end a long stream regardless of server state	Reference ingress configs allow `7200` seconds (2 hours). See the timeout annotations in Other ingress controllers

Two fixed bounds also apply. The gRPC maximum message size is 512 MB on the server, and the Python SDK uses the same value. Exports have no server-side time-range cap: you choose the start and end, and the server validates only that the end is after the start.

Reading the symptoms

Because nothing on the Flight path returns a rate-limit error, capacity problems show up as behavior rather than as a status code. The table maps what export jobs experience to the dimension to scale.

What you observe	Most likely dimension	Where to look first
Many streams run slowly when several jobs overlap	Concurrent streams	Broker replicas and broker resource usage
A single stream is slow even when it is the only job running	Per-stream speed	Historical CPU and memory, segment cache hit rate, whether the window spans many segments
Flight server pods restart or are OOM-killed during dataset or experiment exports	Export service headroom	Flight server memory request versus the size of the largest segment
Streams end at a consistent wall-clock duration (for example, close to two hours) with a connection error	Stream duration	Ingress or load-balancer timeout annotations
Dashboards and monitors slow down while exports run	Shared query capacity	Schedule exports off peak, or add broker and historical capacity for both workloads

Governance values

Value	Default	Effect
`flightserverDownloadEnabled`	`true`	When `false`, the Flight server refuses every export with a permission-denied error. This blocks all Flight exports: tracing, datasets, experiments, and ML data.
`expBaseUrl`	set during ingress setup	The Flight host the UI advertises. Leaving it empty hides export actions in the UI. See Configuring ingress endpoints.
`flightserverSpanLookbackDays`	`31`	How far back the server looks for target spans when the SDK updates evaluations, annotations, or metadata on existing spans. It does not limit export range.

flightserverSpanLookbackDays is often mistaken for an export window. It governs the lookup the server performs when you call update_evaluations, update_annotations, or update_metadata, so that the write can find the spans it targets. Exports are governed only by the time range you pass.

Trade-offs

Capacity is cost. Brokers and historicals also serve dashboards, monitors, and the UI, so sizing the cluster for heavy export concurrency means sizing it for both workloads at once, or scheduling heavy exports off peak. A stream that outlives its ingress window restarts from the beginning rather than resuming, so chunk long exports into time windows that comfortably fit inside the timeout. A failed window then costs one window to rerun, and windows can run in parallel up to the broker capacity you have. The chunking pattern is described in Choosing an interface. If a job re-pulls the same projects on a schedule, it is usually a Data Fabric connector in disguise.

Data Fabric

Data Fabric replicates trace data for selected projects into object storage you own, as Iceberg or Delta tables, where Snowflake, BigQuery, or Databricks can query it. On self-hosted it is off by default and is managed in the UI or through GraphQL; there is no REST surface. Setup, permissions, and query-table steps are in Data Fabric.

Values

Value	Default	Effect
`dataFabricEnabled`	`false`	Deploys the Data Fabric coordinator and worker. When `false`, no Data Fabric pods run.
`dataFabricCoordinatorBatchSize`	`100`	Number of segments the coordinator processes concurrently per sync batch. Affects how quickly a backlog drains, not what eventually arrives.
`dataFabricRetainDataEnabled`	`true`	Keeps segments in your bucket after they age out of Arize retention. Set to `false` to align bucket retention with ADB retention. See the 11.51.0 upgrade notes in Self-hosted releases.
`dataFabricPermissionsCheckEnabled`	`true`	Runs the bucket-ownership and IAM validation when a connector is created.

Capacity and pacing

When enabled, self-hosted runs a single coordinator (the single writer for each table) and a single worker. There is no autoscaling and no sizing lever for these replica counts; contact Arize AI if a high-volume space needs more worker capacity. There is no volume cap on what Data Fabric replicates: it copies every segment for the selected projects, not a sample. Sync runs on a schedule, hourly by default, and picks up segments as the trace store publishes them. Treat it as a replication feed with roughly hourly freshness plus segment publish delay, not as a low-latency API. Under a backlog (for example, after enabling a connector on a project with a long history, or after a large backfill), dataFabricCoordinatorBatchSize and worker capacity determine how fast the feed catches up.

Trade-offs

Storage cost moves to your bucket. With dataFabricRetainDataEnabled left at true, the bucket grows without bound as data ages out of Arize retention. Put a lifecycle policy on the bucket that matches your own retention requirements, or set the value to false if you want the bucket to track ADB retention.
Egress follows the bucket. If the destination bucket is in a different cloud or region from the deployment, cross-cloud egress charges apply to every sync, and a large backfill upstream becomes a large transfer bill downstream.
Freshness is segment cadence. Data lands when the trace store publishes a segment and the next scheduled sync picks it up. If a consumer needs the last few minutes of traces, it should query Arize directly rather than the replicated tables.

Batch span logging and live ingest

Inbound volume reaches Arize AX on two paths that share one ingest pipeline: live OpenTelemetry traces from instrumented applications, and batch uploads from the Python SDK’s client.spans.log (a DataFrame of spans with an optional DataFrame of evaluations). Both pass through the same receiver and the same validation rules. On self-hosted there is no per-request quota on either path; ingest is absorbed by the receiver and importer capacity in your sizing profile.

How a batch upload is bounded

The SDK validates the DataFrame locally (when validate=True), writes it as a single Arrow file, and uploads it in one HTTP request. The server then imports it asynchronously. Three bounds apply:

Per-upload cap. The server accepts at most 40,000,000 rows or 20 GiB in a single upload. Larger jobs must be split into multiple calls.
Per-span payload limits. Individual span fields are bounded on the ingest side, for example input and output values at 6,000,000 characters and LLM message content at 4,000,000 characters. receiverMaxSpanPayloadCharacters adjusts the per-span payload character limit; confirm its exact scope against your release’s values reference. Because the same validator runs on live traces, changing it applies to both paths.
Fail-fast errors. Any non-2xx response raises in the SDK immediately; there is no built-in retry. Because the upload is a single request, a failure means the whole call must be resubmitted.

Value	Default	Effect
`receiverMaxSpanPayloadCharacters`	release default	Adjusts the per-span payload character limit applied to batch uploads and live OTel traces alike; confirm its exact scope against your release’s values reference.

Trade-offs

Backfilled volume is volume. Very large backfills (tens of millions of rows) should be staged as a sequence of bounded, time-contiguous uploads and paced over hours or off peak, so that the ingest pipeline is not competing with live traffic and so that a mid-job failure costs one chunk rather than the whole job. Everything you backfill flows into indexing, retention, and Data Fabric replication exactly like live data: a 50-million-row backfill is also a 50-million-row sync. Endpoint configuration for the SDK on self-hosted is in SDK usage (v8).

Bounds that are not tunable

A few limits are the same on every deployment and are not exposed as Helm values. Plan around them rather than looking for an override.

Bound	Value	Applies to
GraphQL complexity budget	`1000` per operation	Every GraphQL query and mutation
GraphQL page size caps	Typically `50`, varies per connection; the error names the maximum	Every GraphQL connection field
Fixed per-mutation GraphQL limits	Small per-minute limits on a few bulk write mutations	Annotation and dataset-example bulk updates
REST page size caps	`100` on metadata endpoints, `500` on data endpoints such as `POST /v2/spans`, `50` on `POST /v2/traces`	Out-of-range values are rejected, not clamped
Flight gRPC message size	`512 MB`	Every Flight stream, server and Python SDK
Batch upload size	`40,000,000` rows or `20 GiB` per upload	`client.spans.log` and other Arrow uploads

Where to spend the effort

Rate limits guard the control plane and are configuration: one value for GraphQL, a ConfigMap overlay for REST, both multiplied by replica count. Data volume rides paths that scale with provisioned capacity: brokers and historicals for Flight export, coordinator and workers for Data Fabric, receivers and importers for ingest. When a team asks to raise API limits because an export or backfill is slow, the productive conversation is usually a different one: size the cluster for the export concurrency you need, and use Data Fabric for standing replication. Sizing starts with the clusterSizing value Arize AI approves for your environment (see the self-hosting FAQ).

Architecture

Getting started

Platform

Ingress

Use Arize

Advanced

Resources

Programmatic Interface Limits and Capacity

Levers at a glance

GraphQL API

Rate limit

Trade-off

REST API

Defaults

What you can tune

Trade-off

Arrow Flight export

Capacity dimensions

Reading the symptoms

Governance values

Trade-offs

Data Fabric

Values

Capacity and pacing

Trade-offs

Batch span logging and live ingest

How a batch upload is bounded

Trade-offs

Bounds that are not tunable

Where to spend the effort

​Levers at a glance

​GraphQL API

​Rate limit

​Complexity budget and pagination

​Trade-off

​REST API

​Defaults

​What you can tune

​Trade-off

​Arrow Flight export

​Capacity dimensions

​Reading the symptoms

​Governance values

​Trade-offs

​Data Fabric

​Values

​Capacity and pacing

​Trade-offs

​Batch span logging and live ingest

​How a batch upload is bounded

​Trade-offs

​Bounds that are not tunable

​Where to spend the effort

Levers at a glance

GraphQL API

Rate limit

Complexity budget and pagination

Trade-off

REST API

Defaults

What you can tune

Trade-off

Arrow Flight export

Capacity dimensions

Reading the symptoms

Governance values

Trade-offs

Data Fabric

Values

Capacity and pacing

Trade-offs

Batch span logging and live ingest

How a batch upload is bounded

Trade-offs

Bounds that are not tunable

Where to spend the effort