> ## Documentation Index > Fetch the complete documentation index at: https://arize-ax.mintlify.site/docs/llms.txt > Use this file to discover all available pages before exploring further. # Choose the right interface for your workload > How the Arize AX GraphQL and REST APIs, Arrow Flight export, Data Fabric, and batch span logging divide by kind of work, how each is limited, and how to design bulk jobs. Arize AX exposes five programmatic paths. They divide by the kind of work, not by language or convenience. Two of them form the **control plane**. The GraphQL API and the REST API configure resources, look things up, and orchestrate workflows. They answer in milliseconds and are rate-limited so they stay that way. The other three are **transport paths**. Arrow Flight export and Data Fabric move data out; batch span logging moves data in. None of them carries the per-request rate limits that govern REST and GraphQL. What bounds them is the capacity of the trace store and ingest pipeline behind them. (On SaaS, batch span logging also has a per-space cap on upload calls, not on rows.) **Most throughput questions are really interface-selection questions.** A job that pages a month of spans through REST or GraphQL is working against the limiter on purpose. Raising the limit buys a little more load on the query layer, not export bandwidth. Move the job to the path built for its shape and the limit stops mattering. > [!NOTE] > > This page is for every AX deployment, SaaS and self-hosted alike. If you operate your own cluster and want to size the capacity behind these paths (ingress timeouts, query-layer replicas, ingest workers, the export switch), see [Programmatic interface limits and capacity](/docs/ax/selfhosting/architecture/programmatic-interface-capacity). ## How the interfaces divide Everything below reads from or writes to the same trace store. What differs is the shape of the request each path accepts and what protects the platform on that path. ```mermaid theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} %%{init: {"flowchart": {"nodeSpacing": 40, "rankSpacing": 70, "padding": 16, "curve": "basis"}}}%% flowchart LR subgraph inbound["Inbound: bounded by ingest capacity"] direction TB apps["LLM apps, live OTel traces"] sdklog["client.spans.log(), batch spans and evals"] end subgraph platform["Arize AX"] direction TB ingest["Trace ingest"] --> store["Trace store"] end subgraph control["Control plane: rate-limited"] api["GraphQL API and REST /v2"] end subgraph bulk["Bulk out: no request quotas"] direction TB flight["Arrow Flight export"] fabric["Data Fabric sync"] end apps --> ingest sdklog --> ingest store <--> api store --> flight store --> fabric api <--> clients["Automation, CLI, MCP, SDK clients"] flight --> jobs["Your export jobs: DataFrames, Parquet"] fabric --> bucket["Your bucket: Iceberg or Delta, hourly"] ``` Every client you might reach for sits on one or more of these paths. The table includes the SDKs, the CLI, and the MCP server so you can see which path a given call actually takes. | Interface | Primary use | How it is limited | Bulk-data fit | | :- | :- | :- | :- | | **GraphQL API** | Control plane: monitors, dashboards, projects, annotations, metadata, small reads | Per-key request rate per minute, a query complexity budget, and required pagination | Low, by design | | **REST API `/v2`** | Automation and integration: datasets, experiments, prompts, annotation queues, span and trace lookups | Per-user, per-endpoint request rate on a one-minute window, plus page-size caps | Low, by design | | **Python SDK** (`arize>=8`) | Everything above plus bulk export, bulk ingest, and bulk evaluation updates | REST methods share the REST limits; `export_*`, dataset and experiment downloads, and `update_*` methods use Arrow Flight | High: the only SDK on all three transport paths | | **TypeScript SDK** | Resource management from Node and browser tooling | REST only, so REST limits apply | Low | | **Go SDK** | Resource management from Go services | REST only, so REST limits apply | Low | | **`ax` CLI** | Scripting and CI/CD | REST by default. `spans`, `traces`, `datasets`, and `experiments` `export --all` switch to Arrow Flight | High with `--all`, low without | | **MCP server** | Read-only lookups for coding agents | REST only, pages capped at 100 items | None; it is a lookup surface | | **Arrow Flight export** | Bulk export of spans, traces, datasets, and experiment runs into DataFrames or Parquet | No request quota. Bounded by a 512 MB message size, the ingress stream timeout, and query-layer capacity | High: the designed bulk-read path | | **Data Fabric** | Continuous replication of selected projects into your own bucket as Iceberg or Delta | No volume cap. Paced by the hourly sync schedule and the trace store's segment publishing | Highest: full-volume standing egress | | **Batch span logging** (`client.spans.log()`) | Backfills, offline evaluations, environment replays | Ingest validation and per-upload payload bounds; on SaaS, a per-space limit on upload requests per minute | High, inbound | | **Live OTel ingest** | Tracing a running application | Ingest capacity and your space's ingestion quota | Not a bulk tool; use batch span logging for historical data | If you only want the answer, match the job to a row: 1. **Create, update, or read configuration** (monitors, dashboards, evaluators, prompts, annotations, keys, users): GraphQL or REST, whichever has the operation. Same for a handful of spans from a known trace. 2. **Pull a window of spans, a dataset, or an experiment's runs into a file or DataFrame**: Arrow Flight export, through the Python SDK or `ax ... export --all`. 3. **Keep a warehouse current with trace data**: Data Fabric. 4. **Load spans or evaluations that already exist somewhere else**: batch span logging. 5. **Trace a running application**: live OTel ingest, which is instrumentation rather than an API call. See the [integrations](/docs/ax/integrations). The rest of this page explains the limits behind each row and how to design jobs around them. ## GraphQL API **Use it for** configuration and orchestration: creating and updating monitors, dashboards, custom metrics, evaluators, prompts, and annotations, reading back configuration to version it, and traversing the object graph from a space down to a project. Small reads of record data are fine. Whole projects are not. **How it is limited.** Three mechanisms stack: 1. **Request rate.** Each API key gets a per-minute budget, with separate budgets for queries and mutations. A rejected request does not count against the window, so a backoff-and-retry loop loses nothing. When you exceed the budget, the response is an ordinary GraphQL error, not an HTTP 429, and there is no `Retry-After` header: ```json theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} { "errors": [ { "message": "Too many requests", "extensions": { "code": "TOO_MANY_REQUESTS" } } ] } ``` 2. **Complexity budget.** Every operation is scored before it runs, and the maximum is 1000. A connection field costs its page size (`first` or `last`) multiplied by the complexity of what you select inside it, so nesting multiplies: fifty spaces each selecting fifty monitors is already 2,500 before you add a field. Over budget, the request is rejected with `Query is too complex: . Maximum allowed complexity: 1000`. Measure before you ship by adding the free `rateLimit` field to any operation. It costs nothing against the complexity budget (it still counts as one request): ```graphql theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} query { rateLimit { limit cost } account { spaces(first: 10) { edges { node { id name } } } } } ``` 3. **Required pagination.** Connections reject requests without `first` or `last`, and each connection has a maximum page size, typically 50. When you exceed it, the error names the maximum for that field. For the current limits, see [GraphQL resource limitations](/docs/ax/graphql-reference/overview/resource-limitations). On Arize SaaS the request budgets are set per account; contact [support@arize.com](mailto:support@arize.com) to raise them. Self-hosted deployments set them with one configuration value; see the [capacity page](/docs/ax/selfhosting/architecture/programmatic-interface-capacity). **Why it is limited this way.** GraphQL runs on the same application servers and query layer as the UI. The limits protect interactive responsiveness; they are not a volume meter. A budget of a few hundred requests per minute at 50 rows per page caps out in the low tens of thousands of rows per minute, and raising it adds load to the servers that render dashboards without giving you export bandwidth. **Switch paths when** you are paging a connection to drain it. If the goal is "all spans in a window," use Arrow Flight export; if the goal is "all trace data, continuously," use Data Fabric. Keep GraphQL for the configuration around the data. ## REST API **Use it for** automation and integration: CRUDL on datasets, experiments, prompts, annotation queues, API keys, and users, and for targeted span and trace lookups. The TypeScript SDK, Go SDK, `ax` CLI (without `--all`), MCP server, and the REST methods of the Python SDK are all wrappers over this surface and draw from the same limits. **How it is limited.** Limits are per **user**, so all of that user's API keys share one bucket, and they are tracked per endpoint (route pattern plus method) on a fixed one-minute window. The default is 100 requests per minute; a handful of endpoints are lower. When you exceed a limit the API returns `429 Too Many Requests` with an RFC 9457 problem body and a `Retry-After` header holding the seconds left in the window. There are no `X-RateLimit-*` headers. The authoritative table is in the [REST API rate limits](/docs/ax/rest-reference/overview#rate-limits). ```http theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} HTTP/1.1 429 Too Many Requests Content-Type: application/problem+json Retry-After: 41 { "status": 429, "title": "Too Many Requests", "type": "https://arize.com/docs/ax/rest-reference/errors#rate-limit-exceeded" } ``` The endpoint most people hit a limit on is **List spans**, `POST /v2/spans`. It is a search, not an ingest or export call: 10 requests per minute and at most 500 spans per page, so about 5,000 spans per minute per user. List traces (`POST /v2/traces`) returns at most 50 per page. Both default to the last 7 days when you omit a time range. Out-of-range `limit` values are rejected rather than clamped. **Why it is limited this way.** The REST gateway is a thin layer over several backends. Per-endpoint budgets stop one hot loop from saturating the backend behind one route while leaving every other integration untouched, and span lookups get a tighter budget because they query the trace store directly. Bulk span access was deliberately routed to Arrow Flight instead. On Arize SaaS these limits are fixed; self-hosted deployments adjust them through configuration (see the [capacity page](/docs/ax/selfhosting/architecture/programmatic-interface-capacity)). **Switch paths when** a loop exists only to page through data. Thousands of spans per minute is plenty for "find the spans in this trace" and far too slow for "give me last quarter." The Python SDK's `client.spans.export_to_df()` and the CLI's `ax spans export --all` are the same request on the bulk path. ## Arrow Flight export **Use it for** bulk reads: spans and traces by time window, whole datasets, and experiment runs, returned as a DataFrame or written to Parquet. In the Python SDK this is `client.spans.export_to_df()` and `export_to_parquet()`, `client.datasets.export_to_parquet()` and `list_examples(all=True)`, and `client.experiments.export_to_parquet()` and `list_runs(all=True)`. The `ax` CLI uses it whenever you pass `--all`. Flight streams Arrow record batches straight from the trace store's query layer over gRPC, so there is no JSON encoding and no paging. **How it is limited.** There is no per-request rate limit or quota on Flight; a call is authenticated and checked against your role, then it streams. The guardrails are shaped around scope and duration instead: 1. **Message size.** A single gRPC message is capped at 512 MB on both the server and the SDK. The stream is made of many messages, so this bounds memory per step, not the size of the export. 2. **Stream duration.** A stream has to finish inside the ingress timeout in front of the service: about 2 hours on Arize SaaS and in the self-hosted reference ingress configurations; some regions inherit their load balancer's default. A stream that outlives the timeout fails and restarts from the beginning when you retry; it does not resume. 3. **Time range.** There is no server-side cap on the export window. You choose it, and you should choose it with the stream timeout in mind. 4. **Shared capacity.** Concurrent streams share the query layer with dashboards and monitors. Beyond its capacity, additional streams slow everyone down rather than speed anything up. > [!NOTE] > > The 31-day windows you may see on other span operations (the REST annotate endpoint, and the Flight-backed `update_evaluations`, `update_annotations`, and `update_metadata` lookups) do not apply to export. Export windows are whatever start and end times you pass. **Why it is limited this way.** Bulk reads are bounded by real capacity rather than an artificial meter. A rate limit on Flight would only turn a two-minute export into a twenty-minute one without reducing the work the trace store does. Capacity is the lever, and on self-hosted clusters it is yours to size. **Switch paths when** the same export runs on a schedule against the same projects. A nightly Flight job that re-pulls yesterday is a replication feed implemented by hand; Data Fabric is that feed without the keys, scheduling, or retries. Go the other way, from REST to Flight, as soon as a span query needs more than one or two pages. ## Data Fabric **Use it for** standing replication. Data Fabric syncs the projects you select into a bucket you own (S3, GCS, or Azure Blob Storage) as Iceberg (the default) or Delta tables, ready to query from BigQuery, Snowflake, Databricks, and anything else that reads open table formats. Updates to historical rows, such as evaluations and annotations added weeks later, arrive on the next sync. You manage connectors in **Settings > Data Fabric** or through the GraphQL API; there is no REST surface. See [Sync traces with Data Fabric](/docs/ax/security-and-settings/data-fabric) for setup. **How it is limited.** It is not, in the volume sense. There is no cap on what Data Fabric replicates, and it copies everything rather than sampling. What paces it is the sync schedule, hourly by default, which runs as the trace store publishes storage segments. Treat it as a replication feed with roughly hourly freshness, not as a low-latency API. Data Fabric is an enterprise feature on Arize SaaS; on self-hosted clusters the operator enables it. **Why it is limited this way.** The destination is storage you own, so the only sensible goal is a complete and current copy. Pacing on the segment cadence keeps the sync cheap for the trace store (it copies finished segments instead of re-querying live data). **Trade-offs to plan for.** Storage cost moves to your bucket and grows with your trace volume, so set lifecycle policies there. Cross-cloud egress applies if your bucket is in a different provider from your AX region. And a backfill is also a sync: 50 million rows logged into a replicated project are 50 million rows landing in your warehouse on the next sync. **Switch paths when** you need a one-off window in a DataFrame right now (Arrow Flight), or a few hundred spans from a specific trace (REST or the MCP server). Data Fabric is for data that should simply already be in your lake. ## Batch span logging **Use it for** moving historical or offline data in: backfilling traces from another system, logging the results of an offline evaluation run, or replaying an environment into a fresh project. The Python SDK call is `client.spans.log(space_id=..., project_name=..., dataframe=spans_df, evals_dataframe=evals_df)`, where the optional evaluations DataFrame is joined to spans on `context.span_id`. It is Python only. TypeScript teams rarely notice: live tracing from JavaScript is fully supported through the [OpenInference instrumentations](/docs/ax/integrations), and backfills and offline evaluations tend to run in Python pipelines anyway. **How it is limited.** Each call writes the whole DataFrame as one Arrow file and uploads it in a single HTTP request, which the ingest pipeline then processes asynchronously. Several bounds apply: 1. **Client-side validation runs first**, so malformed columns or oversized values fail locally before anything is uploaded. Pass `validate=False` only when you have already validated upstream. 2. **The server caps one upload** at 40 million rows or 20 GiB. Your job's chunking, not the SDK, decides how much memory a call uses and how much work a failed call throws away. 3. **Per-span limits** on attribute sizes are the same ones live tracing enforces, because this is the same pipeline. 4. **The call raises on the first non-2xx response**, including `429`, and does not retry. A `429` means you submitted too many upload requests, not too many rows. 5. **On Arize SaaS, a per-space limit on upload requests applies.** It bounds how many `log()` calls you make per minute, not how many rows each carries, so submit fewer, larger DataFrames rather than many small ones. Self-hosted deployments are bounded by ingest capacity instead. **Why it is limited this way.** Batch logging is ingest, governed like live traces. Anything you backfill is indexed, retained, counted against your space's [ingestion quota](/docs/ax/security-and-settings/space-rate-limiting), and replicated by Data Fabric exactly as live data would be. **Switch paths when** the data is still being produced: instrument the application and let live tracing carry it. Batch logging is for data that already exists somewhere else. ## Design your jobs Two principles cover most of the client-side design, whichever path you are on. **The trace store is time-partitioned, so chunk by time.** Exports scoped to a window touch only the segments in that window, and backfills submitted as time-contiguous batches land in fewer segments and index faster. Time is the efficient axis in both directions; filtering by anything else happens after the partition is already read. **Bound the unit of work so retries are cheap.** Prefer many modest, rerunnable units over one giant operation. A window that fails costs you that window. A stream that fails at hour two costs you two hours. ### GraphQL 1. Measure cost with the `rateLimit` field while you develop the query, and keep it well under 1000 so a growing page does not push you over later. 2. Prefer flat queries at the maximum page size over deep nesting. Two cheap queries beat one that multiplies page sizes together. 3. Rejected requests do not count against the window, so a simple sleep-and-retry on `TOO_MANY_REQUESTS` loses nothing. 4. Spread sustained jobs evenly across the minute instead of bursting at the top of it. 5. Queries and mutations have separate budgets, so a workflow that reads then writes does not contend with itself. ### REST 1. Limits are per endpoint, so a pipeline that touches several endpoints only throttles on its hot loop. Spread work across resources where the data model allows. 2. Use the maximum page size: 500 on data endpoints such as List spans, 100 on metadata endpoints. Fewer requests for the same rows. 3. Pass explicit `start_time` and `end_time` instead of paging the 7-day default lookback. 4. Honor `Retry-After` literally. It is the number of seconds until the window resets, so sleeping for exactly that value is both the fastest and the politest option. ```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} import time import requests def request_with_backoff(method, url, *, headers, max_attempts=5, **kwargs): """Call a REST /v2 endpoint and wait out any 429 using Retry-After.""" for _ in range(max_attempts): response = requests.request(method, url, headers=headers, **kwargs) if response.status_code != 429: response.raise_for_status() return response # Retry-After is the remaining window in whole seconds (omitted if under 1s) time.sleep(int(response.headers.get("Retry-After", "1"))) raise RuntimeError(f"{method} {url} still rate limited after {max_attempts} attempts") ``` ### Arrow Flight export 1. Chunk large exports by time window and keep each window comfortably inside the stream timeout. The retry cost is one window, and windows run in parallel if you need them to. 2. Run a small number of concurrent windowed streams. Beyond the query layer's capacity, extra streams slow everyone down rather than speed anything up. 3. Push filters into the request: time range and project are what the trace store partitions on. Do not export a month and filter to a day in pandas. 4. Schedule heavy exports off-peak. The query layer also serves dashboards and monitors. 5. Make output idempotent: one file per window at a deterministic path, so a rerun skips finished windows and only redoes the failed one. ```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} from datetime import datetime, timedelta from pathlib import Path from arize import ArizeClient client = ArizeClient(api_key="your-api-key") out_dir = Path("./spans") out_dir.mkdir(exist_ok=True) start = datetime(2026, 7, 1) end = datetime(2026, 10, 1) window = timedelta(days=1) cursor = start while cursor < end: window_end = min(cursor + window, end) target = out_dir / f"spans_{cursor:%Y-%m-%d}.parquet" if not target.exists(): # idempotent: a rerun skips finished windows df = client.spans.export_to_df( space_id="your-space-id", project_name="my-llm-app", start_time=cursor, end_time=window_end, ) df.to_parquet(target, index=False) cursor = window_end ``` `client.spans.export_to_parquet(..., path=...)` writes the file for you if you do not need to transform the DataFrame first. The CLI equivalent is `ax spans export my-project --all --space --start-time ... --end-time ...` in a shell loop. ### Batch span logging 1. Chunk the backfill into bounded submissions, hundreds of thousands to a few million rows each. Each call is one upload, so the chunk size sets both your client's memory footprint and the cost of a mid-job failure. 2. Submit time-contiguous chunks in time order. Sorting by `start_time` before chunking is usually enough. 3. Validate a small sample first with a few hundred rows so schema problems surface before the long run. 4. Pace the job over hours or off-peak, and back off when a call raises on `429`. A few concurrent senders are fine; beyond ingest capacity, more parallelism does not make the job finish sooner. 5. Budget for the downstream effects: retention, your space's ingestion quota, and Data Fabric replication all see the backfilled rows. ```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} import time from pathlib import Path import pandas as pd from arize import ArizeClient client = ArizeClient(api_key="your-api-key") # One Parquet file per day, produced upstream and sorted by start_time. for path in sorted(Path("./backfill").glob("spans_*.parquet")): chunk = pd.read_parquet(path) for attempt in range(5): try: response = client.spans.log( space_id="your-space-id", project_name="my-llm-app", dataframe=chunk, ) print(f"{path.name}: {len(chunk)} rows, HTTP {response.status_code}") break except Exception as exc: # log() raises on any non-2xx, including 429 # 401/403 and client-side validation errors will not succeed on retry if "authentication" in str(exc).lower(): raise print(f"{path.name}: attempt {attempt + 1} failed: {exc}") time.sleep(30 * (attempt + 1)) else: raise RuntimeError(f"gave up on {path.name}") ``` ### Data Fabric There is nothing to optimize on the client. The design move is recognizing when a job should be a connector. If you have a scheduled Flight job that re-pulls the same projects every day, replace it with a Data Fabric connector for those projects and point your warehouse at the tables. ## Scenarios **Export three months of spans for offline analysis.** At roughly 5,000 spans per minute, List spans would need more than 33 hours of continuous paging for 10 million spans. Use **Arrow Flight export**: loop over daily windows with `client.spans.export_to_df()` or `export_to_parquet()`, one file per day, and rerun only the days that failed. If the analysis will repeat every month, see the next scenario. **Nightly sync into the warehouse.** A cron job that exports yesterday's spans and loads them into BigQuery is a hand-built replication feed with API keys to rotate and retries to manage. Use **Data Fabric**: create a connector for the projects, register the Iceberg tables, and query them. Evaluations and annotations added later arrive on the next hourly sync without any change to the job. **Backfill 50 million rows from another system.** Use **batch span logging**, chunked by day and submitted in time order. Each `client.spans.log()` call is one upload, so a chunk of a few hundred thousand to a few million rows keeps memory flat and makes a failed call cheap to redo. On SaaS, fewer and larger chunks also stay clear of the per-space upload request limit. Confirm the space's ingestion quota covers the volume, and remember that any Data Fabric connector on the project will replicate all 50 million rows. **A TypeScript team needs bulk span data.** The TypeScript SDK is REST only, so `listSpans` is capped at 500 per page and 10 requests per minute. Use **Data Fabric** if the data belongs in a warehouse the team can query. For a one-off pull, run `ax spans export --all --space ` from a shell, or a short Python job on Arrow Flight; either produces files the TypeScript code can read. **A coding agent needs the spans from one failing trace.** This is a lookup, and the control plane is the right place for it. Use the **MCP server** or **REST** `POST /v2/spans` filtered to the trace ID. One page is usually enough, and 10 requests per minute is plenty for interactive debugging. The same agent paging a busy project's full span list is the pattern this page exists to redirect. **Keep monitors in sync across spaces.** This is configuration, so use the **GraphQL API**. Read each space's monitors at the maximum page size, check cost with `rateLimit { limit cost }` while developing the query, write the differences with mutations (they have their own budget), and sleep-and-retry on `TOO_MANY_REQUESTS`. If you need the underlying metric data rather than the monitor definitions, that is a Flight export. > [!TIP] > > Rate limits guard the control plane and are configuration. Data volume belongs on the paths that scale with provisioned capacity. When a job is slow, the useful question is rarely "can we raise the API limit" and usually "is this job on the right path, and is it chunked so a retry is cheap." > **[REST API reference](/docs/ax/rest-reference/overview#rate-limits)** > > Rate limits, pagination, error format, and the full endpoint reference. > **[GraphQL API reference](/docs/ax/graphql-reference)** > > Resource limitations, the object graph, and every query and mutation. > **[Python SDK: spans client](/docs/api-clients/python/version-8/client-resources/spans)** > > `export_to_df`, `export_to_parquet`, `log`, and the evaluation update methods. > **[Data Fabric](/docs/ax/security-and-settings/data-fabric)** > > Set up a connector and register Iceberg or Delta tables in your warehouse. > **[Programmatic interface limits and capacity](/docs/ax/selfhosting/architecture/programmatic-interface-capacity)** > > Self-hosted sizing and configuration behind each path.