adb in Arize AX

Every trace you send to Arize AX, every evaluation you run, and every annotation you add lands in the same place: adb, Arize’s proprietary, AI-native online analytical processing (OLAP) database. It’s the engine behind every managed Arize AX instance, processing terabytes of data daily and billions of spans and traces monthly, at petabyte scale with sub-second latency.

​

Why we built adb

AI observability isn’t just collecting logs and traces. It’s accelerating the entire AI development lifecycle. That means fast ad-hoc analytics over high-volume generative AI data, fresh events available within seconds, and mass updates when you re-run evaluations on historical traces. Traditional databases weren’t built for this combination at petabyte scale, so Arize built adb to serve all of it at once instead of running on an off-the-shelf solution. adb keeps your data in open formats, that same data can flow directly to your own cloud warehouse through Data Fabric, which continuously syncs your traces, evaluations, and annotations to a warehouse like BigQuery or Snowflake in open Iceberg format. The guiding principle is an anti-monolith approach: instead of one all-in-one system, adb is a universal data layer built on open standard formats, with lightweight, stateless query engines sitting on top of shared object storage. Your generative data is the core asset, accessible to any tool, rather than something locked inside a proprietary engine.

​

adb’s design goals

adb’s architecture is shaped around five goals:

1. Core data asset. Keep generative outputs in open formats (Parquet, Iceberg) so they’re accessible to any tool, with no vendor lock-in.
2. Stateless, elastic compute. Separate storage from compute so query engines can scale up and down, and move, without relocating or losing data.
3. Sub-second performance. Maintain spreadsheet-like responsiveness even at billion-row scale.
4. Continuous real-time updates. Stream fresh events in without blocking historical backfills, and capture evaluations and annotations on data of any age.
5. Internet-scale durability. Reliably process billions of spans and traces per month across hundreds of customers.

​

Learn more

For a deeper look at how adb works, refer to the following topics:

• Architecture: storage/compute separation, the streaming and batch write paths, Apache Flight, compaction, and cache management.
• Real-time ingestion: how events become queryable within seconds, the file formats involved, and adb’s delivery guarantees.
• Performance & benchmarks: how adb performs on dataset uploads, trace ingestion, and full-text search compared to other platforms.

If you self-host Arize AX, adb runs as the “ArizeDB” component in your cluster. See the self-hosting prerequisites for its node-pool and storage requirements.