Skip to main content
Tracing is how you see inside your LLM app. Without it, debugging is guesswork. With it, every call, every tool invocation, every retrieval is visible.

​
How Tracing Works

Your LLM application handles requests — each one might call a model, retrieve documents, run tools, and return a response. A trace captures that entire journey as a tree of spans. Each span represents one operation (an LLM call, a retrieval, a tool invocation) with its input, output, timing, and metadata.
Under the hood, this is powered by two open standards:
  • OpenTelemetry (OTel) — the universal, vendor-agnostic framework for collecting and exporting telemetry data. Your instrumentation isn’t locked to Arize AX — it works with any OTel-compatible backend.
  • OpenInference — GenAI-specific semantic conventions built on top of OTel, created by Arize. OpenInference defines the attributes that make traces meaningful for LLM applications: span kinds, message formats, token counts, cost, and more. Arize accepts standard OTel spans and reads OpenInference attributes for richer GenAI visualization.
Together, they give you 30+ native integrations, portability across backends, and a standardized format for your trace data. Here’s how it all connects:
  1. Instrumentation wraps your function calls (automatically via integrations, or manually) and captures span data following OpenInference semantic conventions.
  2. An exporter sends the spans to Arize AX via OTLP (gRPC by default).
  3. The Arize collector ingests and visualizes them so you can explore, filter, and debug.
How tracing works: from instrumentation to Arize AX UI

​
What’s Inside a Trace

Now that you know how data flows from your app to Arize AX, let’s look at what a trace contains. Each span in the tree is assigned a span kind — the type of operation it represents: If you don’t set a span kind, it defaults to Unknown. Here’s what this looks like in practice — an agent trace with Agent, Retriever, Tool, and LLM spans forming a tree:
Agent trace showing multiple span kinds in a trace tree

​
Span Attributes

Each span also carries attributes — key-value pairs like llm.model_name, llm.input_messages, input.value, output.value. These are the OpenInference semantic conventions — auto-instrumentation sets them for you, and you can add your own via Customize your traces. Check the OpenInference source for every attribute defined per span kind:

Python Semantic Conventions

TS Semantic Conventions

​
Next step

You know how tracing works and what’s inside a trace. Now get traces flowing from your app:

Next: Auto-Instrumentation