OpenInference is an open specification of semantic conventions for tracing LLM and agent applications, along with a set of instrumentation libraries that emit traces following those conventions. It defines what a span representing a model call, a retrieval step, or a tool invocation should be named and which attributes it should carry, so that a trace produced by one framework can be read by tooling that has never heard of that framework. It is maintained as open source by Arize.
The problem it addresses is boring and expensive. One library records the model as model, another as model_name, a third buries it inside a serialized JSON blob. One emits token counts, another does not. Nothing downstream can reliably answer “what did the model see and what did it return” without a per-framework adapter. Semantic conventions fix that by agreeing on names once, so instrumentation and analysis can be built independently of each other.
Key takeaways
- OpenInference is a set of OpenTelemetry semantic conventions plus instrumentation libraries for AI applications. It is a layer on top of OpenTelemetry, not a replacement for it.
- Its central idea is the span kind: an attribute such as
openinference.span.kindthat marks a span as an LLM call, a retriever, a tool, an embedding step, or an agent, so tooling knows how to interpret the payload. - Because the output is ordinary OpenTelemetry spans exported over OTLP, any backend that speaks OTLP can receive them.
- Conventions are what make evaluation possible at scale. A judge or a code check needs to find the input, the output, and the retrieved documents in a predictable place.
- Conventions do not replace judgment about what to record. Payload size, sampling, and sensitive data are still decisions you have to make.
How it relates to OpenTelemetry
OpenTelemetry gives you the parts that are not specific to AI: the tracing API, the SDKs, context propagation across function and service boundaries, batching and exporting, and the OTLP wire protocol. If you are new to that vocabulary, the primer on spans, traces, and sessions in OpenTelemetry covers the model that everything else assumes.
What general-purpose tracing does not tell you is what a “prompt” is, or how to record a list of retrieved documents, or where the completion tokens go. That is the gap semantic conventions fill. OpenInference supplies conventions for those AI-specific concepts and ships instrumentors that attach to popular frameworks and model clients so that the spans get created without hand-writing them.
The OpenTelemetry project has also been developing its own GenAI semantic conventions. If you are deciding what to emit today, the practical question is which conventions your instrumentation produces and whether the backend you send them to can read them.
What the conventions actually describe
The specification is organized around span kinds and the attributes each kind carries. The span kinds cover the components AI applications are built from: an LLM span for a model call, a RETRIEVER span for a lookup against a vector store or search index, a TOOL span for a function the model chose to invoke, an EMBEDDING span for vector creation, a CHAIN span for a logical unit of work that groups children, and an AGENT span for a run that decides its own control flow.
Attributes are then namespaced by concern. Input and output values live in a predictable place regardless of span kind. Model calls carry the model name, the invocation parameters, the structured input messages, and token counts split by prompt and completion. Retriever spans carry the documents that came back, with their content and scores. Tool spans carry the tool name and the arguments that were passed.
Two attributes matter more than their size suggests. Session and user identifiers turn a pile of unrelated traces into conversations you can evaluate as a unit, which is the difference between scoring a single response and scoring whether the user got what they came for. And the span kind itself is what lets an evaluator run automatically: a retrieval relevance check knows to look at RETRIEVER spans without being told where they are.
Why standardized traces change what you can build
Once spans are shaped consistently, a lot of tooling stops being bespoke.
Evaluation can be attached to spans. A judge that scores retrieval relevance or answer faithfulness needs to locate the query, the documents, and the response. With conventions it finds them by attribute name. The full loop is described in the guide to AI agent tracing and evaluation.
Framework changes stop breaking your telemetry. Because instrumentation targets the conventions rather than your application code, swapping an orchestration library changes which instrumentor you install, not what your dashboards can display.
Non-obvious systems become traceable. Anything that makes model calls can emit conforming spans, including coding agents. Instrumenting a terminal-based coding agent uses the same conventions as a customer support bot.
Agent harnesses become inspectable. Understanding why a long-running agent went wrong depends on knowing which step was a tool call and which was a reasoning step, which is the structure discussed in agent harness evaluation and tracing.
Where instrumentation still goes wrong
Conventions cannot save you from partial coverage. If a service calls the model through a wrapper the instrumentor does not recognize, that call produces no span, and the trace has a hole exactly where the interesting thing happened. Manual spans are the fix, and they need the span kind set explicitly.
Context propagation is the second recurring problem. Spans created inside a thread pool, a background task, or a separate service will attach to the wrong parent, or to no parent, unless context is carried across the boundary. The symptom is a flat list of traces where you expected a tree.
The third is payload discipline. Recording full inputs and outputs is what makes traces useful for evaluation and also what puts customer text into your telemetry pipeline. Decide deliberately what gets captured, what gets redacted, and what gets sampled.
FAQ
Is OpenInference the same thing as OpenTelemetry?
No. OpenTelemetry is the tracing framework: API, SDK, context propagation, and the OTLP protocol. OpenInference is a set of semantic conventions for AI-specific spans plus instrumentation that emits them. OpenInference spans are OpenTelemetry spans.
Do I have to use Arize to use OpenInference?
No. The specification and the instrumentation libraries are open source, and the traces they produce are standard OTLP. You can send them to any collector or backend that accepts OTLP.
What does an OpenInference trace capture that a normal trace does not?
The AI-specific payload: the prompt and completion, structured input messages, the model name and invocation parameters, token counts, retrieved documents with their scores, tool names and arguments, and the span kind that says how to interpret all of it.
Can I mix auto-instrumentation with my own spans?
Yes, and most production setups do. Auto-instrumentation covers the framework and model client calls. Manual spans cover your business logic, custom retrieval, and anything the instrumentors do not see. Both end up in the same trace as long as context propagates correctly.
Why do span kinds matter so much?
Because they are what makes automated analysis possible. Without a span kind, a trace is a set of timed operations with opaque payloads. With one, tooling can find every retrieval step in a million traces and score it, or count tool calls per session, without knowing anything about the application that produced them.
