Skip to main content

​
Anatomy of a Trace

A trace represents a single request, which is made up of multiple spans. A trace records the paths taken by requests as they propagate through multiple steps. Traces make it easy to debug issues and understand the flow of your application.

​
Spans

A span represents a unit of work, with an input, output, start, and end time. It tracks specific operations such as a retriever, tool call, or LLM step. Here is an example of an LLM span in JSON form below, with attributes such as the llm input messages and output value. 
{
    "name": "llm",
    "context": {
        "trace_id": "ed7b336d-e71a-46f0-a334-5f2e87cb6cfc",
        "span_id": "ad67332a-38bd-428e-9f62-538ba2fa90d4"
    },
    "span_kind": "LLM",
    "parent_id": "f89ebb7c-10f6-4bf8-8a74-57324d2556ef",
    "start_time": "2023-09-07T12:54:47.597121-06:00",
    "end_time": "2023-09-07T12:54:49.321811-06:00",
    "status_code": "OK",
    "status_message": "",
    "attributes": {
        "llm.input_messages": [
            {
                "message.role": "system",
                "message.content": "You are an expert Q&A system that is trusted around the world.\nAlways answer the query using the provided context information, and not prior knowledge.\nSome rules to follow:\n1. Never directly reference the given context in your answer.\n2. Avoid statements like 'Based on the context, ...' or 'The context information ...' or anything along those lines."
            },
            {
                "message.role": "user",
                "message.content": "Hello?"
            }
        ],
        "output.value": "assistant: Yes I am here",
        "output.mime_type": "text/plain"
    },
    "events": [],
}

​
Span Kinds

Every span has a type, which signifies the span kinds. In OpenInference, span kinds classify spans based on their role. But instead of just focusing on network calls or databases, OpenInference adapts the concept for LLMs.
Span KindDescription
LLMCall to an LLM for a completion or chat
ChainThe starting point and link between application steps
ToolAPI or function invoked on behalf of an LLM
AgentRoot span which contains a set of LLM and tool invocations
EmbeddingEncoding of unstructured data
RetrieverData retrieval query for context from a datastore
RerankerRelevance based re-ordering of documents

​
Span Attributes

Attributes are key-value pairs that contain metadata that you can use to annotate a span to carry information about the operation it is tracking. For example, if a span invokes an LLM, you can capture the model name, the invocation parameters, the token count, and so on. Attributes have the following rules:
  • Keys must be non-null string values
  • Values must be a non-null string, boolean, floating point value, integer, or an array of these values Additionally, there are Semantic Attributes, which are known naming conventions for metadata that is typically present in common operations. It’s helpful to use semantic attribute naming wherever possible so that common kinds of metadata are standardized across systems. See semantic conventions for more information.
📚 Learn more about LLM Tracing.

​
How does tracing work?

Image
There are five key components within tracing - Instrumentation, Span Processor, Exporter, Protocol, and Collector. We’ve built all of these within our tracing tools to make it easy to troubleshoot your application and surface issues.

​
Instrumentation

In order for an application to emit traces for analysis, the application must be instrumented. Your application can be manually or automatically instrumented. Our auto-instrumentation plugins will collect spans for you with just a few lines of code. All of these instrumentors are managed via a single repository called OpenInference. Instrumentation works by automatically detecting specific function calls and logging their attributes to span properties, such as input variables, outputs, latency, and more. Each auto-instrumentor is built specifically to map to the provider’s SDK, such as OpenAI, Bedrock, LlamaIndex, and LangChain.

​
SpanProcessor

Span processors dictate how the spans should be processed during the execution of a request, and whether it should be blocking or not. There are 2 main span processors:
  • Batch Processor - Used for product deployments, this is non-blocking for your application. We default to batch when we register your instrumentation to reduce latency.
  • SimpleSpanProcessor - This is a synchronous logging instrumentation where the log call is blocking, delays in collector will back up into your application. Used for debugging setups.
Read more about how this is setup on our arize-otel page.

​
Exporter

An exporter takes the spans created via instrumentation and exports them to a collector. In simple terms, it sends the data to an Arize URL with a specified format. The exporter used for Arize is the GRPC exporter.

​
OpenTelemetry Protocol

OpenTelemetetry Protocol (or OTLP for short) is the means by which traces arrive from your application to the Arize AX collector. Arize currently supports OTLP over GRPC.

​
Collector

The Arize AX server is a collector and a UI that helps you troubleshoot your application in real time. Arize AX receives your logs and visualizes them on our dashboard. We use many technologies under the hood to ensure we can support millions of traces and return you aggregations and insights on your data. We receive spans at https://otlp.arize.com/v1 in the US and https://otlp.eu-west-1a.arize.com/v1 in Europe.