What Is Distributed Tracing?

Distributed tracing is the practice of following one request as it crosses process and service boundaries, by propagating a shared identifier alongside each call so that work done in one service can be attached to work done in another. The output is a trace whose spans came from several processes instead of one.

The distinguishing part is the propagation. Inside one process, trace context lives in a thread local or an async context variable and every span finds its parent for free. Once a call leaves the process, that context has to travel with it, normally as an HTTP header, and both sides have to agree on the format. That agreement is what makes the discipline distributed rather than local.

For AI systems this is a correctness concern more than an infrastructure one. An agent calling a retrieval service, a payments API, and two separately deployed subagents is the multi-process request distributed tracing was invented for. Without propagation you get five disconnected traces and no way to prove which retrieval fed which answer.

Key takeaways

• Distributed tracing is a trace plus context propagation. The trace is the artifact; propagation across process boundaries makes it whole.
• Context travels as a trace ID and parent span ID in a standard header such as W3C traceparent, injected by the caller and extracted by the callee.
• Sampling is the cost control, and the head-based or tail-based choice decides whether you still have the trace you needed after an incident.
• The common defect is context dropped at an async handoff or a queue, which starts a fresh trace and leaves orphan spans.
• Multi-service AI systems are the case this was built for: one question can touch a retriever, several tools, and subagents in different processes.

Context propagation is the mechanism

Per-service logging works until a request touches more than one service. After that, correlating by timestamp across several log streams stops working, especially with a retry in the middle. Propagation replaces the guesswork with an identifier generated once and carried everywhere. No service needs to know the topology. It only has to pass along what it received.

The context itself is small: a trace ID, the ID of the span making the outbound call, and flags including whether this trace is sampled. On every hop:

1. The caller injects the current context into the carrier, normally the outbound request headers.
2. The transport carries it. Over HTTP that is the W3C Trace Context traceparent header, with tracestate for vendor additions. Messaging systems use message headers.
3. The callee extracts it and starts its spans as children of the caller’s span, reusing the trace ID rather than generating one.
4. Both services export spans to a collector, which stitches them into one trace by trace ID and parent reference.

Propagation is transitive, so a service that forwards context correctly makes everything behind it reachable in the same trace. The converse is why one uninstrumented proxy can hide half a system: a single dropped header breaks the chain for everything downstream.

Libraries handle injection and extraction for common frameworks and clients, using the same span, trace, and session model on both sides. What needs attention are the boundaries they do not cover: queue producers and consumers, background jobs, thread pools, and anything handing work to a scheduler.

Sampling

Keeping every span from every request is often more data than the application itself generates, and there are two broad ways to cut it down.

Head-based sampling decides at the root span, before the request runs, usually by probability. The decision travels in the propagated flags so every service makes the same choice and traces stay complete. Cheap and predictable, and it discards the rare failing request as readily as a boring successful one.

Tail-based sampling buffers spans until the trace finishes, then decides from what happened: keep it if it errored, was slow, or took an unusual number of steps. Better traces, at the cost of collector memory and a requirement that every span reach the same collector.

For AI applications the failures that matter do not error, so error-based tail sampling misses them: a wrong answer looks like a normal, fast, successful trace. Teams commonly keep a high or complete sample of LLM spans and sample surrounding infrastructure harder. Since those spans hold prompts and completions, that is also a retention and privacy decision, worth making deliberately for an agent system whose spans carry user content.

Why agent systems need it

A single-process agent calling a provider API is barely distributed. Real deployments leave that shape quickly:

• Tools as services. Each tool is a call to something you or someone else operates. Propagated context turns a 500 from that tool into a span in the agent’s trace rather than a mystery.
• Subagents deployed separately. A supervisor delegates over the network. Without propagation its trace ends at the delegation and each worker’s run is an orphan.
• Queues and async work. An agent that enqueues a long task needs context in the message, or the completion arrives unlinked to the request that asked for it.
• Model gateways. A gateway fronting several providers is a hop. Strip headers there and provider-side latency sits outside the trace.

The payoff is attribution. When a run takes 40 seconds, the trace says whether the model was slow, the retriever was slow, or a tool retried behind a network boundary. That is what an agent harness has to answer about its own control flow, and the answer only exists if spans from every process landed in one tree.

Where it breaks

• Broken context at an async boundary. A thread, task queue, or fire-and-forget coroutine starts without the parent context and gets a new trace ID. The symptom is a shallow trace plus one-span traces nobody can explain.
• Missing propagation in one hop. A custom HTTP client bypassing the instrumented one, or an ingress filtering unknown headers. Everything downstream disconnects there.
• Clock skew. Timestamps come from the host that produced the span, so hosts disagreeing by tens of milliseconds make a child appear to start before its parent.
• Independent sampling. A service ignoring the propagated sample flag leaves holes in the trace, and a gap looks like work that never happened.

FAQ

What is the difference between tracing and distributed tracing?

A trace is the record of one request or run, made of spans. Distributed tracing is the practice of producing that record when the work crosses process boundaries, which requires propagating context between them. In a single-process application you get traces without thinking about propagation. Once there are two processes you have a distributed tracing problem, whether or not you have solved it.

How does trace context actually cross a service boundary?

The caller writes the trace ID, its own span ID, and the sample flag into a header on the outbound request, conventionally traceparent. The receiver reads that header and parents its spans to the span ID it received, keeping the same trace ID. Libraries do the writing and reading for supported clients and servers, so most of the work is making sure nothing in between strips the header.

Does an LLM application need this if it runs in one process?

You need tracing regardless, and the distributed part as soon as a second process is involved, which happens earlier than teams expect: a separate vector database, a tool implemented as an internal API, a worker running evaluations, a subagent on its own deployment. Using a standard propagation format from the start costs almost nothing and avoids retrofitting later.

Why do traces arrive with orphan or missing spans?

Almost always a propagation gap or a sampling mismatch. The distinction matters: a propagation gap means the spans exist under a different trace ID, while a sampling mismatch means they were never exported.