> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Your First Traces

> Instrument your LLM application and capture complete execution traces

## Building Visibility Into SupportBot

You're building a customer support agent called SupportBot. It classifies incoming queries, routes them to the right handler, and responds appropriately. But when something goes wrong, you're left wondering: *Why did the agent choose that tool instead of this one? What context was actually passed to the LLM when it generated that response?*

Without observability, debugging LLM applications feels like flying blind. In this chapter, you'll learn how to instrument SupportBot with Arize AX to capture complete execution traces that answer these questions.

## What We're Building

SupportBot has three core functions:

1. **Classify** incoming queries (order status vs. FAQ)
2. **Route** to the appropriate handler
3. **Execute** either database lookups or RAG-based knowledge searches

By the end of this chapter, you'll have complete visibility into every step of this process.

> [card] [Follow with Complete Python Notebook](https://github.com/Arize-ai/tutorials/blob/main/python/cookbooks/arize_ax_tracing_tutorial.ipynb)

## Prerequisites

Before starting, make sure you have:

* An Arize AX account (sign up for free at [app.arize.com](https://app.arize.com))
* Your Space ID and API Key (found in Settings → API Keys)
* OpenAI API key (or another supported LLM provider)

## Step 1: Install Dependencies

First, install the necessary packages for tracing:

```bash theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
pip install openinference-instrumentation-openai openai arize-otel
```

## Step 2: Configure Tracing

Next, set up the connection to Arize AX. This is where all your traces will be sent.

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
from arize.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor
import openai
from opentelemetry import trace

# Connect to Arize AX
tracer_provider = register(
    space_id="your-space-id",  # From Settings → API Keys
    api_key="your-api-key",    # From Settings → API Keys
    project_name="supportbot-tutorial",
)

# Instrument OpenAI to automatically capture traces
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

# Initialize OpenAI client and tracer
openai_client = openai.OpenAI()
tracer = trace.get_tracer(__name__)
```

> [!TIP]
>
> **That's it!** With just these few lines, OpenInference will automatically
> capture traces for all your OpenAI calls. The instrumentation handles all the
> OpenTelemetry boilerplate for you.

## Step 3: Trace Your First LLM Call

Let's start simple. Here's how to classify a user query:

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
def classify_query(user_message: str) -> str:
    """Classify if the query is about order status or a general FAQ."""

    with tracer.start_as_current_span("classify-query") as span:
        span.set_attribute("openinference.span.kind", "CHAIN")
        span.set_attribute("input.value", user_message)

        response = openai_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {
                    "role": "system",
                    "content": "You are a classifier. Respond with either 'ORDER_STATUS' or 'FAQ'."
                },
                {"role": "user", "content": user_message}
            ],
        )

        classification = response.choices[0].message.content.strip()
        span.set_attribute("output.value", classification)

        return classification

# Try it out
result = classify_query("Where is my order #12345?")
print(f"Classification: {result}")
```

### What Just Happened?

1. **Created a parent span** named "classify-query" that groups related operations
2. **Automatically captured** the OpenAI call details (the instrumentation did this)
3. **Set OpenInference attributes** to help Arize understand the span type and data
4. **Sent everything to Arize AX** for visualization

Head to your Arize AX dashboard, and you'll see this trace appear in real-time!

**Ask Alyx** what happened in this trace.

## Step 4: Add Tool Call Tracing

Now let's add the ability to look up order status. We'll use OpenAI's function calling:

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
import json

def get_order_status(order_id: str) -> dict:
    """Simulate looking up an order in a database."""
    return {
        "order_id": order_id,
        "status": "In Transit",
        "estimated_delivery": "2024-03-15"
    }

def handle_order_query(user_message: str) -> str:
    """Handle order status queries using tool calling."""

    with tracer.start_as_current_span("handle-order-query") as span:
        span.set_attribute("openinference.span.kind", "CHAIN")
        span.set_attribute("input.value", user_message)

        tools = [
            {
                "type": "function",
                "function": {
                    "name": "get_order_status",
                    "description": "Look up the status of a customer order",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "order_id": {
                                "type": "string",
                                "description": "The order ID (e.g., '12345')"
                            }
                        },
                        "required": ["order_id"]
                    }
                }
            }
        ]

        response = openai_client.chat.completions.create(
            model="gpt-4",
            messages=[{"role": "user", "content": user_message}],
            tools=tools,
        )

        # Check if the model wants to call a tool
        if response.choices[0].message.tool_calls:
            tool_call = response.choices[0].message.tool_calls[0]

            # Execute the tool with tracing
            with tracer.start_as_current_span("execute-tool") as tool_span:
                tool_span.set_attribute("openinference.span.kind", "TOOL")
                tool_span.set_attribute("tool.name", tool_call.function.name)
                tool_span.set_attribute("tool.parameters", tool_call.function.arguments)

                args = json.loads(tool_call.function.arguments)
                result = get_order_status(args["order_id"])

                tool_span.set_attribute("tool.result", json.dumps(result))

            # Generate final response with tool result
            final_response = openai_client.chat.completions.create(
                model="gpt-4",
                messages=[
                    {"role": "user", "content": user_message},
                    response.choices[0].message,
                    {
                        "role": "tool",
                        "tool_call_id": tool_call.id,
                        "content": json.dumps(result)
                    }
                ],
            )

            span.set_attribute("output.value", final_response.choices[0].message.content)
            return final_response.choices[0].message.content

        return response.choices[0].message.content

# Try it out
response = handle_order_query("What's the status of order 12345?")
print(response)
```

<video controls src="https://storage.googleapis.com/arize-phoenix-assets/assets/gifs/ax_trace_with_tool.mp4" />

Trace showing tool invocation nested under parent span

Now in Arize AX, you'll see a complete trace tree showing:

* The parent "handle-order-query" span
* The LLM call that decided to use a tool
* The "execute-tool" span with parameters and results
* The final LLM call that formulated the response

## Step 5: Add RAG Tracing

For FAQ queries, we'll use a simple RAG pipeline. Let's trace both the retrieval and generation steps:

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
def embed_text(text: str) -> list[float]:
    """Generate embedding for text."""

    with tracer.start_as_current_span("embed-text") as span:
        span.set_attribute("openinference.span.kind", "EMBEDDING")
        span.set_attribute("input.value", text)
        span.set_attribute("embedding.model_name", "text-embedding-3-small")
        span.set_attribute(
            "embedding.invocation_parameters",
            json.dumps({"model": "text-embedding-3-small"})
        )

        response = openai_client.embeddings.create(
            model="text-embedding-3-small",
            input=text
        )

        embedding = response.data[0].embedding

        span.set_attribute("embedding.embeddings.0.embedding.text", text)
        span.set_attribute("embedding.embeddings.0.embedding.vector", embedding)

        return embedding

def retrieve_documents(query: str, knowledge_base: list[str], top_k: int = 2) -> list[str]:
    """Retrieve relevant documents using embeddings."""

    with tracer.start_as_current_span("retrieve-documents") as span:
        span.set_attribute("openinference.span.kind", "RETRIEVER")
        span.set_attribute("input.value", query)

        # Generate query embedding (in production, use to search a vector DB)
        query_embedding = embed_text(query)
        span.set_attribute("retrieval.query_embedding_dims", len(query_embedding))

        # In production: search vector DB with query_embedding and return top_k
        retrieved = knowledge_base[:top_k]

        # Record retrieved documents using OpenInference semantic conventions
        for i, doc in enumerate(retrieved):
            span.set_attribute(f"retrieval.documents.{i}.document.id", str(i))
            span.set_attribute(f"retrieval.documents.{i}.document.content", doc)

        return retrieved

def handle_faq_query(user_message: str) -> str:
    """Handle FAQ queries using RAG."""

    # Sample knowledge base
    knowledge_base = [
        "We offer free shipping on orders over $50.",
        "Returns are accepted within 30 days of purchase.",
        "You can track your order using the tracking number in your confirmation email.",
    ]

    with tracer.start_as_current_span("handle-faq-query") as span:
        span.set_attribute("openinference.span.kind", "CHAIN")
        span.set_attribute("input.value", user_message)

        # Retrieve relevant documents
        relevant_docs = retrieve_documents(user_message, knowledge_base)

        # Generate response with retrieved context
        context = "\n".join(relevant_docs)
        response = openai_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {
                    "role": "system",
                    "content": f"Answer the user's question using this context:\n{context}"
                },
                {"role": "user", "content": user_message}
            ],
        )

        answer = response.choices[0].message.content
        span.set_attribute("output.value", answer)

        return answer

# Try it out
response = handle_faq_query("What's your shipping policy?")
print(response)
```

<video controls src="https://storage.googleapis.com/arize-phoenix-assets/assets/gifs/ax_rag_trace.mp4" />

RAG trace showing retrieval and generation steps

Perfect! Now you can see exactly:

* What query was sent to retrieval
* Which documents were retrieved
* What context was passed to the LLM
* What answer was generated

## Putting It All Together

Here's the complete SupportBot with full tracing:

```python theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
def supportbot(user_message: str) -> str:
    """Main SupportBot entry point with complete tracing."""

    with tracer.start_as_current_span("supportbot") as span:
        span.set_attribute("openinference.span.kind", "AGENT")
        span.set_attribute("input.value", user_message)

        # Step 1: Classify
        classification = classify_query(user_message)

        # Step 2: Route and handle
        if classification == "ORDER_STATUS":
            response = handle_order_query(user_message)
        else:
            response = handle_faq_query(user_message)

        span.set_attribute("output.value", response)
        span.set_attribute("classification", classification)

        return response

# Test it out
queries = [
    "Where is my order #12345?",
    "What's your return policy?",
    "Can I track my order?",
]

for query in queries:
    print(f"\nUser: {query}")
    print(f"Bot: {supportbot(query)}")
```

Congratulations! You now have complete visibility into your LLM application!

## What's Next?

Tracing gives you visibility, but how do you measure *quality* at scale? In the next chapter, [Annotations and Evaluations](/docs/ax/observe/tutorial/annotations-and-evaluations.md), you'll learn how to:

* Annotate traces directly in the Arize AX UI
* Run automated quality evaluations
* Identify systemic issues across thousands of traces
* Transform debugging from manual inspection into data-driven analysis

Let's continue! →
