What is a Session?

A session is a grouping of related traces. For example, in a chatbot application, each conversation generates multiple traces — one for each human message and AI response. By organizing these related traces into a single session, you can easily view and analyze the full conversation flow between the human and the AI. Sessions allow you to track context across turns and understand how your application behaves over the course of an interaction.

Why are Sessions Important?

By grouping your traces into sessions, you can:

Find exactly where a conversation “breaks” or goes off the rails. This can help identify if a user becomes progressively more frustrated or if a chatbot is not helpful.
Identify broader performance issues across sessions by running session-level evaluations.
Construct custom metrics based on evals using session.id or user.id to find best/worst performing sessions and users.

Configure Sessions

Adding session.id and/or user.id from an application enables back-and-forth interactions to be grouped. Session and user IDs can be added to a span using auto instrumentation or manual instrumentation of Open Inference. Any LLM call within the context (the with block in the example below) will contain corresponding session.id or user.id as a span attribute. session.id and user.id must be a non-empty string. When defining your instrumentation, you can pass the sessionID attribute as shown below.

Python
JS/TS

`using_session`

Context manager to add session ID to the current OpenTelemetry Context. OpenInference auto instrumentators will read this Context and pass the session ID as a span attribute, following the OpenInference semantic conventions. Its input, the session ID, must be a non-empty string.

from openinference.instrumentation import using_session

with using_session(session_id="my-session-id"):
    # Calls within this block will generate spans with the attributes:
    # "session.id" = "my-session-id"
    ...

It can also be used as a decorator:

@using_session(session_id="my-session-id")
def call_fn(*args, **kwargs):
    # Calls within this function will generate spans with the attributes:
    # "session.id" = "my-session-id"
    ...

`using_user`

Context manager to add user ID to the current OpenTelemetry Context. OpenInference auto instrumentators will read this Context and pass the user ID as a span attribute, following the OpenInference semantic conventions. Its input, the user ID, must be a non-empty string.

from openinference.instrumentation import using_user
with using_user("my-user-id"):
    # Calls within this block will generate spans with the attributes:
    # "user.id" = "my-user-id"
    ...

It can also be used as a decorator:

@using_user("my-user-id")
def call_fn(*args, **kwargs):
    # Calls within this function will generate spans with the attributes:
    # "user.id" = "my-user-id"
    ...

We provide a setSession function which allows you to set a sessionId on context. You can use this utility in conjunction with context.with to set the active context. OpenInference auto instrumentations will then pick up these attributes and add them to any spans created within the context.with callback.

npm install --save @arizeai/openinference-core @opentelemetry/api

import { context } from "@opentelemetry/api"
import { setSession } from "@openinference-core"

context.with(
  setSession(context.active(), { sessionId: "session-id" }),
  () => {
      // Calls within this block will generate spans with the attributes:
      // "session.id" = "session-id"
  }
)

We also provide a setUser function which allows you to set a userId on context. You can use this utility in conjunction with context.with to set the active context. OpenInference auto instrumentations will then pick up these attributes and add them to any spans created within the context.with callback.

import { context } from "@opentelemetry/api"
import { setUser } from "@openinference-core"

context.with(
  setUser(context.active(), { userId: "user-id" }),
  () => {
      // Calls within this block will generate spans with the attributes:
      // "user.id" = "user-id"
  }
)

Additional Examples

Requires pip install openinference-instrumentation-openai

Once you define your OpenAI client, any call inside our context managers will attach the corresponding attributes to the spans.

import openai
from openinference.instrumentation import using_attributes

client = openai.OpenAI()

# Defining a Session
with using_attributes(session_id="my-session-id"):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku."}],
        max_tokens=20,
    )

# Defining a User
with using_attributes(user_id="my-user-id"):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku."}],
        max_tokens=20,
    )
    
# Defining a Session AND a User
with using_attributes(
    session_id="my-session-id",
    user_id="my-user-id",
):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": "Write a haiku."}],
        max_tokens=20,
    )