How to trace LangChain applications with OpenInference and send data to Arize.
Arize provides first-class support for observing LangChain applications. After instrumentation using OpenInference, you will have a full trace of every part of your LLM application, including input, embeddings, retrieval, tool usage, and output messages, all viewable in Arize.
We follow a standardized format for trace data using OpenInference, an open-source standard based on OpenTelemetry. The arize-otel package is a lightweight convenience library to set up OpenTelemetry and send traces to Arize.
See here for more LangChain with Arize tutorials (Python).
This guide covers both Python and JavaScript/TypeScript.
Python Integration
API Key Setup
Before running your Python application, ensure you have the following environment variables set:
export ARIZE_SPACE_ID="YOUR_ARIZE_SPACE_ID"
export ARIZE_API_KEY="YOUR_ARIZE_API_KEY"
export OPENAI_API_KEY="YOUR_OPENAI_API_KEY" # For LangChain examples using OpenAI
# Add other LLM provider API keys if used by LangChain components
You can find your Arize Space ID and API Key in your Arize account settings.
(Add other langchain-* packages as needed, e.g., langchain-anthropic, langchain-google-genai)
Setup Tracing
Use arize.otel.register to configure the OpenTelemetry tracer and then apply the LangChainInstrumentor.
import os
from arize.otel import register
from openinference.instrumentation.langchain import LangChainInstrumentor
# Ensure your API keys are set as environment variables
# ARIZE_SPACE_ID = os.getenv("ARIZE_SPACE_ID")
# ARIZE_API_KEY = os.getenv("ARIZE_API_KEY")
# OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") # For the example
# Setup OTel via Arize's convenience function
tracer_provider = register(
space_id=os.getenv("ARIZE_SPACE_ID"), # or directly pass your Space ID
api_key=os.getenv("ARIZE_API_KEY"), # or directly pass your API Key
project_name="my-langchain-app" # Choose a project name
)
# Instrument LangChain
LangChainInstrumentor().instrument(tracer_provider=tracer_provider)
print("LangChain instrumented for Arize (Python).")
Run LangChain Example (Python)
By instrumenting LangChain, spans will be created whenever a chain, agent, or runnable is invoked and will be sent to Arize.
# Ensure OPENAI_API_KEY is set in your environment
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
prompt = ChatPromptTemplate.from_template("Why is the {celestial_object} {color}?").partial(celestial_object="sky")
# model_name can be gpt-3.5-turbo, gpt-4, etc.
# Ensure your OPENAI_API_KEY has access to the model you choose.
chain = prompt | ChatOpenAI(model_name="gpt-3.5-turbo")
response = chain.invoke({"color": "blue"})
print(response.content)
# Example with a different model (if you have access and the key set up)
# from langchain_anthropic import ChatAnthropic
# chain_anthropic = prompt | ChatAnthropic(model_name="claude-3-opus-20240229")
# response_anthropic = chain_anthropic.invoke({"color": "red"})
# print(response_anthropic.content)
Observe (Python)
Now that you have tracing set up, all invocations of LangChain components will be streamed to your Arize project for observability and evaluation.
JavaScript/TypeScript Integration
The OpenInference LangChain instrumentation is also available for JavaScript/TypeScript.
Below is an example instrumentation.ts file. You'll need to install all imported packages.
/* instrumentation.ts */
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import { ConsoleSpanExporter } from "@opentelemetry/sdk-trace-base"; // Optional: for console logging
import {
NodeTracerProvider,
SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-node";
import { resourceFromAttributes } from "@opentelemetry/resources";
import { OTLPTraceExporter as GrpcOTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-grpc";
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import { Metadata } from "@grpc/grpc-js";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager"; // Critical for LangChain instrumentation
// For troubleshooting, set the log level to DiagLogLevel.DEBUG
// diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.INFO); // Or DiagLogLevel.DEBUG
const spaceId = process.env.ARIZE_SPACE_ID || "YOUR_ARIZE_SPACE_ID";
const apiKey = process.env.ARIZE_API_KEY || "YOUR_ARIZE_API_KEY";
const projectName = process.env.ARIZE_PROJECT_NAME || "my-langchain-js-app";
// Arize specific: Create metadata for OTLP exporter headers
const metadata = new Metadata();
metadata.set('space_id', spaceId);
metadata.set('api_key', apiKey);
// Optional: Console exporter for local debugging
const consoleExporter = new ConsoleSpanExporter();
const consoleProcessor = new SimpleSpanProcessor(consoleExporter);
// Arize OTLP gRPC exporter
const otlpExporter = new GrpcOTLPTraceExporter({
url: "https://otlp.arize.com/v1", // Arize OTLP endpoint
metadata,
});
const otlpProcessor = new SimpleSpanProcessor(otlpExporter);
const provider = new NodeTracerProvider({
resource: resourceFromAttributes({
// Arize specific: Define the project for your traces
"project_name": projectName,
// You can add other resource attributes here if needed
}),
// Add spanProcessors: otlpProcessor for Arize, consoleProcessor for local logs
spanProcessors: [otlpProcessor, consoleProcessor]
});
// Register the LangChain instrumentation
const lcInstrumentation = new LangChainInstrumentation();
// LangChain must be manually instrumented as it doesn't have a traditional module structure
// that OpenTelemetry auto-instrumentation typically relies on.
// The CallbackManagerModule is what OpenInference hooks into.
lcInstrumentation.manuallyInstrument(CallbackManagerModule);
provider.register();
console.log(\`LangChain instrumented for Arize (JS/TS). Project: \${projectName}\`);
// Example of how to run your LangChain code (e.g., in your main app.ts or server.ts)
// import { ChatOpenAI } from "@langchain/openai";
// import { HumanMessage } from "@langchain/core/messages";
// async function main() {
// if (!process.env.OPENAI_API_KEY) {
// throw new Error("OPENAI_API_KEY environment variable is not set.");
// }
// const chat = new ChatOpenAI({ modelName: "gpt-3.5-turbo", temperature: 0 });
// const response = await chat.invoke([
// new HumanMessage("Hello, how are you today?"),
// ]);
// console.log(response);
// }
// main().catch(console.error);
Note on instrumentation.ts: This file should be imported and run at the very beginning of your application's lifecycle to ensure all LangChain operations are instrumented. For example, in a Node.js application, you might import it at the top of your main server file or use the node -r ./instrumentation.js your-app.js command.
Native Thread Tracking (JS/TS)
Arize supports native thread tracking with LangChain by enabling the use of session_id, thread_id, or conversation_id in the metadata of your LangChain calls. This allows for seamless tracking of multi-turn conversations.
