LiteLLM Evals

Using LiteLLM with Phoenix Evals

Requires litellm>=1.0.3

pip install "arize-phoenix-evals>=3" litellm

Create an LLM instance with the LiteLLM provider:

from phoenix.evals import LLM

llm = LLM(provider="litellm", model="gpt-4o")

LiteLLM supports 100+ providers. Set the appropriate environment variables for your provider before creating the LLM. For provider-specific configuration, see LiteLLM provider params.

Ollama example

import os
from phoenix.evals import LLM

os.environ["OLLAMA_API_BASE"] = "http://localhost:11434"

llm = LLM(provider="litellm", model="ollama/llama3")

Using with evaluators

from phoenix.evals import LLM, evaluate_dataframe
from phoenix.evals.metrics import FaithfulnessEvaluator

llm = LLM(provider="litellm", model="ollama/llama3")
evaluator = FaithfulnessEvaluator(llm=llm)

results_df = evaluate_dataframe(dataframe=df, evaluators=[evaluator])

Developer Tools

LLM Providers

TypeScript

Python

Java

.NET

Platform

Vector Databases

Sandboxes

Evaluation Integrations

Coding Agents

Using LiteLLM with Phoenix Evals

Ollama example

Using with evaluators

​Using LiteLLM with Phoenix Evals

​Ollama example

​Using with evaluators

Using LiteLLM with Phoenix Evals

Ollama example

Using with evaluators