Skip to main content

​
Using LiteLLM with Phoenix Evals

Requires litellm>=1.0.3
pip install "arize-phoenix-evals>=3" litellm
Create an LLM instance with the LiteLLM provider: 
from phoenix.evals import LLM

llm = LLM(provider="litellm", model="gpt-4o")
LiteLLM supports 100+ providers. Set the appropriate environment variables for your provider before creating the LLM. For provider-specific configuration, see LiteLLM provider params.

​
Ollama example

import os
from phoenix.evals import LLM

os.environ["OLLAMA_API_BASE"] = "http://localhost:11434"

llm = LLM(provider="litellm", model="ollama/llama3")

​
Using with evaluators

from phoenix.evals import LLM, evaluate_dataframe
from phoenix.evals.metrics import FaithfulnessEvaluator

llm = LLM(provider="litellm", model="ollama/llama3")
evaluator = FaithfulnessEvaluator(llm=llm)

results_df = evaluate_dataframe(dataframe=df, evaluators=[evaluator])