Skip to main content

​
Using Amazon Bedrock with Phoenix Evals

Requires litellm and AWS credentials configured via environment variables or boto3.
pip install "arize-phoenix-evals>=3" litellm boto3
Use the "litellm" provider with a Bedrock model ID: 
from phoenix.evals import LLM

llm = LLM(provider="litellm", model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0")
LiteLLM reads AWS credentials from the standard environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION_NAME).

​
Using with evaluators

from phoenix.evals import LLM, evaluate_dataframe
from phoenix.evals.metrics import FaithfulnessEvaluator

llm = LLM(provider="litellm", model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0")
evaluator = FaithfulnessEvaluator(llm=llm)

results_df = evaluate_dataframe(dataframe=df, evaluators=[evaluator])

​
Authentication with assumed roles

If you need to assume a role, configure your credentials via environment variables before creating the LLM: 
import os

import boto3
from phoenix.evals import LLM

sts_client = boto3.client("sts")
response = sts_client.assume_role(
    RoleArn="arn:aws:iam::123456789012:role/MyRole",
    RoleSessionName="PhoenixEvalsSession",
)

creds = response["Credentials"]
os.environ["AWS_ACCESS_KEY_ID"] = creds["AccessKeyId"]
os.environ["AWS_SECRET_ACCESS_KEY"] = creds["SecretAccessKey"]
os.environ["AWS_SESSION_TOKEN"] = creds["SessionToken"]

llm = LLM(provider="litellm", model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0")