Amazon Bedrock Evals

Using Amazon Bedrock with Phoenix Evals

Requires litellm and AWS credentials configured via environment variables or boto3.

pip install "arize-phoenix-evals>=3" litellm boto3

Use the "litellm" provider with a Bedrock model ID:

from phoenix.evals import LLM

llm = LLM(provider="litellm", model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0")

LiteLLM reads AWS credentials from the standard environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION_NAME).

Using with evaluators

from phoenix.evals import LLM, evaluate_dataframe
from phoenix.evals.metrics import FaithfulnessEvaluator

llm = LLM(provider="litellm", model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0")
evaluator = FaithfulnessEvaluator(llm=llm)

results_df = evaluate_dataframe(dataframe=df, evaluators=[evaluator])

Authentication with assumed roles

If you need to assume a role, configure your credentials via environment variables before creating the LLM:

import os

import boto3
from phoenix.evals import LLM

sts_client = boto3.client("sts")
response = sts_client.assume_role(
    RoleArn="arn:aws:iam::123456789012:role/MyRole",
    RoleSessionName="PhoenixEvalsSession",
)

creds = response["Credentials"]
os.environ["AWS_ACCESS_KEY_ID"] = creds["AccessKeyId"]
os.environ["AWS_SECRET_ACCESS_KEY"] = creds["SecretAccessKey"]
os.environ["AWS_SESSION_TOKEN"] = creds["SessionToken"]

llm = LLM(provider="litellm", model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0")

Developer Tools

LLM Providers

TypeScript

Python

Java

Platform

Vector Databases

Sandboxes

Evaluation Integrations

Coding Agents

Amazon Bedrock Evals

Using Amazon Bedrock with Phoenix Evals

Using with evaluators

Authentication with assumed roles

​Using Amazon Bedrock with Phoenix Evals

​Using with evaluators

​Authentication with assumed roles

Using Amazon Bedrock with Phoenix Evals

Using with evaluators

Authentication with assumed roles