​

Classification Evaluators on AI SDK Decision Models

• Probabilities in metadata. metadata.probabilities maps each label to its probability, and metadata.confidence is included when the provider reports one. Decision models do not return an explanation.
• Refusals throw the AI SDK’s Experimental_DecisionRefusalError. In runExperiment, a refusal is recorded as that evaluation’s error.
• Decision calls are traced. Decide spans record the decision state, question, and answer, and @arizeai/phoenix-otel 2.3.0 traces AI SDK decision models. isDecisionModel and the DecisionModel type are also exported.
• Evaluation models change too. TypeSafe’s Jev now also goes through experimental_decide, and message prompts are sent as plain text instead of JSON state. Use @ai-sdk/openai 4.0.89 or later with OpenAI decision models.

import { openai } from "@ai-sdk/openai";
import { createCorrectnessEvaluator } from "@arizeai/phoenix-evals";

const correctness = createCorrectnessEvaluator({
  model: openai.decisionModel("gpt-6-luna"),
});

const result = await correctness.evaluate({
  input: "What is the capital of Australia?",
  output: "Sydney.",
});
// result.label === "incorrect"
// result.metadata?.probabilities -> { correct: 0.02, incorrect: 0.98 }

​

A Setup Guide for Empty Projects

• The tab strip and Config tab stay usable, so you can configure the project before sending traces.
• Skeleton loaders now stand in for tables, charts, and panels while they load across the app, and their wave animation stops when your system asks for reduced motion.

​

Graded Annotation Score Colors

• Applies everywhere scores are colored: annotation summaries and badges, span previews, metrics tooltips, evaluator previews, and the experiment compare views.
• A score exactly at the threshold is neutral instead of unfavorable.
• Bounded freeform scores without a threshold now get colors in the experiment example details view too, as they already did in the compare table.

​

Server Telemetry for Self-Hosted Deployments

• Browser errors reach the server log. Uncaught errors and unhandled promise rejections in the Phoenix UI are logged at ERROR level on the phoenix.client_errors logger. Identical errors are reported once per page session, and reports are rate limited per client IP.
• service.version on server telemetry. The server’s OpenTelemetry resource now carries the Phoenix version next to service.name, so you can split server traces and logs by release.

​

Additional Improvements

• MiniMax-M3.1-Flash replaces MiniMax-M2.7 in the playground’s MiniMax model list, and you can still type any other MiniMax model name.
• The prompt Versions tab loads every version as you scroll, instead of stopping at the first 50, and its counter shows the total.
• The icon-only Add to Dataset action in the span details header now has a tooltip and an accessible name.
• Cancelled code evaluations delete their sandbox, so a sandbox no longer keeps running on the provider account until it expires.
• Python evals honor overrides: template_format now applies when copying a PromptTemplate, and sync and async client kwargs now override the shared LLM kwargs.
• TypeScript toSDK keeps description, default, and other keywords next to anyOf in prompt tool and response-format schemas.
• createEvaluator keeps the metadata returned by custom TypeScript evaluator functions.