Skip to main content

Add and annotate entire sessions and traces in annotation queues

September 8, 2026 New  Annotations You can now send full multi-turn sessions and complete traces to an annotation queue, not just individual spans and dataset examples, and review them at session or trace granularity. Add sessions to a queue from the Sessions table or a session’s detail view in the app, or add sessions and traces through the REST API and the Python, TypeScript, and Go SDKs, so you can build session-level and trace-level review workflows end to end. One queue can hold a mix of record types, so a whole session, a whole trace, an individual span, and a dataset example can sit side by side, and the annotator completes each record as its own unit of work. A queued session renders as the full session for the annotator, who labels it without leaving the queue. The code example below adds a session to an existing queue for annotation at session granularity:
Learn more about assigning records to a labeling queue.

Inspect remote evaluator runs as traces

September 8, 2026 New  Evaluators Remote evaluator runs now emit a trace for each evaluated row into your space’s evaluations project, with every span capturing the request, response, status code, and latency, so you can inspect and debug remote scoring calls and see retries as sibling spans in one trace. Turn it on with the Enable Tracing toggle under Advanced settings when you create or edit a remote evaluator. Learn more about remote evaluators.

Select OpenAI GPT-6 in the prompt playground

September 8, 2026 New  Models and Integrations You can now select OpenAI’s gpt-6-astra in the prompt playground model picker and run prompts against it. The model supports reasoning-effort levels from low through max, a verbosity control, and structured output, and it routes through the OpenAI Responses API for tool calling. Learn more about the prompt playground.

See GPT model reasoning in Alyx with thinking mode

September 4, 2026 Improvement  Alyx Alyx now streams reasoning summaries for OpenAI GPT models through its thinking view, which previously showed reasoning only for Claude models. When Alyx runs on a supported GPT model, you can follow its reasoning as it works through a request rather than seeing only the final answer. Learn more about Alyx.

Fixes and improvements

September 3–9, 2026 Evaluators
  • Improvement Run evaluations continuously against your space’s evaluations tracing project; tracing is automatically kept off for these tasks so they do not feed their own results back in.
Annotations
  • Improvement Choose which columns annotators see when you create a labeling queue from a dataset’s Examples tab, a trace, a session, or any of their selection toolbars.
Tracing and Sessions
  • Fix See spans that behave like errors in the trace errors tab, including those whose name or status message contains terms such as timeout, failure, or exception, alongside spans that carry a formal error status.
Agents
  • Improvement Start an automation right away: creating an automation now runs it immediately and opens its detail view instead of waiting for the next scheduled run.
SDKs and REST APIs
  • New Detach tags from monitors, evaluators, and prompts through the REST API with DELETE /v2/monitors/{monitor_id}/tags, DELETE /v2/evaluators/{evaluator_id}/tags, and DELETE /v2/prompts/{prompt_id}/tags.

Run Agent-as-a-Judge evaluators on every plan

September 1, 2026 Improvement  Evaluators Agent-as-a-Judge evaluators, which use an agentic judge to inspect traces, find the relevant spans, and label issues, are now available on every plan instead of Enterprise only. Any account can create one from the New Evaluator menu in the evaluator gallery and let the judge adapt its criteria to the failures that emerge in production, so you can evaluate agent behavior that a fixed template cannot anticipate. Learn more about Agent-as-a-Judge.

Run Claude Fable 5.1 in the Playground and evaluators

September 1, 2026 New  Models and Integrations Select Claude Fable 5.1 from the model picker to prototype prompts, run experiments, and score traces with Anthropic’s latest Fable model. It is available through the direct Anthropic API as well as AWS Bedrock and Google Vertex AI, and runs with adaptive extended thinking for complex reasoning.
  • Anthropic: claude-fable-5-1
  • AWS Bedrock: us.anthropic.claude-fable-5-1 and global.anthropic.claude-fable-5-1
  • Vertex AI: claude-fable-5-1
Learn more about model integrations.

Deep dive on Signal issues with Alyx

September 1, 2026 Improvement  Alyx Alyx can now read the issues Signal has recorded for your space, so you can ask what Signal has found across a project or single out one issue and have Alyx analyze it in depth. Signal reports on cost, token waste, latency, prompt quality, and tool choice, much of it on traces that never returned an error.
  • Context from the Signal tab: the issue you have open is attached to each message as a pill in the composer, and Alyx answers questions about issue counts and timing from it instead of querying traces.
  • Attach an issue from any page: add an issue to the conversation through the @ menu, where it is listed under Signal Insights, the same way you attach projects, datasets, and experiments.
  • A pointer to prior analysis: when you ask Alyx to debug a trace that Signal already reviewed, it links you to the matching issue instead of repeating that analysis.
Learn more about Alyx.

Open the filtered traces behind an Alyx answer

September 1, 2026 Improvement  Alyx When you select Apply filter in Alyx or ask it for a link to a set of traces, Alyx now opens the tracing view showing exactly those traces, with the correct filter expression, environment, and time range already applied. This works across projects and for rolling time windows such as the last 15 minutes, which now resolve to live data instead of a fixed time range. Learn more about Alyx.

Annotate entire sessions from the session view

August 31, 2026 New  Annotations Select Annotate Session in a session’s detail view in the tracing page, then apply a label or score to the session as a whole through the annotation pane. Each annotation is recorded as session_annotation.<name>.label, .score, and .text, so you can query and filter on a session annotation the way you would any other attribute.
Session 20405 open over the Sessions tab, showing its total duration, tokens, cost, and trace count, with the Session Conversation tab listing Trace #1 and Trace #2 and a Session Annotations pane on the right where an Accuracy config is set to accurate
Session annotations appear in the sessions table, the session trace view, the tracing table, the trace slideover annotations table, and the dataset examples table and example page, and the query filter dropdown offers them as filter terms. The REST API and the Python, TypeScript, and Go SDKs annotate at session granularity as well. The code example below annotates one session by its session ID:
Learn more about annotating traces.

Switch between trace and session results in the evaluator preview

August 29, 2026 Improvement  Evaluators After choosing a filter, you can now switch between trace and session results in the Preview Matching Data step of the evaluator’s Configuration column, and select any trace or session from the results table to preview its data for easier variable mapping. Choose a row with its checkbox, or move through the results with the up and down arrows beside the Table and Single Trace toggle. Whichever record you select fills the Single Trace view and the per-variable value previews. A session-scoped evaluator leads with the Sessions tab, and the Traces and Spans tabs narrow to the session you selected. The Single Trace view shows one of that session’s traces at a time, newest first, and the arrows step through the rest. Your selection resets when you change the project, filter, scope, or preview window. Learn more about trace and session evals.

Fixes and improvements

August 27 – September 2, 2026 Datasets and Experiments
  • Improvement Large dataset uploads now complete reliably, with transient chunk failures retried automatically before the upload returns.
  • Fix The Experiments REST API (beta) now reports accurate created and updated timestamps for each experiment.
Evaluators
  • Fix Template and LLM evaluators now keep the model’s explanation even when a provider returns free-form text instead of structured output.
Tracing and Sessions
  • Fix Spans from Bedrock native InvokeModel calls now include the output message role, so you can filter, group, and evaluate on it.
Playground
  • Fix Prompt Playground experiment runs now finish when the worker queue is busy, giving each run time to start before any timeout applies.
Agents
  • Fix Signal issue deep links now open the exact issue, even when it is resolved or older than the 30-day issue window.
SDKs and REST APIs
  • Fix The Python SDK now resolves a space name to that exact space before listing resources, so a name that is a substring of another space no longer returns the wrong space’s resources.
  • Fix The TypeScript and Go SDKs now let you switch a task between span and trace or session query shapes when you update it.