Release Notes October 3, 2024

Arize Release Notes: Embeddings Tracing, Experiments Details, and More.

Sarah Welsh

Contributor

Welcome to our regular update on new releases, enhancements, and changes.

What’s New

Embeddings Tracing

With Embeddings Tracing, you can effortlessly select embedding spans and dive straight into the UMAP visualizer, simplifying troubleshooting for your genAI applications. How this works:

  • Users can now select embedding spans and go to the embedding visualizer (also available on project level nav for all generative models)
  • All embedding spans get pulled over for UMAP + clustering
  • Users can select an individual point/cluster and the embedding span attributes get pulled over
  • “Color by” span attributes is also available, and whichever attribute users “color by” can also be viewed as a column in the table

Note: this functionality is only viewable for embedding spans after October 2, 2024.

Learn more about our embeddings visualizer here.

Experiments Details Visualization

Users can now view a detailed breakdown of labels for their experiments on the Experiments Details page.

Note: this feature is currently only available for label-based evaluations. Support for score-exclusive evaluations will be coming in a future update.

Compare Experiments
Experiments Details Visualization

Enhancements

Prompt Playground Improvements

  • Full OpenAI Models Support: We’ve added full support for all available OpenAI models in the playground including the o1-mini and o1-preview.
  • OpenAI Function/Tool Calls: With OpenAI’s addition of function calling in their models, we have added support in Arize starting from the trace all the way to the playground. View function/tool calls in your traces, and open the playground to further experiment and test.
  • Full-screen Data Mode: Enter full screen mode to view your data more easily.
  • Features For Datasets:
    • Prompt Overriding: When using a dataset to test out different prompts, users can now replace the prompt template with the dataset.
    • Pop Up Windows for Long Outputs & Variables: Viewing longer outputs is now easier with pop up windows.

Along with these features, we’ve added improvements such as better input variable behavior, autocompletion enhancements, support for mustache/f-string input variables, and more.

prompt playground

Filters Updates

  • Filter History: We now store the last three filters used by a user! Users can easily access their filter history in the query filters dropdown, making it simpler to reuse filters for future queries.

filter history Traces Page Updates

  • Parent Spans: We’ve enhanced the Traces page to give users a better experience when filtering spans. Now, even if the filter doesn’t match the parent span, users will still see the the parent span with the relevant spans nested under it.
  • Quick Filters: We’ve introduced quick filters, allowing users to apply filters directly from the table by hovering over the text to reveal the filter icon.

Spans

New Arize-Otel Package for LLM App Tracing

We heard you, and we made it much simpler to add automatic tracing to your applications! It’s now just a few lines of code to use OpenTelemetry to trace your LLM application. Check out our new quickstart guide which uses our arize-otel package.

Workflow Improvements

  • Support for new line creation while reviewing prompts.
  • Easily add spans to a dataset from the Traces page using the “Add to Dataset” button.
  • Quickly create an evaluation task within a trace if no evaluations currently exist using the “Setup Task” button.

trace details

📚 New Content

The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts:

🧑‍⚖️ Selecting the Right Model for LLM-as-a-Judge Evaluations
🤖 Leveraging Agent Evaluation and Memory to Build Robust Agentic Systems
🔭 Exploring OpenAI’s o1 Models
🪞Breaking Down Reflection Tuning