Set Up Dashboards

You’ve defined custom metrics to measure what matters. Dashboards put those metrics — plus token counts, latency, error rates, and eval trends — on one page you can check daily. Two ways to get started: pick a pre-configured template or start with a blank dashboard. Either way, you shape the dashboard by adding widgets.

Start from a template or build a custom dashboard

Go to Dashboards in the sidebar and click + New Dashboard. Pick a template for a pre-configured layout, or Blank Dashboard to start from scratch.

Dashboard Templates selection showing available starter dashboards

For LLM tracing projects, two templates cover most needs:

Tracing Project Overview

A comprehensive view of project health:

Trace Volume and Errors: total requests and error counts
Latency: average span latency and distribution
Model Performance: token count and cost per model
Token Usage: prompt, completion, and total tokens over time
Average Cost: cost over time (if cost tracking is set up)
Average Eval Scores: eval score trends over time

Tracing Project Overview dashboard showing traces over time, latency, distribution, and token usage widgets

Token Tracking and Latency

Focused on usage and performance:

Total and Average Token Counts: prompt and completion tokens over time
Number of Requests: request volume
Average Latency: response time trends

Token Tracking and Latency dashboard showing token counts, request volume, and latency

Create grouped line charts with Alyx

Ask Alyx to create or update a line chart with one line per value of a column. For example: “Show trace count over time, split by status code.” You can request all values or name specific values to compare. For all values, the chart resolves up to 30 distinct values from the data and updates them as new values appear. A split-by chart supports one plot and cannot use trace-level or session-level metric aggregation. For trace counts, Alyx filters to root spans before grouping them. Every dashboard, template or blank, is shaped by the widgets you add. Open a dashboard and click + Add Widget in the top right. You’ll see a list of widget types: Time Series, Distribution, Statistic, Pivot Table, Scatter Plot, Text, Experiment, and more. Pick one to open the widget config.

Creating a dashboard widget with configuration panel

Inside the widget config, the Metric dropdown lets you pick a specific span metric under Span Metrics, or a roll-up like Trace Average or Session Average. Pick the level you actually want to measure.

Span-level vs. trace-level. Metrics under Span Metrics count every span. A count of 1,000 error spans could be 1,000 failing requests, or 10 failing traces with 100 errors each. Pick Trace Average or Session Average from the Metric dropdown to roll up to trace-level or session-level numbers.

The widget types below are the ones customers reach for most. Pick a tab to see configuration and a real example.

Time Series widgets show how a metric changes over time. A good first widget: p95 latency for LLM spans, so you can see slow responses at a glance.

Widget type: Time Series
Metric: p95 latency (add a custom metric using APPROX_QUANTILE)
Filter: attributes.openinference.span.kind = 'LLM'

Time Series widget showing p95 latency trending over time

Most production dashboards track an eval score over time — hallucination, relevance, or QA correctness. A Time Series of the Hallucination Eval Score tells you whether response quality is trending up or down.

Widget type: Time Series
Metric: Hallucination Eval Score — add a custom metric:

-- Hallucination rate percentage
SELECT COUNT(*) FILTER (WHERE "eval.Hallucination.label" = 'hallucinated') /
       COUNT(*) FILTER (WHERE "eval.Hallucination.label" IS NOT NULL) * 100
FROM MODEL

Time Series widget showing Hallucination Eval Score trending over time

Statistic widgets show a single aggregate value — the current state of things. Pair them with a Time Series for a “right-now” reading alongside the trend.

Widget type: Statistic
Metric: error rate as a percentage (custom metric using COUNT(*) FILTER (WHERE "status_code" = 'ERROR') / COUNT(*))

Statistic widget showing current error rate

Distribution widgets show the spread of values across a dimension. Group errors by graph.node.id to see which part of your agent is failing most.

Widget type: Distribution
Metric: count of error spans
Group by: attributes.graph.node.id

Distribution widget showing error counts by agent node

Pivot Table widgets compare a metric across categories in a table. Group by span kind to see how latency breaks down across LLM, Retriever, Tool, and Agent spans at a glance.Rows sort by the first metric, highest first. Click any column header to sort by that column instead. Sorting lasts for the current page visit rather than being saved to the widget, and it is applied before the top 1,000 rows are selected, so re-sorting can bring in rows the previous order left out.

Widget type: Pivot Table
Group By Dimension: attributes.openinference.span.kind
Value: latency_ms with Metric Avg, labeled “Average Latency”

Pivot Table widget showing average latency by span kind (Agent, LLM, Retriever, Tool)

Scatter Plot widgets show how two attributes correlate. Plot token count against latency for LLM spans to see whether longer prompts actually take longer to respond.

Widget type: Scatter Plot
X-Axis: latency_ms
Y-Axis: attributes.llm.token_count.total
Filter: attributes.openinference.span.kind = 'LLM'

Scatter Plot widget showing token count vs. latency for LLM spans

Save the dashboard. The rest of this page is about refining it — filters, exports, and gotchas.

Using your dashboards

Global filters and time range

The selectors at the top apply to every widget on the dashboard.

Time range — presets (Last 15 Min to Last 6 Months), custom range, or typed value. Timezone persists across sessions.
Metadata filters — scope by attributes like model name, environment, or custom metadata.

Dashboard with global filters and time range selector at top

Each widget can override the global filter. Open the widget’s config panel and add a filter there — useful when one widget needs a narrower or different scope than the rest of the dashboard.

Widget config panel with a per-widget filter on attributes.openinference.span.kind equal to LLM

Dashboards can include widgets from other projects in the same space. When configuring a widget, select the project you want to use as its data source. After the widget is added, that project will appear in the dashboard-wide project filter. The most common dashboard workflow: click a data point on a chart to jump straight to the traces behind it. When you notice an anomaly — a latency spike, an error surge — one click takes you to the filtered trace list, where you can open any trace or span.

Interact with widgets

Click a legend entry to isolate that series. Click the widget itself to open its config panel (metric, filter, visualization, title). Use the widget’s ... menu to duplicate, resize, or delete; drag to reorder.

Widget context menu on a dashboard widget showing Edit Widget, Duplicate Widget, Export Data, Export as PDF, View In Fullscreen, and Delete Widget

Option	How
Export as PDF (full dashboard)	Dashboard `...` menu → Export as PDF
Export as PDF (single widget)	Widget `...` menu → Export as PDF
Export widget data as CSV	Widget `...` menu → Export Data. Per-widget only — no dashboard-wide CSV
Share a link	Copy the URL. Anyone with access to the space can open it; permissions follow the space’s role settings
Programmatic access	Dashboard and widget data are accessible via the GraphQL API

Copy a dashboard

Duplicate a dashboard using the dashboard’s ... menu → Copy Dashboard. To recreate a dashboard in a different project, start from the same template there.

Dashboard context menu showing Edit Dashboard Title, Copy Dashboard, Export Dashboard as PDF, and Delete Dashboard options

Next step

You’ve put your metrics on a dashboard. Now set monitors on the same ones so regressions page you instead of waiting to be noticed:

Quickstart

Instrument

Observe

Agents

Evaluate

Experimentation

Machine Learning

Settings

Security

Start from a template or build a custom dashboard

Tracing Project Overview

Token Tracking and Latency

Create grouped line charts with Alyx

Add a widget

Using your dashboards

Global filters and time range

Per-widget filters

Add a widget from another project

View traces behind a widget

Interact with widgets

Copy a dashboard

Next step

Next: Continuously Monitor

​Start from a template or build a custom dashboard

​Tracing Project Overview

​Token Tracking and Latency

​Create grouped line charts with Alyx

​Add a widget

​Using your dashboards

​Global filters and time range

​Per-widget filters

​Add a widget from another project

​View traces behind a widget

​Interact with widgets

​Export and share

​Copy a dashboard

​Next step

Next: Continuously Monitor

Start from a template or build a custom dashboard

Tracing Project Overview

Token Tracking and Latency

Create grouped line charts with Alyx

Add a widget

Using your dashboards

Global filters and time range

Per-widget filters

Add a widget from another project

View traces behind a widget

Interact with widgets

Export and share

Copy a dashboard

Next step