Build dashboards to track your app’s health, token usage, eval scores, and experiment results at a glance
You’ve defined custom metrics to measure what matters. Dashboards put those metrics — plus token counts, latency, error rates, and eval trends — on one page you can check daily.Two ways to get started: pick a pre-configured template or start with a blank dashboard. Either way, you shape the dashboard by adding widgets.
Every dashboard, template or blank, is shaped by the widgets you add. Open a dashboard and click + Add Widget in the top right. You’ll see a list of widget types: Time Series, Distribution, Statistic, Pivot Table, Scatter Plot, Text, Experiment, and more. Pick one to open the widget config.
Inside the widget config, the Metric dropdown lets you pick a specific span metric under Span Metrics, or a roll-up like Trace Average or Session Average. Pick the level you actually want to measure.
Span-level vs. trace-level. Metrics under Span Metrics count every span. A count of 1,000 error spans could be 1,000 failing requests, or 10 failing traces with 100 errors each. Pick Trace Average or Session Average from the Metric dropdown to roll up to trace-level or session-level numbers.
The widget types below are the ones customers reach for most. Pick a tab to see configuration and a real example.
Time Series
Eval score
Statistic
Distribution
Pivot Table
Scatter Plot
Text
Time Series widgets show how a metric changes over time. A good first widget: p95 latency for LLM spans, so you can see slow responses at a glance.
Widget type: Time Series
Metric: p95 latency (add a custom metric using APPROX_QUANTILE)
Filter:attributes.openinference.span.kind = 'LLM'
Most production dashboards track an eval score over time — hallucination, relevance, or QA correctness. A Time Series of the Hallucination Eval Score tells you whether response quality is trending up or down.
Widget type: Time Series
Metric: Hallucination Eval Score — add a custom metric:
-- Hallucination rate percentageSELECT COUNT(*) FILTER (WHERE "eval.Hallucination.label" = 'hallucinated') / COUNT(*) FILTER (WHERE "eval.Hallucination.label" IS NOT NULL) * 100FROM MODEL
Statistic widgets show a single aggregate value — the current state of things. Pair them with a Time Series for a “right-now” reading alongside the trend.
Widget type: Statistic
Metric: error rate as a percentage (custom metric using COUNT(*) FILTER (WHERE "status_code" = 'ERROR') / COUNT(*))
Distribution widgets show the spread of values across a dimension. Group errors by graph.node.id to see which part of your agent is failing most.
Widget type: Distribution
Metric: count of error spans
Group by:attributes.graph.node.id
Pivot Table widgets compare a metric across categories in a table. Group by span kind to see how latency breaks down across LLM, Retriever, Tool, and Agent spans at a glance.Rows sort by the first metric, highest first. Click any column header to sort by that column instead. Sorting lasts for the current page visit rather than being saved to the widget, and it is applied before the top 1,000 rows are selected, so re-sorting can bring in rows the previous order left out.
Widget type: Pivot Table
Group By Dimension:attributes.openinference.span.kind
Value:latency_ms with Metric Avg, labeled “Average Latency”
Scatter Plot widgets show how two attributes correlate. Plot token count against latency for LLM spans to see whether longer prompts actually take longer to respond.
Widget type: Scatter Plot
X-Axis:latency_ms
Y-Axis:attributes.llm.token_count.total
Filter:attributes.openinference.span.kind = 'LLM'
Text widgets add context, section headers, or notes alongside your charts. Use them to explain what a dashboard tracks, link to runbooks, or separate dashboard sections.
Widget type: Text
Content: Markdown — headings, bold, lists, and inline code all render
Save the dashboard. The rest of this page is about refining it — filters, exports, and gotchas.
Each widget can override the global filter. Open the widget’s config panel and add a filter there — useful when one widget needs a narrower or different scope than the rest of the dashboard.
The most common dashboard workflow: click a data point on a chart to jump straight to the traces behind it. When you notice an anomaly — a latency spike, an error surge — one click takes you to the filtered trace list, where you can open any trace or span.
Click a legend entry to isolate that series. Click the widget itself to open its config panel (metric, filter, visualization, title). Use the widget’s ... menu to duplicate, resize, or delete; drag to reorder.
Duplicate a dashboard using the dashboard’s ... menu → Copy Dashboard. To recreate a dashboard in a different project, start from the same template there.
