What Observability Dashboards Show
Every call evaluated through the observability pipeline feeds into your dashboards. The data includes:- Custom Metric scores — tracked over time so you can spot trends and regressions
- Hallucination and redundancy rates — built-in quality signals aggregated across all evaluated calls
- Latency metrics — average agent latency, time to first utterance, and per-turn response times
- Call volume and status — how many calls are being evaluated and how they’re distributed
- Alert badges — visual indicators when metrics cross configured thresholds
Common Use Cases
Production Pass-Rate Monitoring
Track your most important Custom Metrics across all production calls. Set up a dashboard that shows daily or weekly pass rates for metrics like compliance adherence, resolution success, or customer satisfaction. Use it in team standups or leadership reviews to confirm your agent is meeting its quality bar.Customer Sentiment Analysis
Build dashboards around customer-facing metrics — sentiment scores, escalation rates, and interaction quality. These help you understand how customers experience your agent over time and catch shifts before they become problems.A/B Testing Agent Releases
When rolling out a new prompt version or model change, compare metric scores between the old and new variants. Observability dashboards let you filter by metadata fields likeprompt_version or experiment_group to see side-by-side performance.
Operational Health
Monitor operational signals like average latency, token usage, and error rates. These dashboards help engineering teams catch infrastructure-level issues — slow model responses, integration timeouts, or transcript quality degradation — before they affect customer experience.Structuring Your Dashboards
| Strategy | When to use |
|---|---|
| One dashboard per agent | Teams with a small number of agents that need unified monitoring |
| One dashboard per business function | Organizations where different teams own different quality domains (support, sales, compliance) |
| One dashboard for leadership | High-level view of pass rates and volume across all agents |
| One dashboard for engineering | Operational metrics like latency, errors, and token usage |
Next Steps
Alerts
Configure threshold-based notifications for production metrics.
Custom Metrics
Define the evaluation criteria your dashboards track.
API Integration Tutorial
Connect your production pipeline to start feeding dashboards.
Dashboards Overview
Learn about both simulation and observability dashboard surfaces.