Skip to main content
Bluejay provides two separate dashboard surfaces — one for observability and one for simulations. Each tracks a distinct stage of your agent’s lifecycle, and the data between them is kept separate so you always have a clear picture of what is happening in production versus what happened during testing.

​
What You’ll Learn

  • The difference between observability and simulation dashboards
  • Common use cases and standard practices for each
  • How to structure dashboards as your agent grows

​
Two Dashboard Surfaces

​
Observability Dashboards

Observability dashboards let you monitor your agent’s production performance and track how your customers behave over time as they interact with your agent. Common use cases:
  • Customer sentiment analysis — understand how customers feel about interactions and detect shifts in sentiment over time.
  • A/B testing for agent releases — compare metrics across feature variants to decide which changes to keep.
  • Production pass-rate monitoring — make sure key KPIs are maintained as your product grows and conversation volume scales.
Standard practice: build custom observability dashboards that cover the key performance areas for your agent in production, alongside a set of customer-behavior metrics that matter to the business and leadership.

​
Simulation Dashboards

Simulation dashboards help you validate that agents meet a quality bar before they ship, and verify that new features work as intended. Common use cases:
  • Pre-release quality gates — confirm that an agent reaches the required quality threshold before a production release.
  • New-feature verification — validate that newly built functionality performs correctly across a range of test scenarios.
Standard practice: create a dedicated dashboard for each new feature or category you add to your agent. This gives you a focused view of that feature’s functionality during testing and a clear record of its readiness over time.

​
Next Steps

Simulation Dashboards

Track simulation performance and compare results across runs.

Observability Dashboards

Monitor production agent performance with live dashboards.

Alerts

Configure threshold-based notifications for production metrics.