Your quality dashboard
Use the dashboard metrics to understand results from your test suite. They summarize completed evaluations and depend on the assertions and stage data your tests produce. They are not independent measurements of every response.
Choose a metric scope
Section titled “Choose a metric scope”If your project has more than one Agent, use the selector beside Dashboard to choose which Agents the metric panels include. All agents includes every non-demo Agent. An Agent name limits the panels to that Agent. Demo shows the seeded demo data.
The selected scope applies to Quality, Determinism, Hallucination, Stage Pipeline Health, Test vs live traffic, Scenario Coverage, Assertion Library In Use, and Test × Run. Latest Runs is separate and remains project-wide, including runs from other Agents.
Quality metrics
Section titled “Quality metrics”Your project dashboard shows three summary metrics:
Quality
Section titled “Quality”The average assertion score across the completed test results included in the dashboard. It reflects the checks you wrote, not an independent assessment of overall response quality. A narrow test suite can produce a high score while leaving other behavior untested.
Determinism
Section titled “Determinism”The percentage of consecutive runs of the same test whose four stage statuses match: Router, Args, Tool, and Response. It measures stage-outcome stability, not identical response text. The dashboard needs at least two runs of the same test to calculate it.
Hallucination
Section titled “Hallucination”The failure rate of evaluated semantic assertions that you explicitly mark as hallucination_check. The metric stays unavailable until one of these checks has run. It does not independently detect hallucinations or review every response. To measure factuality, add criteria to a hallucination check and run the test. See AI-powered quality checks.
A card can show — with a reason instead of a metric. For example, Quality can show Run your first test, Determinism can show Needs ≥2 runs of the same test, and Hallucination can show Add a hallucination check. A metric that is not available on your plan shows Not available on your plan.
Stage pipeline health
Section titled “Stage pipeline health”The Stage Pipeline Health card shows where test results pass or fail. It has four stages:
| Stage | What it covers |
|---|---|
| Router | Intent-to-handler routing |
| Args | Argument validation |
| Tool | Tool execution |
| Response | Response quality |
Each stage shows its pass and fail percentages for the evaluated data. A stage with no evaluated data shows No data. If the project has no test runs, the card shows No test runs yet. Run your suite to see stage health.
Select Show step-by-step per test to see the stage status for each test case. Select a test case to open its details. A row can show Passed, Skipped, No data, or the first failed stage.
Compare test runs with live traffic
Section titled “Compare test runs with live traffic”The Test vs live traffic card compares assertion pass rates over the displayed time window, which defaults to 30 days. Test runs are manual evaluations. Live traffic is passive evaluation from traces sent to Mibo.
When both types of data are available, the card lists Top drifts (>10pp). pp means percentage points. Use these rows to find checks that behave differently on live traffic than in your test runs. The card also lists assertion types that only run in tests when they cannot be computed from passive traces.
If no live traces were evaluated in the window, check your passive testing setup, then send a fresh trace. If no test runs were evaluated, run your test suite.
Scenario coverage
Section titled “Scenario coverage”The Scenario Coverage card shows conditional scenarios over the displayed time window, which defaults to 7 days. It separates scenarios With traffic from scenarios with No traffic.
The card also reports:
- Traces that did not match any scenario. Consider adding a test for those conversations.
- Traces that could not be routed because the classifier was unavailable. Coverage counts can remain incomplete after routing recovers. Resending an accepted trace through Your API with the same identity returns
200 OKwithout queuing another passive evaluation. Identical OTLP retries do not reevaluate the trace either. - Traces without usable input, for which conditional coverage was skipped.
- Active-only checks that stayed outside passive evaluation.
Other dashboard panels
Section titled “Other dashboard panels”- Assertion Library In Use lists each assertion type, its total count, and the number of test cases that use it.
- Test × Run shows test cases as rows and runs as columns. Its cells identify passed, failed, error, skipped, and pending results. Hover over a cell for its full date.
- Latest Runs lists the five most recent runs. Select a run to open its Agent’s test list.
Inspect an individual test result
Section titled “Inspect an individual test result”Open a test case to inspect its Last Result. See Read the result status for status meanings and Watching live logs for response, error, and assertion details.
Inspect traces and live traffic
Section titled “Inspect traces and live traffic”Open an Agent’s Traces page to review traces from both active test runs and passive evaluations. Use the traffic filter to choose All traffic, Live traffic, or Test runs. If your Agent sends an environment value, you can also filter by All environments or a specific environment.
Select a trace to open its details. Evaluated traces show an Active or Passive badge. A trace that is waiting for passive evaluation has no source badge yet. Active traces show Not checked because passive testing does not run during an active test.
The detail view has four tabs:
- Overview shows the trace date and Conversation. It shows a source badge when available. When checks were evaluated, it also shows a Results summary with Tests ran, Passed, Failed, Skipped, and Did not apply. Scenario-routing information appears for passive traces when it is available.
- Needs attention (N) lists failed, errored, and skipped tests. Select the Review [number] issue(s) action to inspect a test and its failure details. Use View test to open the test case.
- All tests (N) lists every result for the trace. Select a test to inspect its assertions and evidence.
- Raw data shows the Raw trace payload and, when available, Execution metadata.
If the trace has no extractable user or Agent text, the overview shows Conversation unavailable. Select View raw data to inspect the received payload. If a passive evaluation produces no result, the trace can show No checks were evaluated. It can also show Routing unavailable or No usable input when Mibo cannot route conditional tests. These states do not mean that the trace was rejected. For recovery steps, see If a trace appears without an evaluated result.
Using results to improve your system
Section titled “Using results to improve your system”For execution errors, follow Recover from a run error. If a semantic assertion has no usable evaluator score, use the matching recovery path before judging the Agent response. For an active test, follow Recover from a semantic evaluation issue. For a passive evaluation, follow Recover from a passive semantic evaluation issue. Once you have response evaluations, use this approach to improve their quality:
- Check Stage Pipeline Health. Look for the stage with the most failures, then expand Show step-by-step per test and open a relevant test case.
- Check the Error and logs first. A connection or execution error can leave no response or assertion results.
- Review the Hallucination metric. If it is high, inspect the failed hallucination checks and their reasoning.
- Compare Test vs live traffic. Use the drift rows and Scenario Coverage to find checks that behave differently on real conversations or scenarios that have no traffic.
- Save test edits before rerunning. Then run the same tests again to verify the change.
What’s next
Section titled “What’s next”Set up passive testing to evaluate traces from production.