You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(frontend): Evaluation run page and Overview tab in /m behind the developer view switch - #8042
Do not merge into v0.128.4. This goes into the next release.
Context
The /m app has no page for a single evaluation run, so users still open the old /w run page to see the results of a run. This PR adds the run page to /m with its Overview tab. It is the first of four stacked PRs for the run page. The Scenarios tab, the Configuration tab and the scenario drawer come in the PRs stacked on this one.
It ships hidden
Nothing here is reachable until the evaluations developer view is enabled. The PR adds the route /evaluations/results/[evaluation_id] to that view, with stage hidden, and no navigation link. While the view is hidden, the route shows "page not found". The PR adds no dependency, no antd, and nothing in _app, AppProviders, GlobalDrawers or the app shell. The charts are the SVG charts from @agenta/ui (#8011), and they load only on this route.
What the page does
Title: the run name, or the run id when the run has no name.
Live runs: a "Pause" or "Resume" button, shown only to readers with edit_evaluation_runs. Toasts "Evaluation stopped", "Evaluation resumed", "Failed to stop evaluation", "Failed to resume evaluation".
Tabs "Overview", "Scenarios", "Configuration", and "Annotate" for human runs, kept in ?view=. In this PR only Overview has a body, and the other tabs show as disabled. The kind of run comes from the run itself, not from a URL parameter.
Overview, summary table: test set, evaluation, status, filters and queries, created and updated dates, application and variant, then cost, duration, tokens and errors, then one row per evaluator metric. On a phone the table is a two column list.
Overview, radar chart of the evaluator metrics, with duration and cost added when there are fewer than three evaluator metrics.
Overview, one chart card per evaluator metric: a True and False bar chart for boolean metrics, counts per label for categorical metrics, and a histogram for numeric metrics. Live runs show one time chart per metric instead.
While the run is not finished, the metrics refresh every 5 seconds, and they are fetched once more when the run finishes. A failed refresh keeps the cards and shows "Unable to refresh evaluator metrics. The scores below may be out of date." with "Try again".
Page states: skeleton while loading, "page not found" for an unknown run, and "Could not load this evaluation" with a retry when the run does not load.
Opening the page only reads. A test opens a run with test set, application and evaluator references, records every request, and checks that no PATCH, PUT or DELETE is sent.
Left out on purpose: the "Run actions" menu with "Edit evaluation" (it needs an edit mode in the new evaluation wizard), and comparing runs on the page (?compare=). The data functions take a list of run ids, so compare can be added later.
/w bugs fixed in /m
The /w page read the run kind from the type URL parameter. A human run opened without ?type=human showed no Annotate tab. /m reads the kind from the run.
/w writes to the run when you open it: when an annotation step has no evaluator revision reference, the run query sends a PATCH with the latest evaluator revisions. /m never writes on read.
"Cost (Total)", "Duration (Total)" and "Tokens (Total)" showed the per scenario average. /m labels them "(Avg)" and keeps the average.
"Insufficient metrics for spider chart" showed as a loading placeholder, so the page looked like it never finished loading. /m shows it as an empty state.
"Unable to load evaluator metrics" printed the raw error text. /m shows the title only and logs the detail.
A run that failed to load showed empty sections. /m shows an error with a retry.
Pause and Resume showed to readers without edit rights, and the API then refused. /m hides them.
Histogram labels were computed from the bin index, so every label after an empty bin that the API left out was wrong. /m uses each bin's own interval.
For live runs, /w sent the same metrics request twice. /m sends one request per set of runs.
Each fix has a test.
Package changes
@agenta/entities/evaluationRun: run header and references selectors, a batched run level metrics query that polls while the run is not finished and fetches once more when it finishes, a metric catalog, stat helpers for the charts (summary values, histogram and frequency bars, radar axes), and the scenario page query that the Scenarios tab uses. Reading a run never writes it.
web/storybook/.storybook/main.ts: Storybook resolves the /m @ alias, so stories of /m screens build.
New user-facing text
Strings not copied from /w:
"Cost (Avg)", "Duration (Avg)", "Tokens (Avg)".
"Could not load this evaluation".
"1 query" or "N queries" in the "Filters / Queries" row.
"Unable to refresh evaluator metrics. The scores below may be out of date." with "Try again".
The test files this PR adds or changes: 28 mobile tests and 26 @agenta/entities tests passed.
Recorded on the branch before the rebase:
Types: @agenta/mobile, @agenta/oss (/w) and @agenta/entities pass on the final head.
Full mobile tests: only agentPermissions.render.test.tsx and membersTabRoleChange.render.test.tsx fail. Both also fail on the release branch.
Entities unit tests: 2818 passed. One file, mcp-policy-ladder.test.ts, failed to run. It fails only on the remote test machine, because that copy has no .git.
next build and speed:check pass. The new route is 1340.7 KB on first load and has no budget entry yet. These ran before the last two fix commits.
Storybook screenshots of the page and Overview states at 1440 and 390 px, light and dark.
Not run: the page against a live API.
Review results
Round 1 rated slop Low, simplify Medium and Codex correctness Medium, and all seven Codex findings were fixed. Round 2 rated slop Medium and simplify Medium, and Codex did not run because the model was at capacity. A last Codex pass then rated Medium with three findings: a race in the final metrics fetch, a categorical line dropped when a value was missing from the latest point, and old scores shown with no warning after a failed refresh. All three are fixed with a test. Codex did not run again after those fixes.
Open problems
The overview has its own cost, duration, token and number formatters. The ones in @agenta/shared give different output for the same input, for example formatSignificant(100) returns "1", which looks like a bug in @agenta/shared. Swapping them is a follow-up.
API bug: PATCH /evaluations/runs/{id} without data sets the run's references to null, so queries that filter runs by reference stop finding the run. This PR has no edit call because of it. An issue is drafted, not filed yet.
Enable the view with AGENTA_DEVELOPER_VIEWS=evaluations. The /m runs list is a separate PR, so take a run id from the /w run page URL and open /w/<workspace>/p/<project>/evaluations/results/<run id> in /m.
Open a finished auto run. The Overview shows the summary table, the radar and one chart per evaluator metric. The cost, duration and tokens rows say "(Avg)".
Open a human run without any URL parameter. The Annotate tab is listed (disabled in this PR).
Open a live run as a project admin. Pause stops it and shows "Evaluation stopped". Resume starts it again. As a viewer, there is no Pause button.
Open a run with only one or two evaluator metrics. The radar shows "Insufficient metrics for spider chart" as an empty state, not a loader.
Open a run that is still running. The charts update every few seconds without a skeleton flash.
Watch the browser network panel while the page is open. It shows no PATCH, PUT or DELETE request.
Regression: open the same run in /w. It still loads as before.
Added an evaluation run details view with an overview of run metadata, evaluator scores, invocation metrics, and charts.
Added scenario, configuration, and annotation tabs where available, with tab selection reflected in the URL.
Added controls to pause and resume eligible online runs, with status feedback.
Added loading, empty, and error states, including retry options when metrics fail to load.
This change adds run-detail data queries and metric processing, then uses them to render an evaluation overview and a hidden mobile run-details page. The page includes run-specific tabs and permission-gated pause and resume controls for online runs.
Changes: Evaluation run details and overview
Layer / File(s)
Summary
Run data contracts and queries web/packages/agenta-entities/src/evaluationRun/api/*, web/packages/agenta-entities/src/evaluationRun/state/runDetails.ts, web/packages/agenta-entities/src/evaluationRun/index.ts, web/packages/agenta-entities/tests/unit/evaluation-run-details-api.test.ts, web/packages/agenta-entities/tests/unit/evaluation-run-header.test.ts
Adds validated scenario-page and run-level metrics queries, selectors for run headers and references, and public exports. Tests cover query payloads, cursor handling, and derived run data.
Metric normalization and catalog web/packages/agenta-entities/src/evaluationRun/core/metricStats.ts, web/packages/agenta-entities/src/evaluationRun/state/runMetrics.ts, web/packages/agenta-entities/tests/fixtures/evaluationRun/*metrics.json, web/packages/agenta-entities/tests/unit/evaluation-metric-stats.test.ts, web/packages/agenta-entities/tests/unit/evaluation-run-metrics.test.ts
Adds metric-stat normalization, summary and formatting helpers, chart data and radar-axis calculations. Adds a run-level metric catalog with polling and temporal points, plus fixtures and tests.
Overview data and summary rows web/mobile/src/features/evaluation-run/overview/useRunOverview.ts, web/mobile/src/features/evaluation-run/overview/runSummaryRows.tsx, web/mobile/src/features/evaluation-run/overview/RunOverviewView.tsx, web/mobile/src/features/evaluation-run/overview/RunSummaryTable.tsx, web/mobile/src/features/evaluation-run/overview/RunStatusTag.tsx, web/mobile/src/features/evaluation-run/overview/OverviewTab.tsx, web/mobile/tests/unit/runSummaryTable.render.test.tsx, web/mobile/tests/unit/useOverviewNames.render.test.tsx
Resolves run-related display names and builds ordered summary rows from run references and metrics. Composes the overview summary table and status tag.
Overview charts and display states web/mobile/src/features/evaluation-run/overview/MetricCard.tsx, web/mobile/src/features/evaluation-run/overview/MetricCardsSection.tsx, web/mobile/src/features/evaluation-run/overview/RunRadarCard.tsx, web/mobile/src/features/evaluation-run/overview/TemporalMetricCard.tsx, web/mobile/src/features/evaluation-run/states/*, web/mobile/tests/unit/runOverview*, web/storybook/stories/developer-views/RunOverview.stories.tsx
Adds evaluator distribution cards, online temporal charts, and radar charts. Adds loading, error, and empty states, with related tests, fixtures, and stories.
Run-details route and controls web/mobile/src/features/developer/developerViews.ts, web/mobile/src/features/evaluation-run/*, web/mobile/src/pages/w/[workspace_id]/p/[project_id]/evaluations/results/[evaluation_id].tsx, web/mobile/tests/unit/runDetailsShell.render.test.tsx, web/storybook/stories/developer-views/RunDetailsScreen.stories.tsx
Adds the hidden evaluations route and run-details screen. The screen selects tabs from the URL, renders loading and run-query states, and provides permission-gated controls to pause or resume eligible online runs.
sequenceDiagram
participant EvaluationRunPage
participant RunDetailsScreen
participant runHeaderAtomFamily
participant RunDetailsBody
participant OverviewTab
participant runLevelMetricsQueryAtomFamily
participant queryRunLevelMetrics
EvaluationRunPage->>RunDetailsScreen: Pass workspace, project, and run IDs
RunDetailsScreen->>runHeaderAtomFamily: Read header for runId
RunDetailsScreen->>RunDetailsBody: Render body for runId
RunDetailsBody->>OverviewTab: Render selected Overview tab
OverviewTab->>runLevelMetricsQueryAtomFamily: Read run metrics
runLevelMetricsQueryAtomFamily->>queryRunLevelMetrics: Request metrics for project and run
queryRunLevelMetrics-->>runLevelMetricsQueryAtomFamily: Return validated metrics
runLevelMetricsQueryAtomFamily-->>OverviewTab: Provide metrics state and catalog
Loading
Merge Risk: Low · previous review 620b9
The hidden run page remains usable, but an initial metrics failure requires a reload. The API integration and initial page load also need improvement; these are bounded issues to address or accept before merging.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title "feat(frontend): Evaluation run page and Overview tab in /m behind the developer view switch" names the main change. The changeset adds the evaluation run page with its Overview tab to /m, g…
Description check
The description states the purpose (adding the /m run page with its Overview tab), the hidden developer-view gating, the page behavior (title, Pause/Resume, tabs, summary table, radar chart, metric ch…
Docstring Coverage
Docstring coverage is 69.70% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentati…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title "feat(frontend): Evaluation run page and Overview tab in /m behind the developer view switch" names the main change. The changeset adds the evaluation run page with its Overview tab to /m, gated behind the hidden evaluations developer view. The summary also covers supporting package helpers, fixtures, and stories, which the title does not mention, but that is expected.
Full details: Description check
Explanation
The description states the purpose (adding the /m run page with its Overview tab), the hidden developer-view gating, the page behavior (title, Pause/Resume, tabs, summary table, radar chart, metric charts, refresh and error states), the /w bug fixes, package changes, tests, and open problems. All of these match the changeset summaries.
Full details: Docstring Coverage
Explanation
Docstring coverage is 69.70% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 33 functions across 40 files. (3 skipped: 3 unsupported.)
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 1 remain after this review.
Preview use finished or expired. Environment absence verified; retained storage requires separate provider accounting.
Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.
Run header and references selectors, a batched run-level metrics query
that polls while the run is not finished, a metric catalog, pure stat
helpers for the overview charts, and the scenario page, run metrics and
run edit API calls. Reading a run never writes it.
A hidden developer view at /evaluations/results/[evaluation_id]: the run
title, Pause and Resume for live runs, the tabs in ?view=, and the
Overview tab with the summary table, the radar and one chart per
evaluator metric. Opening the page only reads the run.
Metrics query follows the project and the run reactively, so polling
stops when the run is gone and starts again after a resume. Radar
fallback counts only axes with values and picks units from the metric
type. Online categorical metrics draw one line per value, and online
cards name their evaluator. Drop the unused run edit call: a PATCH
without data clears the run references on the API.
The comment restates what the code shows across several lines.
Proposed fix
-/**- * One card per evaluator metric: a distribution for a finished run, a time chart for an online- * run. Invocation metrics stay in the summary table.- */+/** One card per evaluator metric: a distribution, or a time chart for online runs. */
As per coding guidelines: "At most ONE short line per comment. No multi-line blocks narrating why, no restating what the code shows."
The three-line block restates the branches of timeChart.
Proposed fix
-/**- * Boolean: the share of true, 0..1. Numeric: the mean (else median, else max). Categorical: one- * line per frequent value, with its count.- */+/** Boolean share (0..1), numeric summary, or one count line per frequent category. */
As per coding guidelines: "At most ONE short line per comment. No multi-line blocks narrating why, no restating what the code shows."
Route the new backend calls through the Fern client.
queryEvaluationScenariosPage and queryRunLevelMetrics are new backend functions. They use raw axios.post and Axios-style {params: {project_id}}, which violates the frontend API convention. Use getEvaluationsClient() with its generated scenario and metrics query methods, passing {queryParams: {project_id: projectId}}.
Keep the local Zod validation and the parseOrThrow failure contract from the file header. Implement the validation through safeParseWithLogging so structured validation errors are logged before the wrapper throws. Update web/packages/agenta-entities/tests/unit/evaluation-run-details-api.test.ts to mock the Fern client instead of axios.post.
Load OverviewTab with next/dynamic to reduce the route's first-load size.
RunDetailsBody statically imports OverviewTab, which reaches the overview chart components. Use an overview-only loading fallback instead of RunDetailsSkeleton, because the parent already renders the tab bar.
♻️ Proposed change
-import {OverviewTab} from "./overview/OverviewTab"+import dynamic from "next/dynamic"+
import {RunDetailsSkeleton} from "./states/RunDetailsSkeleton"
+import {ChartSkeleton} from "./states/ChartSkeleton"
import {useRunDetailsView, type RunDetailsView} from "./useRunDetailsView"
import {PageNotFound} from "@/components/PageNotFound"
++const OverviewTab = dynamic(+ () => import("./overview/OverviewTab").then((module) => module.OverviewTab),+ {loading: () => <ChartSkeleton label="Loading evaluation overview" height={300} />},+)
Condense the three-sentence race comment at lines 87-89.
The block at lines 87-89 has three sentences. The web/ comment rule allows at most two sentences for a documented race or ordering constraint. The block at lines 79-80 is one sentence and can stay as written.
🔧 Suggested fix
- // Refresh the run too, so the poll sees it finish. When it finished since the last- // look, the numbers above may be older than its last scenario: fetch them once- // more. A run that was not loaded yet gets no second fetch.+ // Refresh the run so the poll sees a finish. If it finished since the last look,+ // the metrics above may be stale: fetch them once more, unless the run was not loaded yet.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Do not merge into v0.128.4. This goes into the next release.
Context
The /m app has no page for a single evaluation run, so users still open the old /w run page to see the results of a run. This PR adds the run page to /m with its Overview tab. It is the first of four stacked PRs for the run page. The Scenarios tab, the Configuration tab and the scenario drawer come in the PRs stacked on this one.
It ships hidden
Nothing here is reachable until the
evaluationsdeveloper view is enabled. The PR adds the route/evaluations/results/[evaluation_id]to that view, with stagehidden, and no navigation link. While the view is hidden, the route shows "page not found". The PR adds no dependency, no antd, and nothing in_app,AppProviders,GlobalDrawersor the app shell. The charts are the SVG charts from@agenta/ui(#8011), and they load only on this route.What the page does
edit_evaluation_runs. Toasts "Evaluation stopped", "Evaluation resumed", "Failed to stop evaluation", "Failed to resume evaluation".?view=. In this PR only Overview has a body, and the other tabs show as disabled. The kind of run comes from the run itself, not from a URL parameter.Opening the page only reads. A test opens a run with test set, application and evaluator references, records every request, and checks that no PATCH, PUT or DELETE is sent.
Left out on purpose: the "Run actions" menu with "Edit evaluation" (it needs an edit mode in the new evaluation wizard), and comparing runs on the page (
?compare=). The data functions take a list of run ids, so compare can be added later./w bugs fixed in /m
typeURL parameter. A human run opened without?type=humanshowed no Annotate tab. /m reads the kind from the run.Each fix has a test.
Package changes
@agenta/entities/evaluationRun: run header and references selectors, a batched run level metrics query that polls while the run is not finished and fetches once more when it finishes, a metric catalog, stat helpers for the charts (summary values, histogram and frequency bars, radar axes), and the scenario page query that the Scenarios tab uses. Reading a run never writes it.web/storybook/.storybook/main.ts: Storybook resolves the /m@alias, so stories of /m screens build.New user-facing text
Strings not copied from /w:
Checks
release/v0.128.4(atd5bc67afec, with feat(frontend): evaluators and archived evaluators in /m behind the developer view switch #8007 and fix(frontend): pass the leave question to the shared unsaved guard in the test set editor #8031):@agenta/mobileand@agenta/oss(/w) pass.@agenta/entitiestests passed.Recorded on the branch before the rebase:
@agenta/mobile,@agenta/oss(/w) and@agenta/entitiespass on the final head.agentPermissions.render.test.tsxandmembersTabRoleChange.render.test.tsxfail. Both also fail on the release branch.mcp-policy-ladder.test.ts, failed to run. It fails only on the remote test machine, because that copy has no.git.RunDetailsScreen.tsxand two test files). They are fixed in the stacked Scenarios PR feat(frontend): Evaluation run Scenarios tab in /m behind the developer view switch #8045.next buildandspeed:checkpass. The new route is 1340.7 KB on first load and has no budget entry yet. These ran before the last two fix commits.Review results
Round 1 rated slop Low, simplify Medium and Codex correctness Medium, and all seven Codex findings were fixed. Round 2 rated slop Medium and simplify Medium, and Codex did not run because the model was at capacity. A last Codex pass then rated Medium with three findings: a race in the final metrics fetch, a categorical line dropped when a value was missing from the latest point, and old scores shown with no warning after a failed refresh. All three are fixed with a test. Codex did not run again after those fixes.
Open problems
@agenta/sharedgive different output for the same input, for exampleformatSignificant(100)returns "1", which looks like a bug in@agenta/shared. Swapping them is a follow-up.PATCH /evaluations/runs/{id}withoutdatasets the run'sreferencesto null, so queries that filter runs by reference stop finding the run. This PR has no edit call because of it. An issue is drafted, not filed yet.evaluationsentry todeveloperViews.ts. Whichever PR merges second must merge the two entries into one with both routes.How to QA once enabled
Enable the view with
AGENTA_DEVELOPER_VIEWS=evaluations. The /m runs list is a separate PR, so take a run id from the /w run page URL and open/w/<workspace>/p/<project>/evaluations/results/<run id>in /m.