You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(frontend): metric, cost and per-tab columns in the /m evaluations list - #8185
Stacks on #8164 (shared evaluation logic). Base: feat/m-parity-evaluation-logic.
Context
The /m Evaluations list showed the same seven columns on every tab: Status, Name, Kind, Application, Test set, Created on, Created by. It had no scores, no cost and no duration, so a user had to open each run to see if it passed. On Live Evals, Application and Test set were empty and nothing said which traces a run watches. The test set and app cells also had no version, so "default v3" and "verbose v5" looked the same.
The old app (/w) shows one column per evaluator metric, the invocation metrics, and columns that follow the tab.
Changes
The list now picks its columns the way /w does, from the tab and the runs on screen:
One column per evaluator metric, labelled with the evaluator and the metric (for example Exact Match · Success). Boolean metrics show the true/false bar with the share of true ("100.0%"), as in /w. Other metrics show the /w cell text from formatMetricValue. Metrics the evaluator's output schema types as text (a comment) get no column, as in /w.
Cost, Duration, Tokens and Errors for every tab.
Reference columns follow the runs: Test set and Application on Auto, Human and SDK; Query on Live; all that apply on All. The Kind column shows on All only.
Test set cells show the revision ("Capitals v3"). Application cells show the variant and revision under the app ("Capital finder", then "verbose v5").
The Query cell prints the live run's trace filter, one condition per line ("Trace Type is invocation"), with the same labels as the /w preview.
Speed: the metric and query cells load once a row is near the screen. The rows that ask in the same frame share one request (runLevelMetricsQueryAtomFamily and a new batched query revision atom), so a page of 30 runs makes one metrics request, not one per cell. Phones (under 640 px) keep the current layout (status, name, kind and date, menu) and load no metrics.
Shared logic added to @agenta/entities/evaluationRun, with tests:
runListColumns(runs, tab): the column plan (reference columns, evaluator groups and their metrics per run step, invocation columns), copied from /w useEvaluationRunsColumns and referenceSchema.
runMetricCellView: the /w rule for a bar, a top category or text.
isTextOutputMetric: the /w rule that hides text metrics.
buildRunStatsMap(run, metrics, "latest"): a live run's list cell reads the newest window, as /w does. The run page keeps the static row (the default).
flattenQueryConditions, queryLiveQueryRevisions and liveQueryRevisionQueryAtomFamily for the Query column.
The app overview runs table passes no tab, so it keeps its basic columns without metrics.
Tests
New evaluation-run-list-columns.test.ts (entities): column plans for the Auto, Live, All and empty tabs on runs captured from a test stack, metric labels, cell views, the latest row and the text metric rule.
New evaluationRunsColumns.render.test.tsx (mobile): headers per tab, cell text (0.85, 70.0%, $0.0012, 522ms), one metrics request for three rows, test set and variant versions, the Live query line, text metrics hidden, and phones load no metrics.
Type checks for @agenta/mobile, @agenta/entities and @agenta/oss; eslint and prettier on changed files.
Checked in the browser against the parity stack on the Auto, Human, Live and All tabs and at 390 px.
Note: a run that has not finished (a live run, or a human run waiting for annotations) refreshes its metrics every 5 seconds while its row is on screen. This is the existing behavior of runLevelMetricsQueryAtomFamily; the refreshes of the rows on screen are batched.
What to QA
Evaluations, Auto Evals tab on a wide screen: one column per evaluator metric with the /w values, then Cost, Duration, Tokens and Errors. The table scrolls sideways.
Test set shows "Capitals v3"; Application shows the app and, under it, "verbose v5".
Live Evals tab: a Query column with "Trace Type is invocation", no Test set or Application column, and the live evaluator metrics.
All Evals tab: the Kind column comes back, with the reference columns of every run on the page.
Human Evals tab: no column for a text metric such as a comment.
Regression: at 390 px the list looks as before (status, name with kind and date, menu).
…evaluations list
The runs list now picks its columns like the old list: one column per
evaluator metric, cost, duration, tokens and errors, reference columns
that follow the tab (Query on Live, Kind on All), and test set and
variant revisions in the cells. Metrics and queries load for rows near
the screen in one batched request.
Shared column plan, cell rule and query revision fetch live in
@agenta/entities/evaluationRun with tests.
Entire-Checkpoint: 01M6RC8BT7P134CEN7ZCZTVXG8
Evaluation run tables now show columns tailored to the selected tab and available run data, including application, test set, query, evaluator, and invocation details.
View application and test set revisions, query filters and sampling rates, and readable evaluator metrics in the run list.
Live evaluation metrics use the latest available results.
Bug Fixes
Text-based evaluator outputs no longer appear as metric columns.
Narrow-screen run lists remain focused on essential details and avoid loading metrics.
The evaluation runs table now derives wide-screen columns from run data and the selected tab. It displays configured references and evaluator or invocation metrics, and loads query revisions and metrics as rows approach the viewport.
Changes: Evaluation run list
Layer / File(s)
Summary
Column planning and metric selection web/packages/agenta-entities/src/evaluationRun/core/runListColumns.ts, web/packages/agenta-entities/src/evaluationRun/core/runStatsMap.ts, web/packages/agenta-entities/src/evaluationRun/core/index.ts, web/packages/agenta-entities/src/evaluationRun/index.ts, web/packages/agenta-entities/tests/unit/evaluation-run-list-columns.test.ts
Run-list helpers derive reference and metric columns from run data, identify text-output metrics, and select metric cell views. Run-level statistics can prefer the latest timestamped row or the static row. Unit tests cover column selection, metric display, and row selection.
The query revision API validates revision responses and skips requests when no IDs are supplied. The state atom batches revision IDs by project. Helpers flatten stored query conditions and format them for display.
Tab-specific column planning and table wiring web/mobile/src/features/evaluations/list/useRunListColumns.ts, web/mobile/src/features/evaluations/list/useEvaluatorColumns.ts, web/mobile/src/features/evaluations/list/EvaluationRunsTable.tsx, web/mobile/src/features/evaluations/list/EvaluationsScreen.tsx
The selected evaluation kind is passed to the table. The table builds wide columns and minimum width from the column plan, and evaluator metadata filters text-output metrics.
Configured run references and metrics web/mobile/src/features/evaluations/list/RunRowCells.tsx, web/mobile/src/features/evaluations/list/RunApplicationCell.tsx, web/mobile/src/features/evaluations/list/RunTestsetCell.tsx, web/mobile/src/features/evaluations/list/RunVersionChip.tsx, web/mobile/src/features/evaluations/list/RunQueryCell.tsx, web/mobile/src/features/evaluations/list/RunMetricCells.tsx, web/mobile/src/features/evaluations/list/RunMetricValue.tsx, web/mobile/src/features/evaluations/list/runReferences.ts, web/mobile/tests/unit/evaluationRunsColumns.render.test.tsx
Rows render references and metrics selected by the column plan. Query and metric cells use row visibility to control loading. Render tests cover tab columns, displayed values, batched metric requests, and the phone layout.
sequenceDiagram
participant EvaluationsScreen
participant EvaluationRunsTable
participant useRunListColumns
participant runListColumns
participant RunRowCells
participant RunMetricCells
EvaluationsScreen->>EvaluationRunsTable: Pass selected evaluation kind and runs
EvaluationRunsTable->>useRunListColumns: Build the selected tab column plan
useRunListColumns->>runListColumns: Derive columns from runs and tab
runListColumns-->>useRunListColumns: Return references and metric columns
useRunListColumns-->>EvaluationRunsTable: Provide column plan
EvaluationRunsTable->>RunRowCells: Render each run with the plan
RunRowCells->>RunMetricCells: Render configured metric cells
Loading
Possibly related PRs
Agenta-AI/agenta#8040: Adds the evaluation runs table that this change extends with tab-specific reference and metric columns.
Agenta-AI/agenta#8042: Adds run-level metrics queries and statistics helpers used by the metric cells.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title identifies the main change: metric and tab-specific columns in the /m Evaluations list.
Description check
The description matches the changes to columns, metrics, references, query data, and tests.
Docstring Coverage
Docstring coverage is 100.00% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
Explanation
Docstring coverage is 100.00% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 9 functions across 24 files.
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Dependent tests finished. Post /preview for manual review. Environment absence verified; retained storage requires separate provider accounting.
Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.
Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 2 remain after this review.
…heir own chunk
The metric bar, the run stats and the trace filter labels moved the /m
evaluations page from 1063 KB to 1078.5 KB. They now load with React.lazy
the first time a wide row draws them; phones never load them. The page is
1066.1 KB.
Entire-Checkpoint: 01M6RT95WR2E2G05H4X5YBJNXT
…olumns' into feat/m-parity-evaluations-list-filters
Takes the #8185 speed fix and Fern query revisions. queryLiveQueries moves
to the Fern client too, since the axios import is gone from that file.
Entire-Checkpoint: 01M6RWAR1W6M4H6RA55XKXG8BY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Stacks on #8164 (shared evaluation logic). Base:
feat/m-parity-evaluation-logic.Context
The /m Evaluations list showed the same seven columns on every tab: Status, Name, Kind, Application, Test set, Created on, Created by. It had no scores, no cost and no duration, so a user had to open each run to see if it passed. On Live Evals, Application and Test set were empty and nothing said which traces a run watches. The test set and app cells also had no version, so "default v3" and "verbose v5" looked the same.
The old app (/w) shows one column per evaluator metric, the invocation metrics, and columns that follow the tab.
Changes
The list now picks its columns the way /w does, from the tab and the runs on screen:
Exact Match · Success). Boolean metrics show the true/false bar with the share of true ("100.0%"), as in /w. Other metrics show the /w cell text fromformatMetricValue. Metrics the evaluator's output schema types as text (a comment) get no column, as in /w.Speed: the metric and query cells load once a row is near the screen. The rows that ask in the same frame share one request (
runLevelMetricsQueryAtomFamilyand a new batched query revision atom), so a page of 30 runs makes one metrics request, not one per cell. Phones (under 640 px) keep the current layout (status, name, kind and date, menu) and load no metrics.Shared logic added to
@agenta/entities/evaluationRun, with tests:runListColumns(runs, tab): the column plan (reference columns, evaluator groups and their metrics per run step, invocation columns), copied from /wuseEvaluationRunsColumnsandreferenceSchema.runMetricCellView: the /w rule for a bar, a top category or text.isTextOutputMetric: the /w rule that hides text metrics.buildRunStatsMap(run, metrics, "latest"): a live run's list cell reads the newest window, as /w does. The run page keeps the static row (the default).flattenQueryConditions,queryLiveQueryRevisionsandliveQueryRevisionQueryAtomFamilyfor the Query column.The app overview runs table passes no tab, so it keeps its basic columns without metrics.
Tests
evaluation-run-list-columns.test.ts(entities): column plans for the Auto, Live, All and empty tabs on runs captured from a test stack, metric labels, cell views, the latest row and the text metric rule.evaluationRunsColumns.render.test.tsx(mobile): headers per tab, cell text (0.85,70.0%,$0.0012,522ms), one metrics request for three rows, test set and variant versions, the Live query line, text metrics hidden, and phones load no metrics.@agenta/mobile,@agenta/entitiesand@agenta/oss; eslint and prettier on changed files.Note: a run that has not finished (a live run, or a human run waiting for annotations) refreshes its metrics every 5 seconds while its row is on screen. This is the existing behavior of
runLevelMetricsQueryAtomFamily; the refreshes of the rows on screen are batched.What to QA