You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(frontend): Evaluation runs list in /m behind the developer view switch - #8040
Do not merge into v0.128.4. This goes into the next release.
Context
The /m app has no page for evaluation runs, so users still go to the old /w table for them. This PR adds the evaluation runs list to /m. It is the first part of the evaluations work. The new evaluation wizard is stacked on this branch, and the run details page is #8042.
It ships hidden
Nothing here is reachable until the evaluations developer view is enabled. The view adds an "Evaluations" link under Developer in the navigation and the route /evaluations. While the view is hidden, the route shows "page not found". The PR adds no dependency, no antd, and nothing in _app, AppProviders, GlobalDrawers or the app shell. The only change outside features/evaluations/ is that useConfirmModal now accepts rich content as the message, which the delete confirm uses.
What the list does
Header "Evaluations" with an app filter. Picking an app writes ?app=<id> to the URL. In app scope the "Live Evals" tab is hidden.
Kind tabs "All Evals", "Auto Evals", "Human Evals", "Live Evals" and "SDK Evals", kept in ?kind=.
Search (kept in ?q=) and a status filter (kept in ?status=). Both are sent to the API.
Rows in pages of 30 with a "Load more" button. Wide screens show status, name, kind, application, test set, created on and created by. Phones show status, name with the kind and date under it, and the row menu.
The list refreshes every 5 seconds while an auto or SDK run is in progress, and not in a background tab.
Row menu: "Open details", "Copy run ID", "Stop evaluation" or "Resume evaluation" for live runs, "View variant" (opens the shared revision drawer), "View test set" (only while the testsets view is on), and "Delete" with a confirm.
Empty states per tab with the /w text, without the videos. A no-match state, an error state with retry, and a skeleton while loading.
Readers without edit_evaluation_runs see no delete, no stop or resume, and no create button. The page has no create button yet. The wizard PR adds it.
Left out on purpose: metric columns, CSV export, column visibility, bulk delete, the evaluator, test set, variant, query and date filters, and "Edit evaluation".
/w bugs fixed in /m
Search only filtered the rows already loaded. /m sends the search to the API.
A kind tab could look empty while runs of that kind existed. /m sends the kind to the API, and the package fills the page.
A queued run never started the 5 second refresh. /m refreshes for every run in progress.
The delete guard compared upper-case statuses to a lower-case set, so it never blocked anything. /m allows delete at any status, which is what /w does in practice.
A failed load showed the onboarding empty state. /m shows "Could not load evaluations" with a retry.
A search or status filter with no match showed the onboarding empty state. /m shows "No evaluations match your filters" with "Clear filters".
Delete, stop, resume and create showed to readers without edit rights, and the API refused with a generic message. /m hides them.
Live runs read "Online" in the kind column and "Live" everywhere else. /m says "Live" everywhere.
Each fix has a test.
Package changes
None.
New user-facing text
Strings not copied from /w:
"No evaluations match your filters", "Try a shorter search or fewer statuses.", "Clear filters".
"No matching evaluations in the newest runs", "Load more to look through older runs." (a filtered page that is empty while older runs remain).
"Could not load evaluations", "Couldn't load more evaluations", "Couldn't copy the run ID".
"New evaluation" on every tab (/w says "New Evaluation" on one of them).
The auto evaluation description says "test sets" in two words, as the Test sets list does.
Checks
Types: @agenta/mobile passes. No package changed, so /w types were not run.
Lint passes.
Full mobile tests: 1050 run, 2 failed. The two failures are agentPermissions.render.test.tsx and membersTabRoleChange.render.test.tsx. Both also fail on the release branch. The last full run was before the last two commits (the empty filtered page fix and a shorter status filter on phones). After those commits the screen test file (22 tests) and the /m types were run again and pass.
The row action tests use the real delete, stop and start functions from the package and mock only HTTP. They assert the method, the path and the body.
Storybook screenshots were taken for 11 states at 1440 and 390 px, light and dark (44 images). The capture reported no render errors and no horizontal overflow.
Types: @agenta/mobile passes. This PR does not change web/packages.
The test files this PR adds or changes: 53 passed.
Review results
Two rounds with the slop, simplify and Codex correctness lenses. Round 1 rated slop Medium, simplify Low and correctness Medium. The round 1 fix is the "Load more to look through older runs" state. Round 2 on the final head (b0f987ec21) rated all three Low.
One round 1 finding was rejected. "Open details" goes to /evaluations/results/{id}, which the run details PR #8042 adds under the same gate.
"View test set" opens /testsets/{id}. The test set detail route arrives with the Test set detail PRs.
Stop and Resume treat a live run as active when flags.is_active is true and flags.is_closed is not. is_live is left out because every live run has it. Please confirm this rule.
The full screen has no Storybook story because its shell needs the router. The stories cover the table, the toolbar and the menus.
When the view goes to on, the /w per-app path needs a mobileRouteFor entry that maps to /evaluations?app=<id>.
How to QA once enabled
Enable the view with AGENTA_DEVELOPER_VIEWS=evaluations. Then:
Open Developer > Evaluations in a project with auto and human runs. The rows show status, name, kind, application, test set and created on and by.
Search for part of a run name that is not on the first page. The run appears.
Pick a status filter that matches nothing. You see "No evaluations match your filters", and "Clear filters" brings the rows back.
Pick an app in the app filter. The URL gets ?app=, the "Live Evals" tab goes away, and only that app's runs show.
Start an auto run from /w and open the list. Its status updates without a reload.
Delete a run. The confirm names the run, and the row goes away after "Deleted successfully".
Open the page as a viewer. There is no Delete, no Stop or Resume, and no create button.
Regression: with the view off, /evaluations shows "page not found" and the navigation has no Evaluations link.
Reviewing files that changed from the base of the PR and between 417f855 and 417f855.
You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.
Use the checkbox below for a quick retry:
Trigger review
Summary
Summary by CodeRabbit
New Features
Added an Evaluations view with a searchable, filterable list of evaluation runs, including app-level views and pagination.
Added controls to create evaluations and manage runs, including opening details, copying IDs, navigating to related items, and stopping, resuming, or deleting eligible runs.
Added responsive layouts, status indicators, and helpful empty states for evaluations.
Tests
Added coverage for evaluation lists, filters, run actions, permissions, and the evaluations page.
The change adds a developer-only evaluations page. It loads evaluation runs with URL-backed filters and pagination, displays responsive run tables and empty states, and provides creation and row-action controls.
Changes: Evaluation runs
Layer / File(s)
Summary
Filters and paginated run loading web/mobile/src/features/evaluations/list/useEvaluationRunsFilters.ts, web/mobile/src/features/evaluations/list/useEvaluationRunsList.ts, web/mobile/tests/unit/evaluationRunsFixtures.ts, web/mobile/tests/unit/evaluationRunsList.test.tsx
URL parameters control kind, app, search, and status filters. The list query fetches pages of 30 runs, seeds the run cache, and polls every five seconds while loaded runs are in progress. Tests cover filter parsing, pagination, cache seeding, and polling.
Run references and actions web/mobile/src/features/evaluations/list/runReferences.ts, web/mobile/src/features/evaluations/list/useRunRowActions.tsx, web/mobile/src/features/evaluations/list/RunRowMenu.tsx, web/mobile/src/features/settings/useConfirmModal.tsx, web/mobile/tests/unit/evaluationRunRowActions.render.test.tsx
Run references support navigation to application revisions and test sets. Row actions copy IDs, stop or resume live runs, and delete runs after confirmation. The menu reflects permissions, capabilities, and run state.
List screen and responsive presentation web/mobile/src/features/evaluations/list/*, web/mobile/tests/unit/evaluationsScreen.render.test.tsx, web/storybook/stories/developer-views/EvaluationRunsTable.stories.tsx
The screen combines kind tabs, search and status controls, creation options, a responsive table, and filter-aware empty states. Tests and stories cover loading, app scope, permissions, rows, empty states, and pagination.
Developer view and page route web/mobile/src/features/developer/developerViews.ts, web/mobile/src/pages/w/[workspace_id]/p/[project_id]/evaluations/index.tsx, web/mobile/tests/unit/evaluationsRoute.render.test.tsx
The developer navigation registers the hidden Evaluations view. The route renders the screen inside the developer-view gate when both workspace and project IDs are available.
sequenceDiagram
actor User
participant EvaluationKindTabs
participant EvaluationsScreen
participant useEvaluationRunsFilters
participant useEvaluationRunsList
participant EvaluationRunsTable
User->>EvaluationKindTabs: Select a kind
EvaluationKindTabs->>EvaluationsScreen: Call onChange with the selected kind
EvaluationsScreen->>useEvaluationRunsFilters: Set the URL-backed kind
useEvaluationRunsFilters-->>EvaluationsScreen: Return filter state
EvaluationsScreen->>useEvaluationRunsList: Load runs with current filters
useEvaluationRunsList-->>EvaluationsScreen: Return runs and pagination state
EvaluationsScreen->>EvaluationRunsTable: Render runs
Loading
Merge Risk: Low · up to 417f8
A failed deletion closes its confirmation dialog, so retrying requires reopening it. This is a bounded issue to fix or accept before merging.
Pre-merge checks: 4 · 1
Failed checks: 1 warning
Check name
Status
Explanation
Resolution
Docstring Coverage
Docstring coverage is 0.00% which is insufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed checks: 4 passed
Check name
Status
Explanation
Title check
The title clearly identifies the main change: adding the evaluation runs list in /m behind the developer view switch.
Description check
The description is directly related to the changeset and explains the route, filters, row actions, permissions, testing, and known integration items.
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
Explanation
Docstring coverage is 0.00% which is insufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 23 files.
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 3 remain after this review.
Preview use finished or expired. Environment absence verified; retained storage requires separate provider accounting.
Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.
The web frontend guidelines set a hard limit of one short line per comment. This block narrates behavior in four lines. Similar multi-line blocks appear in useRunRowActions.tsx, RunRowMenu.tsx, RunRowCells.tsx, EvaluationsScreen.tsx, EvaluationsEmpty.tsx, EvaluationsKeepLooking.tsx and CreateEvaluationMenu.tsx. Cut each to one line. Keep a longer comment only where it documents a surprising constraint.
♻️ Example
-/**- * The runs list, 30 at a time. Search, status, kind and app go to the server (the package fills- * a page that the kind or app filter would leave short). The list refreshes every five seconds- * while a run is in progress.- */+/** The runs list, 30 per page. Filters go to the server. Polls while a run is in progress. */
As per coding guidelines: "Keep in-code comments terse ... At most ONE short line per comment."
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Do not merge into v0.128.4. This goes into the next release.
Context
The /m app has no page for evaluation runs, so users still go to the old /w table for them. This PR adds the evaluation runs list to /m. It is the first part of the evaluations work. The new evaluation wizard is stacked on this branch, and the run details page is #8042.
It ships hidden
Nothing here is reachable until the
evaluationsdeveloper view is enabled. The view adds an "Evaluations" link under Developer in the navigation and the route/evaluations. While the view is hidden, the route shows "page not found". The PR adds no dependency, no antd, and nothing in_app,AppProviders,GlobalDrawersor the app shell. The only change outsidefeatures/evaluations/is thatuseConfirmModalnow accepts rich content as the message, which the delete confirm uses.What the list does
?app=<id>to the URL. In app scope the "Live Evals" tab is hidden.?kind=.?q=) and a status filter (kept in?status=). Both are sent to the API.testsetsview is on), and "Delete" with a confirm.Readers without
edit_evaluation_runssee no delete, no stop or resume, and no create button. The page has no create button yet. The wizard PR adds it.Left out on purpose: metric columns, CSV export, column visibility, bulk delete, the evaluator, test set, variant, query and date filters, and "Edit evaluation".
/w bugs fixed in /m
Each fix has a test.
Package changes
None.
New user-facing text
Strings not copied from /w:
Checks
Types:
@agenta/mobilepasses. No package changed, so /w types were not run.Lint passes.
Full mobile tests: 1050 run, 2 failed. The two failures are
agentPermissions.render.test.tsxandmembersTabRoleChange.render.test.tsx. Both also fail on the release branch. The last full run was before the last two commits (the empty filtered page fix and a shorter status filter on phones). After those commits the screen test file (22 tests) and the /m types were run again and pass.The row action tests use the real delete, stop and start functions from the package and mock only HTTP. They assert the method, the path and the body.
Storybook screenshots were taken for 11 states at 1440 and 390 px, light and dark (44 images). The capture reported no render errors and no horizontal overflow.
Not run: the list against a live API.
After the rebase onto
release/v0.128.4(atd5bc67afec, with feat(frontend): evaluators and archived evaluators in /m behind the developer view switch #8007 and fix(frontend): pass the leave question to the shared unsaved guard in the test set editor #8031):@agenta/mobilepasses. This PR does not changeweb/packages.Review results
Two rounds with the slop, simplify and Codex correctness lenses. Round 1 rated slop Medium, simplify Low and correctness Medium. The round 1 fix is the "Load more to look through older runs" state. Round 2 on the final head (
b0f987ec21) rated all three Low.One round 1 finding was rejected. "Open details" goes to
/evaluations/results/{id}, which the run details PR #8042 adds under the same gate.Open problems
evaluationsentry todeveloperViews.ts. Whichever PR merges second must merge the two entries into one, with both routes and the nav link./testsets/{id}. The test set detail route arrives with the Test set detail PRs.flags.is_activeis true andflags.is_closedis not.is_liveis left out because every live run has it. Please confirm this rule.on, the /w per-app path needs amobileRouteForentry that maps to/evaluations?app=<id>.How to QA once enabled
Enable the view with
AGENTA_DEVELOPER_VIEWS=evaluations. Then:?app=, the "Live Evals" tab goes away, and only that app's runs show./evaluationsshows "page not found" and the navigation has no Evaluations link.