You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(frontend): /w steps and tables in the /m new evaluation sheet - #8149
In /m developer mode, "New evaluation" opened a narrow drawer with an accordion. Each step was a short popover list: apps without their type or date, revisions without model or deployment status, test sets without dates or commit messages, and evaluators without the human feedback fields. There was no "Create new evaluator", the test set list showed the author as a raw user id, and one Escape dropped every pick without asking. /w shows all of this in a wide modal with a step list and a table per step.
Changes
The sheet keeps the /m drawer, but it is now as wide as the /w modal (1200 px, at most 90% of the window). The name sits on top, the step list on the left (Application, Revision, Test set, Evaluators, Advanced Settings, each with a check and chips of the picks), and the open step's table on the right. Below 768 px the step list sits above the table; at 390 px the page does not scroll sideways.
Each step now shows what /w shows:
Application: a table with Name, Kind, Type (Completion, Chat) and Created At, a type filter and a search.
Revision: one row per variant with its newest revision: name, version tag, "Last modified", deployment dots, Model, Created on, Created by and Commit notes. Older revisions open under it. An auto run picks several revisions with checkboxes; a human run picks one.
Test set: the expected input variables, then an inline table with Name, Commit Message, Date Created and Created by. A pick takes the latest revision and moves to Evaluators, as in /w.
Evaluators: the /w columns (Type, Tags, dates, Modified by, Commit message), or for a human run the Feedback column ("score, comment", "approved"). Older versions open under each evaluator. "Create new evaluator" opens the Evaluators page on the right tab in a new tab, so the draft stays. (The in-place evaluator drawer is separate work.)
Authors show as names: the sheet loads the workspace members the same way the other /m screens do. No new API.
Close: Escape, the close button, a click outside and Cancel now ask "Discard this evaluation?" when the draft holds input (a name, a pick, changed settings). An untouched draft closes at once. Picks the playground passed in do not count as input.
Kept as they were: the auto name, the Advanced Settings defaults (10, 3, 3), the validation toasts and the start flow.
The playground opens the same NewEvaluationSheet, so it gets the same layout with Application and Revision filled in and the test set table on screen. No playground file changes.
The sheet still loads with next/dynamic from the runs list and from the playground. It uses no antd and adds no dependency. It reuses the /m data hooks of the Variants, Test sets and Evaluators pages (useAppRevisions, useTestsetsList, useEvaluatorEntries, useEvaluatorRevisions) and the shared ListTable. Those files are not changed.
Tests / notes
Render tests over the sheet and each step: newEvaluationSheet, newEvaluationSections and newEvaluationEvaluators (.render.test.tsx), plus pure tests in newEvaluationDraft.test.ts for draftHasInput, the step table tree and the app type filter. Against the old code, 36 of these tests fail.
playgroundNewEvaluation.render.test.tsx runs the real stores with HTTP mocked. It now checks that the sheet opens on Test set and that the revision table shows both playground revisions as checked.
/m without developer mode does not change. Evaluations is a hidden view. The new test developerViews.render.test.tsx > "Evaluations stay off without beta developer mode or a forced view, even for a preview opt-in" proves the view is off, and the existing playground test "Stays hidden while the evaluations view is off" covers the other entry point. Checked live with beta off and no forced views: no Evaluations item in the nav, /evaluations shows the 404 page, and the playground has no "New Evaluation".
Speed: the speed:check size budget passes. The app shell is 884.6 KB. Every route, this one included, is about 2 KB over the budget baseline, which is drift on main and not from this change. The new tables are in the sheet's dynamic chunk.
Types, pnpm lint-fix and the full @agenta/mobile suite were run. One unrelated test, appActionsMenu.render.test.tsx, failed once under load and passes when run again.
Live check on a parity stack: one real auto run (Capital finder, Capitals, Exact Match) started from the new sheet and finished. It was then deleted.
Not in this PR: the /w "Show evaluators" toggle on the Application step (it evaluates evaluators) and the /w test set filter button. /w also shows a raw user id for test set authors; /w code is not changed here.
What to QA
Evaluations > New evaluation > Auto evaluation. The step list is on the left and the app table is on the right, with Type and Created At. Filter by Chat: only chat apps remain.
Pick an app. The revision table shows the model, the author name and the commit notes. Open the older revisions with the caret and pick two revisions.
Test set: the table shows dates, commit messages and author names (not ids). Click a test set: its chip shows the latest version and the Evaluators step opens.
Evaluators: "Create new evaluator" opens the Evaluators page in a new tab, and the draft is still there.
Human evaluation > Evaluators: the Feedback column lists each evaluator's fields.
Pick something, then press Escape: you are asked to confirm. Press Cancel and the picks stay. Open a fresh sheet and press Escape: it closes at once.
Playground (Capital finder) > New Evaluation: Application and Revision are filled in and the test set table shows. Start a run.
Narrow the window to phone width: the step list sits above the table.
Regression: an auto run still starts, shows "Evaluation started" and lands on the Auto tab.
The new evaluation sheet in /m now uses the /w layout: the name on top,
a step list on the left with a check and chips for each pick, and the
open step's table on the right. Each step shows the /w columns.
- Application: Name, Kind, Type, Created At, a type filter and a search.
- Revision: one row per variant with version, deployments, model, date,
author and commit notes; older revisions open under it.
- Test set: an inline table with Name, Commit Message, Date Created and
Created by (a name, not a user id). A pick takes the latest revision
and moves to Evaluators.
- Evaluators: the /w columns, the Feedback column for human evaluators,
older versions, and Create new evaluator (opens Evaluators in a new tab).
- Close, Escape, overlay click and Cancel ask before they drop a draft
with input; an untouched draft closes at once.
The playground opens the same sheet, so it gets the same steps.
Entire-Checkpoint: 01M6R2JESW6N3AG1ZFAC6A5Z3T
Added a guided, step-based evaluation setup with searchable tables for applications, revisions, test sets, and evaluators.
Added filters for application type, expandable older revisions, and selection options tailored to evaluation type.
Added a confirmation prompt before closing a draft with unsaved input.
Added clearer loading, empty, and retry states, plus responsive table layouts.
Added links to create evaluators and new examples for test-set and human-evaluator setup.
The evaluation creation sheet changes to a step-based form with searchable application, revision, test-set, and evaluator tables. The sheet tracks draft input and confirms before closing a populated draft. Tests and Storybook stories are updated for the new flow.
The form uses step navigation and active section content. The sheet tracks draft input, closes untouched drafts directly, and requests confirmation before closing populated drafts.
The sections render searchable responsive tables with selection controls. Revision rows can expand to show older versions. Test-set selection uses its latest revision.
The evaluator section loads project-scoped entries and revisions, then displays searchable rows with expandable older versions. The prior evaluator picker components and revision label renderer are removed.
Wizard integration and examples web/mobile/tests/support/newEvaluationMocks.tsx, web/mobile/tests/unit/developerViews.render.test.tsx, web/mobile/tests/unit/playgroundNewEvaluation.render.test.tsx, web/storybook/stories/developer-views/NewEvaluationSheet.stories.tsx
Mocks, playground tests, and Storybook fixtures use the wizard tables. The developer-view test checks the evaluations routes and view state.
sequenceDiagram
actor User
participant NewEvaluationForm
participant NewEvaluationSheet
participant ConfirmationModal
NewEvaluationForm->>NewEvaluationSheet: report draft input state
User->>NewEvaluationSheet: request close
NewEvaluationSheet->>ConfirmationModal: show confirmation for populated draft
ConfirmationModal->>NewEvaluationSheet: confirm discard
NewEvaluationSheet->>User: close sheet
Loading
Possibly related PRs
Agenta-AI/agenta#8110: Opens NewEvaluationSheet from the playground with a fixed app and saved revisions, which the updated wizard retains as initial selections.
Agenta-AI/agenta#8007: Adds evaluator list and revision hooks and row types used by the evaluator table.
Merge Risk: Low · previous review dabc1
Test-set rows briefly become unselectable while searching or loading more. A small convention violation also needs cleanup. Both are low-risk and easy to fix.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title identifies the step-and-table layout for the new evaluation sheet.
Description check
The description covers the new layout, selection tables, discard confirmation, and tests.
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 28 files.
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 1 remain after this review.
Dependent tests finished. Post /preview for manual review. Environment absence verified; retained storage requires separate provider accounting.
Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Context
In /m developer mode, "New evaluation" opened a narrow drawer with an accordion. Each step was a short popover list: apps without their type or date, revisions without model or deployment status, test sets without dates or commit messages, and evaluators without the human feedback fields. There was no "Create new evaluator", the test set list showed the author as a raw user id, and one Escape dropped every pick without asking. /w shows all of this in a wide modal with a step list and a table per step.
Changes
The sheet keeps the /m drawer, but it is now as wide as the /w modal (1200 px, at most 90% of the window). The name sits on top, the step list on the left (Application, Revision, Test set, Evaluators, Advanced Settings, each with a check and chips of the picks), and the open step's table on the right. Below 768 px the step list sits above the table; at 390 px the page does not scroll sideways.
Each step now shows what /w shows:
Kept as they were: the auto name, the Advanced Settings defaults (10, 3, 3), the validation toasts and the start flow.
The playground opens the same
NewEvaluationSheet, so it gets the same layout with Application and Revision filled in and the test set table on screen. No playground file changes.The sheet still loads with
next/dynamicfrom the runs list and from the playground. It uses no antd and adds no dependency. It reuses the /m data hooks of the Variants, Test sets and Evaluators pages (useAppRevisions,useTestsetsList,useEvaluatorEntries,useEvaluatorRevisions) and the sharedListTable. Those files are not changed.Tests / notes
newEvaluationSheet,newEvaluationSectionsandnewEvaluationEvaluators(.render.test.tsx), plus pure tests innewEvaluationDraft.test.tsfordraftHasInput, the step table tree and the app type filter. Against the old code, 36 of these tests fail.playgroundNewEvaluation.render.test.tsxruns the real stores with HTTP mocked. It now checks that the sheet opens on Test set and that the revision table shows both playground revisions as checked.hiddenview. The new testdeveloperViews.render.test.tsx> "Evaluations stay off without beta developer mode or a forced view, even for a preview opt-in" proves the view is off, and the existing playground test "Stays hidden while the evaluations view is off" covers the other entry point. Checked live with beta off and no forced views: no Evaluations item in the nav,/evaluationsshows the 404 page, and the playground has no "New Evaluation".speed:checksize budget passes. The app shell is 884.6 KB. Every route, this one included, is about 2 KB over the budget baseline, which is drift on main and not from this change. The new tables are in the sheet's dynamic chunk.pnpm lint-fixand the full@agenta/mobilesuite were run. One unrelated test,appActionsMenu.render.test.tsx, failed once under load and passes when run again.What to QA