You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(frontend): create a new evaluator from the /m app playground evaluator picker - #8203
The evaluator picker in the /m app playground had no "Create new evaluator", which /w has. The picker could already show the action (#8192), but the editable evaluator drawer (#8187) owns the playground state while it is open: it would replace the app on the page with the new evaluator and clear the playground when it closed.
Changes
A drawer mode that leaves the playground alone.EditableRevisionDrawer takes playground={false}. In that mode it does not add its revision to the playground state, does not clear it on close, shows no "Test" button, counts only its own revision as unsaved when asked to close, and discards its local draft on close (nothing else would). The default is unchanged.
Create new in the app playground (beta developer mode only). The playground header now uses PlaygroundEvaluatorPicker, which passes onCreateNew to the shared picker. It opens the template picker, then the drawer in that mode, then the naming step. After Create the drawer closes and the new evaluator runs after the app, exactly like a picked one. The app, its revision and the test cases stay as they were. Without beta developer mode the picker is unchanged.
Stacked on #8197 (evaluator create entry points), based on main. The playground PR #8188 changes two lines in the shared picker file (chevron, chip colour); expect a small conflict there.
Tests
editableRevisionDrawer.render.test.tsx: off the playground the drawer leaves the page's playground nodes as they were, shows no Test, and drops its local draft on Discard (real molecule and playground controller, HTTP mocked).
playgroundEvaluatorCreate.render.test.tsx: no create without beta mode; with it the flow opens off the playground and the new evaluator is connected after the app.
The playground screen, compare, refine, revision actions, evaluator and execution header tests pass, including the phone-width check.
Types for @agenta/mobile and @agenta/oss, lint on the changed files.
Checked in the browser on a dev build against a test stack: Capital finder playground (verbose v5) > Evaluator > Create new evaluator > Exact Match. The drawer has no Test pane; Create with a name; the drawer closes, the new evaluator's chip shows next to Evaluator and its result card shows under the test case; the playground still shows verbose v5.
What to QA
Beta developer mode on. Open an app playground, load a test set. Evaluator > "Create new evaluator", pick a template, change a setting, Create, name it. The playground keeps its app revision and test cases, and the new evaluator shows as a chip and runs on Run.
Open the create drawer and close it without creating: the playground is unchanged.
Regression: without beta developer mode the picker has no Create new; at phone width the Evaluator button shows its icon only.
No actionable comments were generated in the recent review.
Summary
Summary by CodeRabbit
New Features
Added evaluator creation from the playground in beta developer mode. Newly created evaluators connect to the playground when an app node is available.
Evaluator drawers can open outside the playground without changing its displayed state. Testing controls are hidden, and local drafts are discarded when the drawer closes.
Connected evaluator revisions are grouped and shown as selected. Deselecting a revision disconnects it from the playground.
Style
On phones, the evaluator picker shows its icon without the text label.
The playground header now supports evaluator creation in beta developer mode and connects created evaluators to the root node when present. The drawer can operate without changing playground state. The evaluator picker supports connected-revision selection and disconnection, plus responsive placeholder text and scoped colors.
Changes: Playground evaluator creation and selection
Layer / File(s)
Summary
Drawer mode without playground state web/packages/agenta-playground-ui/src/panels/drawer/types.ts, web/packages/agenta-playground-ui/src/panels/drawer/EditableRevisionDrawerBody.tsx, web/packages/agenta-playground-ui/src/panels/drawer/useDrawerPlaygroundSession.ts, web/packages/agenta-playground-ui/src/panels/drawer/EditableRevisionDrawer.tsx, web/mobile/tests/unit/editableRevisionDrawer.render.test.tsx
The drawer accepts a playground flag. When false, it hides the test toggle, skips playground session setup, checks unsaved edits for the shown revision, and discards a displayed local draft on close. The test checks that the displayed revision remains unchanged.
The picker groups connected revisions by evaluator and supports selecting or disconnecting revisions. It shows selected-child summaries and revision counts. The popup accepts custom styling, the placeholder accepts ReactNode, and playground colors are scoped. Tests cover displayed revision metadata and disconnection.
Creation flow and playground integration web/mobile/src/features/evaluators/create/CreateEvaluatorFlow.tsx, web/mobile/src/features/evaluators/create/EvaluatorEditorDrawer.tsx, web/mobile/src/features/evaluators/create/PlaygroundEvaluatorPicker.tsx, web/mobile/src/features/playground/PlaygroundHeader.tsx, web/mobile/tests/unit/playgroundEvaluatorCreate.render.test.tsx
The header uses a picker that enables creation in beta developer mode. The creation flow opens with playground mode enabled and connects a created evaluator after the root node when one exists. Tests cover the creation gate and connection.
sequenceDiagram
participant PlaygroundHeader
participant PlaygroundEvaluatorPicker
participant CreateEvaluatorFlow
participant EvaluatorEditorDrawer
participant PlaygroundState
PlaygroundHeader->>PlaygroundEvaluatorPicker: render picker
PlaygroundEvaluatorPicker->>CreateEvaluatorFlow: open with automatic kind and playground mode
CreateEvaluatorFlow->>EvaluatorEditorDrawer: render creation drawer
EvaluatorEditorDrawer-->>PlaygroundEvaluatorPicker: return created evaluator
PlaygroundEvaluatorPicker->>PlaygroundState: connect evaluator after root node when present
Loading
Possibly related PRs
Agenta-AI/agenta#8087: Both changes preserve playground state when an evaluator creation drawer closes.
Agenta-AI/agenta#8075: Both changes connect evaluators downstream of the app in playground state.
Agenta-AI/agenta#8070: Both changes support downstream evaluator connections in the playground.
Merge Risk: Low · previous review 5d5c5
Evaluator creation appears mergeable with awareness that a discarded local draft may remain in memory after creation; no visible saved-result error was established.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title describes the main change: creating an evaluator from the /m app playground picker.
Description check
The description matches the evaluator creation flow, drawer mode, responsive picker, and tests.
Docstring Coverage
Docstring coverage is 100.00% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage
Explanation
Docstring coverage is 100.00% which is sufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 14 files. (1 skipped: 1 unsupported.)
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.
Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 1 remain after this review.
Ticks on each row to connect several evaluators, the selected summary
line, coloured type tags (colour tokens scoped to the picker popover
with .ag-playground-colors) and a chevron on the Evaluator button.
Entire-Checkpoint: 01M6RZE1A6BGP2YHBF6ZXJVC3G
Dependent tests finished. Post /preview for manual review. Environment absence verified; retained storage requires separate provider accounting.
Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.
@mmabrouk I’ll start an incremental review of the PR.
Action performed
Review finished.
Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Context
The evaluator picker in the /m app playground had no "Create new evaluator", which /w has. The picker could already show the action (#8192), but the editable evaluator drawer (#8187) owns the playground state while it is open: it would replace the app on the page with the new evaluator and clear the playground when it closed.
Changes
EditableRevisionDrawertakesplayground={false}. In that mode it does not add its revision to the playground state, does not clear it on close, shows no "Test" button, counts only its own revision as unsaved when asked to close, and discards its local draft on close (nothing else would). The default is unchanged.PlaygroundEvaluatorPicker, which passesonCreateNewto the shared picker. It opens the template picker, then the drawer in that mode, then the naming step. After Create the drawer closes and the new evaluator runs after the app, exactly like a picked one. The app, its revision and the test cases stay as they were. Without beta developer mode the picker is unchanged.smagain, so the playground bar keeps room for Compare. feat(frontend): automatic evaluators with types and categories in the /m playground evaluator picker #8192 had lost that when it moved the picker onto the shared cascader (the existingplaygroundComparetest caught it). The cascader'splaceholdernow takes an element for that.Stacked on #8197 (evaluator create entry points), based on main. The playground PR #8188 changes two lines in the shared picker file (chevron, chip colour); expect a small conflict there.
Tests
editableRevisionDrawer.render.test.tsx: off the playground the drawer leaves the page's playground nodes as they were, shows no Test, and drops its local draft on Discard (real molecule and playground controller, HTTP mocked).playgroundEvaluatorCreate.render.test.tsx: no create without beta mode; with it the flow opens off the playground and the new evaluator is connected after the app.@agenta/mobileand@agenta/oss, lint on the changed files.What to QA