You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(frontend): keep the playground test cases after creating an evaluator from New Evaluation in /w - #8087
For the release after v0.128.5. The bug is already in production; v0.128.5 did not cause it.
In the /w playground, connect a test set. Open New Evaluation, then Evaluators, then "Create new evaluator" (for example Exact Match). Close the drawer and close the modal. The playground now has 0 test case rows.
The modal opened the evaluator drawer without isolatedPlayground: true. The drawer then took over the app's playground store, and its close cleanup deleted the app's rows. The playground header already passes isolatedPlayground: true and keeps the rows.
Change
The modal now opens the drawer with isolatedPlayground: true, through a small helper (newEvaluationEvaluatorDrawerParams).
OpenDrawerParams is exported from evaluatorDrawerStore.ts for the helper.
Isolated mode in the modal
onEvaluatorCreated still fires with the new revision id, so the modal still selects the new evaluator.
The drawer skips the playground's own commit callback, which the modal does not need.
The drawer gets its own store, so it works the same when the modal opens from the evaluations page.
initialAppSelection is not passed. The modal has no app revision to hand over, so the persisted app selection stays the fallback.
Tests
evaluatorDrawerParams.test.ts opens the drawer the way the modal does. It checks that the drawer is isolated and that onEvaluatorCreated still receives the revision id. It fails without the fix. It does not render the drawer, so the row-deletion path is covered by the isolation flag, not by a rendered close.
Checks: pnpm --filter @agenta/oss types:check, the test, eslint and prettier on the changed files all pass.
Creating an evaluator from a template now opens the evaluator drawer in an isolated playground and stays in the current view after creation.
The created evaluator’s configuration ID is passed to the creation callback when available.
The New Evaluation modal now uses a helper to build evaluator drawer parameters after template creation. The helper enables isolated-playground mode and keeps navigation in the modal. Tests cover drawer state and callback forwarding.
Changes: New Evaluation evaluator drawer
Layer / File(s)
Summary
Drawer parameter helper and tests web/oss/src/state/evaluator/evaluatorDrawerStore.ts, web/oss/src/components/pages/evaluations/NewEvaluation/Components/evaluatorDrawerParams.ts, web/oss/src/components/pages/evaluations/NewEvaluation/Components/evaluatorDrawerParams.test.ts
OpenDrawerParams is now exported. The new helper returns create-mode parameters with isolated-playground mode enabled and post-create navigation set to "stay". Tests check drawer state and callback forwarding.
Agenta-AI/agenta#8065: Changes the same New Evaluation drawer flow and sets postCreateNavigation: "stay" when opening the evaluator drawer.
Merge Risk: Low · up to 59994
The drawer may require selections to be restored manually, and a concurrent save may leave the playground on its previous revision. These risks are bounded but should be understood before merging.
Pre-merge checks: 5
Check name
Status
Explanation
Title check
The title names the actual fix: the playground test cases are kept after an evaluator is created from New Evaluation in the /w playground. The changeset adds isolatedPlayground: true to the modal's dr…
Description check
The description is directly related to the changeset. It explains the bug (the modal opens the evaluator drawer without isolatedPlayground: true, so the drawer's close cleanup deletes playground rows)…
Docstring Coverage
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the actual fix: the playground test cases are kept after an evaluator is created from New Evaluation in the /w playground. The changeset adds isolatedPlayground: true to the modal's drawer parameters through a new helper, which matches this. The title is specific and follows the fix(scope) convention. It is long (99 characters) but still reads as one clear sentence.
Full details: Description check
Explanation
The description is directly related to the changeset. It explains the bug (the modal opens the evaluator drawer without isolatedPlayground: true, so the drawer's close cleanup deletes playground rows). It describes the change (the newEvaluationEvaluatorDrawerParams helper and the exported OpenDrawerParams), the behavior of onEvaluatorCreated, the new tests, and the checks that were run.
Full details: Docstring Coverage
Explanation
No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 4 files.
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
The tests validate the helper, but they do not validate that NewEvaluationModalContent uses the helper when a template creates an evaluator. A regression in that caller could omit isolatedPlayground: true while both tests still pass. Add a modal-level test that invokes the template handler and asserts the parameters passed to openEvaluatorDrawer.
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.
Dependent tests finished. Post /preview for manual review. Environment absence verified; retained storage requires separate provider accounting.
Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.
🤖 The AI agent says: Live check of this PR on a fresh OSS dev stack at head 59994e2, in the old /w app at 1440 px.
The release QA steps (connect a test set with 2 rows, New Evaluation > Evaluators > Create new evaluator > Exact Match, close the drawer, close the modal): pass. The playground keeps its 2 test case rows, and the shown revision stays v1 (no evaluator draft).
Save the new evaluator in the drawer first: pass. The modal selects the new evaluator and the rows stay.
New Evaluation from the Evaluations page (outside the playground), Create new evaluator, save: pass.
The playground header's own evaluator drawer: pass, rows stay.
Console: no new errors.
One note, not caused by this PR: after any evaluator drawer closes in the playground (this path and the header path, which this PR does not touch), the URL loses ?revisions= and the #pgSnapshot hash. A page reload then shows 1 empty row and no test set. Opening and closing only the modal keeps both.
A video of the steps is in my local evidence folder and can be attached on request.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
For the release after v0.128.5. The bug is already in production; v0.128.5 did not cause it.
In the /w playground, connect a test set. Open New Evaluation, then Evaluators, then "Create new evaluator" (for example Exact Match). Close the drawer and close the modal. The playground now has 0 test case rows.
The modal opened the evaluator drawer without
isolatedPlayground: true. The drawer then took over the app's playground store, and its close cleanup deleted the app's rows. The playground header already passesisolatedPlayground: trueand keeps the rows.Change
isolatedPlayground: true, through a small helper (newEvaluationEvaluatorDrawerParams).OpenDrawerParamsis exported fromevaluatorDrawerStore.tsfor the helper.Isolated mode in the modal
onEvaluatorCreatedstill fires with the new revision id, so the modal still selects the new evaluator.initialAppSelectionis not passed. The modal has no app revision to hand over, so the persisted app selection stays the fallback.Tests
evaluatorDrawerParams.test.tsopens the drawer the way the modal does. It checks that the drawer is isolated and thatonEvaluatorCreatedstill receives the revision id. It fails without the fix. It does not render the drawer, so the row-deletion path is covered by the isolation flag, not by a rendered close.Checks:
pnpm --filter @agenta/oss types:check, the test, eslint and prettier on the changed files all pass.