[mmabrouk]

Do not merge into v0.128.4. This goes into the next release.

Stacked on #8046 (playground input extras and focus drawer), which is stacked on #8041 (playground revision actions). The base of this PR is feat/m-view-playground-inputs-extra. It needs both: the compare columns reuse the revision bar from #8041 and the row parts from #8046. Merge #8041, then #8046, then this one.

Context

The /m playground shows one revision at a time. In /w you can put two or more revisions side by side and run the same test cases or chat turn on all of them. This PR adds that compare view to /m. It is playground PR 6.

Before the review loop finished, the work also changed shared state code in @agenta/playground, which the /w compare view uses. Those changes need a /w compare live check before this ships (see Package changes and Open problems).

It ships hidden

The playground is reachable only when the playground developer view is enabled. This PR adds no route, no navigation entry, no dependency and no antd. The compare view loads with next/dynamic only when two or more revisions are selected, so the single view never downloads it.

What the compare view does

• "Compare" in the playground header copies the last revision into a local draft and adds it as a new column. Its caret, "Compare options", opens the revision tree; picking a revision adds it. Revisions already on screen are disabled in the tree.
• With two or more revisions, the config area sits on top and the run area below, each half the height from md up. The config area has a "Variants" column with one card per revision, then one config column per revision, each with its own revision bar (switch, Commit, Deploy, menu). The run area has a sticky input column, then one output column per revision. Both areas scroll sideways together.
• Variant card: name and version (or the Draft tag and "from {name} v{n}" for a local draft), "Average latency" and "Average cost" from the runs' traces, "Move left" and "Move right", and close. Clicking the card scrolls its column into view.
• Completion rows: Run all runs every test case on every revision, and Run on a row runs that row on every revision. Cancel stops them all. The input column asks for every variable any compared revision uses.
• Chat rows: the shared user message sits in the input column, Run sends the last turn to every revision, and each reply shows in its own column.
• Close (card X or "Close panel" in the revision menu): a column with unsaved edits asks first ("Discard unsaved changes?"). A column without edits closes at once. With one revision left, the single view comes back.
• A phone shows the Configuration and Generations switch, the sticky columns stop being sticky below md, and each column is 85% of the screen wide.

Left out on purpose: drag and drop reorder (Move left and Move right replace it), a drag handle on the split, and the per-row "Create local copy for comparison" button in the picker (it shows on hover only, and "Compare" already copies the last column).

/w bugs fixed in /m

• Closing a revision in /w (card X and "Close panel") removes it from the selection only. Its draft stays in the store and comes back on a later open of the same revision. /m discards it. Tests: "Discard closes the column, drops its edits and shows the single view again" and "Closing an edited local draft names the draft, and Discard drops it".
• "Close panel" in /w closes a revision with unsaved edits without asking. /m asks, as the card does. Test: "Close panel in the revision menu asks before it drops unsaved edits".
• The /w chat compare Run bar never knows a run is busy, so Cancel never shows and Run can start a second run. /m reads the row's busy state. Test: "Cancel shows while the turn runs".
• In /m before this PR, a commit followed the new revision in the first column. Now a commit in any column replaces that column only. Test: "A commit in the second column shows the new revision in that column only".

Package changes and their effect on /w

@agenta/playground (used by /w). These came out of review rounds 1 and 2 and change how the /w compare view behaves:

• The test cases, messages and results live under the anchor revision (derivedLoadableIdAtom). setEntityIds treated the column at index 0 as the anchor, so after a reorder a commit moved the shared state to the wrong place. The anchor swap now follows the real anchor.
• Closing the anchor column used to leave the shared state behind, so the rows, results and chat were lost. The state now moves to the new first column.
• Before the anchor state moves (swap or close), the controller cancels the runs in flight. A run writes to the loadable it started under, so without the cancel the other column waited forever.
• The row clean (at run time and after an anchor commit) deleted inputs that another compared revision reads. It now keeps every compared revision's input keys. The request is still filtered per revision.
• referencedVariableKeysAtomFamily now returns the variables of every revision on screen, not only the anchor's, so the input column shows the union.

In a single-revision playground these paths do not run, but in /w compare they do. Tests in playgroundCompareRuns.render.test.tsx cover each case on the real controllers.

@agenta/playground-ui (/m only): the new panels/compare folder, CompareButton and RevisionTree in panels/config (the switcher now shares the tree), usePlaygroundScrollSync moved into the package (without the use-animation-frame dependency /w uses), an optional entityId on RunsPanelHeader (none means all revisions), hideUserMessage on ChatTurn, and an onRerun override on ChatMessage.

This PR also makes the delete tests from #8041 wait for the selection in place of a fixed 180 ms wait, which was flaky on a busy machine.

New user-facing text

Strings not copied from /w:

• "Move left" and "Move right" on the variant card.
• "Close" as the card close button's label.
• "Average latency" and "Average cost" are the /w strings in sentence case.

The other strings ("Compare options", "Discard unsaved changes?", the two close messages, "Close panel", "Generations", "Chat") are copied from /w.

Checks

• After the rebase onto release/v0.128.4 (at d5bc67afec, with feat(frontend): evaluators and archived evaluators in /m behind the developer view switch #8007 and fix(frontend): pass the leave question to the shared unsaved guard in the test set editor #8031):
  • Types: @agenta/mobile and @agenta/oss (/w) pass.
  • The test files this PR adds or changes: 65 tests. playgroundRevisionActions.render.test.tsx passed 27 of 27 when it ran alone. When the machine was busy, four of its delete tests timed out at 5 seconds. The head before the rebase fails the same way, so the rebase did not cause it.

Recorded runs on the final head (36ad244091), before the rebase:

• Types: @agenta/mobile, @agenta/oss (/w) and @agenta/playground pass.
• Mobile tests: 1195 passed, 1 failed. The failure is membersTabRoleChange.render.test.tsx, which also fails on the release branch. New test files: playgroundCompare, playgroundCompareRuns and playgroundScrollSync.
• @agenta/playground tests: 281 passed.
• Speed (measured at 56f8d8fb8d): budget passes, the playground route minus the app shell changed by 0.1 KB gzip or less, the compare chunk is 6.1 KB gzip and loads on demand, and the route has no antd.
• Storybook: the CompareLayout stories (completion and chat, two and three columns) captured at 1440 px and 390 px, light and dark, 16 captures with no errors or overflow.
• Not run: the compare view against a live API, and a live check of the /w compare view after the @agenta/playground changes.

Review results

Three rounds of Codex slop, simplify and correctness. Round 1: correctness High, slop Medium, simplify Medium; the findings were the anchor swap after a reorder, lost state when the anchor closes, inputs from the anchor only, and averages read from a wrong result shape. Fixed. Round 2: correctness High, simplify High, slop Medium; the findings were the row clean that deleted other revisions' inputs, and a column that waited forever when the anchor moved during a run. Fixed. Round 3 (head 36ad244091): simplify Low, correctness Medium, slop Medium, with one finding left (the first item below).

Open problems

• Closing or committing the anchor column while a chain with a downstream evaluator runs: the cancel stops root sessions only, so the moved evaluator sessions (sess:<root>:<evaluator>) stay "running". /m has no evaluator chain until PR 7, but /w can hit it. Fix before PR 7 ships evaluator chains in compare: cancel every pending or running result in the old loadable, downstream too, before the move, with a chain test.
• Run a /w compare live check before this ships: reorder, close the first column, commit a moved column, and run while you close a column.
• A browser reload with local drafts in compare drops them, because ?revisions= leaves drafts out. /w does the same. The playground snapshot in PR 8 fixes it.

How to QA once enabled

Enable the view with AGENTA_DEVELOPER_VIEWS=playground, then open a prompt app's playground.

• Press Compare. A local draft of the last revision opens as a second column. Use Compare options to add a third revision.
• Run all. Each column shows its own output, and the variant cards show average latency and cost.
• Press Move right on the first card, then commit a change in the moved column. The other column keeps its outputs and the test cases stay.
• Close the first column. The test cases and the other column's output stay.
• Edit a column, then close it from the card and from "Close panel". Both ask first.
• On a chat app, compare two revisions, send a turn and press Cancel while it runs.
• Regression in /w: compare two revisions, reorder them, close the first one and commit from a moved column. The rows, inputs and outputs stay where they belong.

[coderabbitai]

Caution

Review failed

The pull request is closed.

Summary

Summary by CodeRabbit

• New Features
  • Compare multiple workflow revisions side by side, with controls to add, reorder, switch, and close revisions.
  • Compare chat and completion runs, including per-revision outputs and average latency, token, and cost metrics.
  • On smaller screens, switch between configuration and generation views; on larger screens, view both with synchronized scrolling.
  • Closing a revision with unsaved changes prompts for confirmation.
  • Run actions can target a single revision or all revisions in the comparison.

The playground now supports side-by-side comparison of multiple revisions. It adds revision selection and close flows, responsive configuration and generation views, per-revision run outputs, and state handling that preserves inputs shared across revisions.

Changes: Playground revision comparison

Layer / File(s)	Summary
Workspace selection and revision lifecycle web/mobile/src/features/playground/PlaygroundScreen.tsx, PlaygroundWorkspace.tsx, PlaygroundCompare.tsx, RevisionConfig.tsx, RevisionActions.tsx, RevisionMenu.tsx, useCloseRevision.ts, useSwitchRevision.ts, web/packages/agenta-playground-ui/src/panels/config/*, web/mobile/tests/unit/playgroundCompare.render.test.tsx, web/mobile/tests/unit/playgroundRevisionActions.render.test.tsx	The workspace passes ordered revision IDs into a single- or multi-revision view. Compare selection can add a local draft or an existing revision. Per-revision configuration supports switching, committing, closing, and dirty-state discard confirmation. Integration tests cover selection, ordering, closing, and committing columns.
Shared inputs and revision state web/packages/agenta-playground/src/state/controllers/playgroundController.ts, state/execution/selectors.ts, state/execution/webWorkerIntegration.ts, state/helpers/entityInputContract.ts	Variable collection and testcase reconciliation include input keys from other root revisions. Anchor changes move shared loadable state and cancel in-flight tests when the loadable ID changes.
Comparison layout and revision columns web/packages/agenta-playground-ui/src/hooks/usePlaygroundScrollSync.ts, panels/compare/, web/storybook/stories/playground/CompareLayout.stories.tsx, web/mobile/tests/unit/playgroundScrollSync.test.tsx	The responsive layout displays configuration and generation areas together on wider screens and switches between them on narrower screens. Revision cards include ordering actions and run averages. Scroll synchronization and completion/chat Storybook examples are added.
Per-revision run and output views web/packages/agenta-playground-ui/src/panels/chat/, panels/compare/ChatCompareRow.tsx, CompletionCompareRow.tsx, CompareRuns.tsx, panels/runs/, web/mobile/tests/unit/playgroundCompareRuns.render.test.tsx	Chat and completion rows render outputs for each revision and expose run and cancel controls. The run header supports running all revisions, and trace footers and run metrics appear in comparison views. Tests cover outputs, inputs, cancellation, and column changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  actor User
  participant CompareButton
  participant PlaygroundWorkspace
  participant CompareLayout
  participant CompareRuns
  User->>CompareButton: Select Compare
  CompareButton->>PlaygroundWorkspace: Add a local draft revision
  PlaygroundWorkspace->>CompareLayout: Render the selected revision columns
  CompareLayout->>CompareRuns: Render comparison runs

Loading

Possibly related issues

• [bug] Flaky acceptance test: annotation queue run still active right after stop and close #7772 — This change adds cancellation when anchor state changes and tests closing a column during an in-flight run, which overlaps with the issue’s concern about run lifecycle during stop or close.

Possibly related PRs

• Agenta-AI/agenta#8012: Adds the playground route, selection flow, workspace, and split layout that this change extends with multiple revision columns.
• Agenta-AI/agenta#8041: Adds revision actions and switching/commit flows that this change composes into each comparison column.
• Agenta-AI/agenta#8064: Changes the same workspace and config composition that this change updates for multi-revision views.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[github-actions]

Website preview

Preview URL: https://pr-8049-agenta-website-preview.mahmoud-637.workers.dev

Built from e1bb11363d96e97548ad3a727e1eba225578ba1d. This comment updates in place on every push.

[github-actions]

Railway Preview Environment

Item	Value
Status	stopped
Commit	767480a9fe3b60979885b2b5f4862044f7d3c8f4
Manual expiry (UTC)	Not granted
CI deadline (UTC)	Not granted
Workflow	https://github.com/Agenta-AI/agenta/actions/runs/41684856134

Preview use finished or expired. Environment absence verified; retained storage requires separate provider accounting.

Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.

[mmabrouk]

🤖 The AI agent says: Live QA on a fresh OSS stack (head 4c66443), with Mistral runs.

/w playground (shared code changed by this PR): all 8 checks pass.

• A single revision: run, edit and commit work as before.
• Compare with 2 revisions: both columns run, and the input column shows the variables of both revisions. On release/v0.128.4 the second revision's variable is missing, so this PR fixes that.
• Committing in the second column leaves the first column and both results in place.
• Closing the first column and then committing keeps the result and the inputs. On release/v0.128.4 the result is lost as soon as the first column closes, so this PR fixes that too.
• Closing a column with unsaved edits asks first, with the /w text.
• Chat compare: both columns answer, and a second message and Run all work.
• Closing the first column while a run is going leaves nothing stuck in a running state.
• The console shows no new errors. /w compare logs Maximum update depth exceeded many times, but release/v0.128.4 logs it on the same steps, so this PR does not cause it.

/m compare view (hidden): at 1440 px, the variables, the commit in the second column and the close prompt all pass. At 390 px the same checks pass, but the result columns sit to the right of the input column, so the page scrolls sideways. That is a follow-up.