[mmabrouk]

Context

On an evaluation run, the /w Configuration tab shows the prompt that was evaluated, links to the test set and variant, the evaluator type and output metrics, and lets you edit the run and add an evaluator to it. The /m tab showed plain text, no prompt, no evaluator type or metrics, and could not edit anything. Users could not see the prompt that was evaluated, add an evaluator to a finished run, or rename the run from this tab.

Changes

Configuration tab

• Summary card: "Edit" (opens the run's edit dialog) and the short id 01a1acfc…d003 with copy, as in /w.
• Test set: a chip with its version that links to the test set (when that page is on).
• Application: the app and variant as chips; the variant has its version and links to the variants page with the revision open. The "Open variant" button is gone; the chip replaces it.
• Prompt Template: the revision the run used, read only, as Form, JSON or YAML. It is the revision drawer's own config view (RevisionConfigView, now exported from @agenta/entity-ui/variant), loaded with next/dynamic. A custom workflow without a schema still shows its JSON.
• Evaluators: each open card adds "Evaluator type" (the template tag, for example "Exact Match") and "Output metrics" (name, type, Required) from the revision's outputs schema. "Add evaluator" at the end opens the edit dialog.

Edit dialog (also opened from the run's "⋮" menu): name, description, and an Evaluators list with an "Add evaluator" picker, the one the new evaluation uses (human evaluators for a human run, automatic ones for the others). The run's evaluators show as picked. Picks are staged and can be removed; nothing is saved until Update.

Saving, as /w's saveEvaluationEditAtom does:

PATCH /simple/evaluations/{id}
{evaluation: {id, name, description, data: {
  testset_steps: {...unchanged},
  application_steps: {...unchanged},
  evaluator_steps: {"<existing>": "auto", "<added>": "auto"}
}}}

An added evaluator gets the origin of the run's other evaluators, so a human run stays human. Then, only when evaluators were added, every scenario id of the run is read page by page and sent to POST /simple/evaluations/{id}/process with overwrite: false: the new evaluator fills its cells, the old results stay. Every query keyed by the run and the runs list then reload. If the process call fails, the edit is still saved, the page reloads, and the dialog says the update failed.

New in @agenta/entities/evaluationRun: buildRunEditBody takes addedEvaluatorRevisionIds; processNewEvaluatorSteps.

Who can edit: unchanged from #8183. A live run, a closed run, and a reader without edit_evaluation_runs get no Edit and no Add evaluator.

Stacked on #8183 (base feat/m-parity-evaluation-run-header).

Not matched yet

• /w shows "Matchers" style categories as the evaluator type; /m shows the template tag the /m Evaluators list uses.
• /w has a Details/JSON switch per section; /m keeps its JSON disclosure for evaluators and uses the Form/JSON/YAML switch for the prompt.
• /w polls the run until the new evaluator finishes; /m reloads the run once and relies on the run page's own polling while the run is running.

What the tests cover

Risks, in order: the edit sends a wrong body (loses steps, flips origins, drops the new evaluator); new evaluators never run, or old results are overwritten; someone who may not edit sees the controls; the tab shows the wrong revision.

• runDetailsHeader.render.test.tsx (real run page, real save code, only HTTP mocked):
  • Update with a picked evaluator sends the exact PATCH body with the existing steps plus the new evaluator, then the exact process body with every scenario and overwrite: false.
  • A rename alone sends no process request.
  • A removed pick is not saved, and Update stays off with nothing else changed.
  • A failed process keeps the dialog open and shows the error.
  • A human run gets the human picker.
  • The Configuration tab's Edit and Add evaluator open the dialog; a reader who may not edit gets neither.
• evaluation-run-edit.test.ts (entities): the added evaluator's origin for auto and human runs, a duplicate changes nothing, the scenario ids are read across pages, no scenarios means no process request, and a failed process throws.
• runConfigSections.render.test.tsx: the test set and variant chips link with the run's revision ids; the prompt template shows the run's revision; a custom workflow shows JSON instead.
• runConfigEvaluators.render.test.tsx: the output metrics and type come from the revision the run used.
• runSummaryCard.render.test.tsx: Edit shows for an editor only (this replaces the old "no Edit" test).

Tests / notes

• vitest on the run page tests (170 tests) and the entities edit test: pass. Type checks for @agenta/mobile, @agenta/entities and @agenta/oss (/w, because two shared packages changed): pass. /w sees no change: it does not import RevisionConfigView, and the new buildRunEditBody argument is optional.
• In the browser on a test stack I checked the tab and opened the picker. I did not save an added evaluator on the shared stack, to keep its data as it is.

What to QA

• Open a finished auto run, Configuration. The test set and variant are chips with versions, and the prompt template shows the System and User messages.
• Add evaluator, pick an evaluator, Update. The run shows the new evaluator; its scores fill in on Scenarios while the old scores stay.
• Edit on the summary card renames the run.
• A human run: Add evaluator offers human evaluators.
• Regression: a live run has no Edit and no Add evaluator.

[coderabbitai]

Caution

Review failed

The pull request is closed.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[github-actions]

Website preview

Preview URL: https://pr-8193-agenta-website-preview.mahmoud-637.workers.dev

Built from 495c67a3242d558cda48cbe740f43b1d88c4d6d3. This comment updates in place on every push.

[github-actions]

Railway Preview Environment

Item	Value
Status	stopped
Commit	495c67a3242d558cda48cbe740f43b1d88c4d6d3
Manual expiry (UTC)	Not granted
CI deadline (UTC)	Not granted
Workflow	https://github.com/Agenta-AI/agenta/actions/runs/42256587551

Dependent tests finished. Post /preview for manual review. Environment absence verified; retained storage requires separate provider accounting.

Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.

[github-actions]

📘 Docs preview

Status	✅ Ready
Preview	https://pr-8193-agenta-docs-preview.mahmoud-637.workers.dev/docs/
Inspect	Actions run
Commit	a7c5457b16b7c3448256b217d35744816816237a

This comment updates in place on every push.

[mmabrouk]

🤖 The AI agent says: @coderabbitai review

[coderabbitai]

@mmabrouk I’ll review the changes.

Action not completed

Review skipped: 361 files exceed the limit of 300.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.