Repository navigation
test(evals): harness hygiene — complete locate() miss diagnostics and route-commit wait in world open() - #4810
Conversation
The eval.task_activity.seed hook rendered an in-flight child under a parent whose thread status stayed "ready": it only marked the activity store busy. Since #4635 MessageList reads that as a stopped parent and the unobserved child shows "Waiting for task result" instead of the Working shimmer, so task-activity-shimmer and parent-child-permission-approval lost their [data-subagent-activity="shimmer"] row. A delegating parent is mid-run; mark the eval thread streaming like the chat-loading seed already does.
…immediately A freshly created session had no cached snapshot, so deriveSessionRenderModel reported "switching" while the first snapshot round trip ran and the composer disabled Run task. On Daytona that window is ~0.9 s and a click or Enter inside it was silently dropped with the draft intact. Seed the snapshot cache from the create response (stale, so the real fetch still runs as a background refresh).
…ad of wedging A prompt_async answered with 5xx/408 is an uncertain admission (#4635), but the hold it created could never end: the acceptance check treated absence as no evidence, Stop refused while the admission was unknown, and the failure body was dropped, so the person saw a paused composer with no explanation. - Keep the settled response body on PromptAdmissionUnknownError and show it as the session error card right away; the hold still prevents any resend. - readPromptAdmission: after the exact-message GET misses, list the idle conversation; when native lists it without the message the POST is settled and did not admit, so the drain receives admission_rejected and halts like any definite failure (explicit retry only, never an automatic resend). - On release the pending row is removed and the submitted composer is kept as an unsent message to restore; on confirmed acceptance the stale failure card is cleared.
Lexical keeps one empty paragraph in an empty editor, which innerText reports
as "\n". readComposerState now drops that trailing line break so draftText
matches what the person sees; user.see("composer", { text }) already trimmed.
The emptySession world declared /tmp/openwork-empty-session-<ts> but
seed.workspace() adopted the existing first-launch default workspace
instead ("OpenWork Chat" under the dev profile home). On Daytona that
profile lives inside the repo checkout, so the engine merged
/workspace/.opencode/opencode.json ("permission": "allow") over the
workspace's own catch-all and composer-run-mode never saw the workspace
MCP rule take effect.
…ce is server-owned Workspaces created from the app live only in the server registry, so a renderer reload reaches startServerWithoutDesktopWorkspace and restarted the embedded server (and its engine) although it was healthy and already serving that workspace. Reuse it when it is ready with the preferred remote-access mode; only restart when nothing usable is running.
…o e2e/saved-app-browser-requesting-conversation
…d 'origin/e2e/fresh-session-send-gate' into e2e/child-checklist-after-stop
… new tab returnVerification drove the agent browser entry point (browser.openUrl, the IPC behind browser.open_url) with no requesting conversation. Since #4434 the task host refuses that, and it never accepted an about:blank destination. The tab only stands in for the person's own browser so the openwork:// return link can be navigated without OS protocol registration, so create it the way a person opens a new tab and find its CDP target by diffing the page targets.
… edge An interrupted question tool leaves the engine's pending list through its finalizer without a question.rejected event, so Stop on a delegated child left its question (and option buttons) in the parent view. Reconcile questions alongside permissions on terminal edges, for both engines.
…terminal edge" This reverts commit 9e70982.
OpenCode v1 runs tools on detached fibers: aborting a session marks the question tool part as aborted but leaves its request in GET /question, so a parent that stopped a delegated child kept showing the child's question and its option buttons. After Stop confirms the tree idle, reject the pending questions owned by the stopped sessions; other roots are left untouched.
…-conversation' into e2e/dashboard-delete-no-buttons
…root has rendered
… into e2e/permission-ask-and-orphans-after-abort Stacks on #4747: creating the world's declared tmp workspace needs its desktop-runtime-boot fix so user.reload() reuses the running server instead of restarting the engine (which would drop every pending request). Its files are not modified here.
…lared tmp path
Test 2 of parent-child-permission-approval ran bash without asking although the
world wrote permission.bash="ask" to the workspace opencode.json. The world
called seed.workspace() without create, so it adopted the first-launch default
workspace under the dev profile, which on Daytona lives inside the repo
checkout; the engine merges /workspace/.opencode/opencode.json
("permission": "allow") after the workspace file and the agent ruleset ends
as [* allow, bash ask, * allow] (last match wins; permissions/effective reported
shell allow, source engine, workspace file .../home/OpenWork Chat/opencode.jsonc).
Same trap and same remedy as #4747 for emptySession; opt-in so the other
splitPaneQuestions worlds keep their arrangement.
…after Stop OpenCode v1 runs tools on detached fibers: aborting a session marks the tool part as aborted but leaves its Permission.ask pending in /permission, exactly as #4786 found for questions. The stopped tree would keep showing Allow once / Deny in every pane and after reload although nothing can consume the answer. After stopForegroundTree confirms every stopped session idle, also list pending permissions and reject the ones owned by the stopped tree; other roots are never touched and each withdrawal kind is best-effort on its own.
…fter-abort' into e2e/harness-hygiene-locate-workspaces-open
… into e2e/harness-hygiene-locate-workspaces-open
…ed role plus route and page roots A miss used to quote only the first 8 visible buttons in DOM order, which are always the shell rail, so every miss read as "the page has no buttons" (#4794's diagnosis detour). The browser-side report now lists every rendered element of the requested role (buttons/links for role-less targets), the current route/hash, and whether the [data-app-header] / [data-dashboard-page] roots are mounted. The Node side caps the list at 40 and says how many the page really had. Matching is unchanged.
… page DOM Proves the browser-side branch the way Chromium will execute it: the report lists every rendered menuitem on a dashboard whose shell rail has nine buttons, names the route and mounted page roots, and a role-less miss lists all nine buttons instead of the first eight.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
3 Skipped Deployments
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
Passed · 12/12 tests · 18/18 assertions · 8 images Commit |
Warden review summary — ClearNative findings: 0. Blocking policy matches before consolidation: 0. Full finding text remains in Warden's native checks and review threads. This comment retains only native IDs, severity, skill attribution, and safe locations. Analysis run Reported trigger coverage (not proof of complete repository or context coverage): diff-security-review (34111 ms), confidentiality-review (36137 ms). Blockers
Advisories
Needs recheck
Observed review-thread metrics
Thread states are read-only collection snapshots. These metrics are neither lifetime totals nor exact resolution times, and this summary is not clearance authority. |
…ess-hygiene-locate-workspaces-open
![diff-warden[bot]](https://avatars.githubusercontent.com/u/118544361?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (6d94944450f45856537eec87c8b7f10bc1b1e463). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
Resync onto
|
| test | result |
|---|---|
create, preview, save and reopen an app without changing already-open results (this PR's proof spec; all 14 steps incl. step 13 delete-from-dashboard and step 14 Dashboard Add) |
✓ passed (424.9 s) |
APP-ISOLATION embedded MCP Apps isolate siblings… (#4829) |
× world provisioning: Mock issuer gate failed: health reported https://3979-…, expected https://3979-… |
APP-ARCHIVE archived conversations render Apps without actions (#4863, new today) |
× same Mock issuer gate failed signature |
"passed":1,"failed":2,"skipped":0. Both red cases are appWeb worlds seeding two seed.mock(...) without explicit ports into one sandbox, so both default to daytonaPort: 3979 (evals/packages/env/src/mock.ts:56) and the second mock's pkill + rebind trips the issuer gate — now 5/5 across #4779, #4794 and this branch; #4829 and #4863 proved those cases with --local only. Attribute to #4829 / #4863 (give the second mock a distinct port). This PR does not touch that code; its harness change is inert on the green path (the miss branch is unit-tested).
Squash diff shrinks to af2b46ae3 + 7a31c4917 once #4798 and #4794 (and their ancestors) land.
…ess-hygiene-locate-workspaces-open
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (489ffd7192d7f953e1ce991e634ba31dfef55fcc). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
|
Restacked after #4786 landed: merged the updated #4798 ( |
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (4f0f2798207aed990fd590660077c4808faff432). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
Restack after #4770 landed (
|
| test | result |
|---|---|
create, preview, save and reopen an app without changing already-open results (this PR's proof spec, all 14 steps) |
✓ passed (342.1 s) |
APP-ISOLATION … (#4829) / APP-ARCHIVE … (#4863) |
× both in world provisioning: Mock issuer gate failed — the two-mocks-on-daytonaPort 3979 collision in those appWeb worlds (6/6 reproductions across #4779/#4794/#4810 now); not this PR's code |
"passed":1,"failed":1+1,"skipped":0; the harness change is inert on the green path and its miss branch is unit-tested. MERGEABLE.
This branch was successfully deployed
1 participant
Stacked on
origin/e2e/permission-ask-and-orphans-after-abort(#4798, which carries #4786, #4747, #4757, #4723) andorigin/e2e/dashboard-delete-no-buttons(#4794, which carries #4779) via merge commitsae3533bdcand20a3aa597; none of their files are modified here. Net change over the merges: two commits, both inevals/packages/cdp(af2b46ae3,7a31c4917). Depends on #4798 and #4794 landing first. Evals-only; no product code.Context: 2026-09-09 E2E triage wave-4 follow-ups. The brief originally carried a third item (make the four remaining
splitPaneQuestionsworlds create their own workspace); that item was reassigned mid-session to another session that will open its own PR, so this PR does not touchevals/worlds/chat.tsand nothing about it was committed or needed reverting.1.
locate()miss diagnostics (af2b46ae3,7a31c4917)Root cause.
cdp/src/input.ts#locatereported a miss with the first 8 rendered buttons/links in DOM order. The shell rail always comes first, so every miss read as "the page has no buttons", which sent #4779's diagnosis after a dashboard bug that did not exist (#4794 "Noticed, not touched" item 1).What changed. No change to matching. On a miss the page-side callback now returns:
location.hash(or pathname);[data-app-header]and[data-dashboard-page]are mounted.The Node side renders that as, e.g.
Could not locate {...}. Route #/dashboard. Page roots: appHeader=true dashboardPage=false. Visible menuitem candidates (2): menuitem "Remove Team briefing from dashboard", menuitem "Delete Team briefing".The list is capped at 40 (MISS_CANDIDATE_LIMIT) and says(showing first 40 of N)when truncated; an empty list saysNo visible <role> candidates.Older/partial reports (mocked surfaces) still produce a usable message.Unit proof.
node --test packages/cdp/test/input.test.ts→ 11/11. New tests: message rendering for role/route/roots; the 40-cap with truncation note and the empty-list wording; and the serialized page callback itself run in avmcontext against a fake DOM (9 shell-rail buttons + 2 dashboard menu items): amenuitemmiss lists both menu items andappHeader=false dashboardPage=true, a bare miss lists all 9 buttons (the old 8-cap is gone).pnpm --dir evals run check:browser→ 832 callbacks clean.2. Route-commit wait in other worlds'
open()/go()— audit, no change#4794 made the saved-apps world's
open()wait for the destination's own root aftergo()becausego()only setslocation.hash; the page being left stays mounted until the router commits, and the dashboard and app page share control labels. I audited every hash-only navigation inevals/worlds/**,packages/behaviors, andpackages/env, looking for the same shape: hash set → immediate click/read that the predecessor page can satisfy with the same label/text. None of the other worlds has that race, so per the brief ("only touch worlds where you can name the race") nothing was changed. Inventory:worlds/first-run.ts:1291-1292,:1346-1347(backgroundUpdateWorld,revokedUpdateWorldopenSettings/openWorkspace) — used bybackground-desktop-updateopenSettings()→click("Check now")/click(switch "Check automatically");openWorkspace()→see/notSee("Restart to update")clickblocks until the destination renders. "Restart to update" is the shell titlebar button, identical on both pages, and at everynotSeeafteropenWorkspace()the download has not finished, so the text is absent app-wide; the outcome cannot differ by which page is mounted. StructurallynotSee-right-after-gois the weak spot; listed, not changed.worlds/first-run.ts:296(artifactCodeBrowserWorld)click("Select tab: overflow-tab-12.md")worlds/first-run.ts:536,:675(alphaUpdateEligibilityWorld,updaterChannelWorld)updaterChannelWorld.relaunch(:692-698) already waits for[aria-label="Release channel"]after itsgo().worlds/first-run.ts:1155(openMcpSettings)worlds/library.ts:209, 236, 297, 426, 443, 462, 489, 531, 546, 572, 582, 595(setup-timelocation.hash = …/settings/generalor/extensions)connect-readiness-preseeded,library-add-connector-discovery) nextclick("Library")(shell rail; the click's own navigation is ordered after the queuedhashchange) orgo(/extensions)+waitFor(Add button)worlds/session-shell.ts:1529behaviors/desktop-boot.ts:92(waitForTaskUi)waitFor(route matches && Run task/hero)behaviors/desktop.ts:320,:386(waitForConnectionCard,openConnectionsSurface)/extensions+ destination text.env/desktop-app.ts:109location.reload()then interactive-probe pollSpec-level
go()call sites (not worlds; listed for completeness, not touched):workspace-engine-upgrade,opencode-v2-chat-routing,engine-v2-preview-flag,responsive-session-layout,cross-workspace-split-view,active-session-workspace-storm,workspace-storm-auth-coherence,library-add-connector-discovery,cloud-provider-local-credential-fallback,org-team-lifecycle-critical-pathall followgo()with a destination-onlywaitFor/see/model-picker read. The one I would flag:cloud-provider-sync-contract.e2e.test.ts:602goes fromsession/<id>to/sessionand immediatelyselectModel(...)— the model picker is on both pages, so the pick can land on the previous session's picker; it sets the global default so the outcome is the same today, but it is the shape of the race. Out of scope for this PR.Proof
pnpm evals:e2e saved-app-creation --daytonawithOPENWORK_EVAL_REF=7a31c49175f9666732ee138c3bcc6dee4e10b532exported (==> source verified 7a31c49175f9…in both sandboxes), fresh Den + admin desktop + web sandboxes, all three deleted by the run:All 14 steps ok, including step 13 "an admin cancels deletion in the app and confirms it on the dashboard" (the #4794 fix) and step 14 "Dashboard Add opens a creation conversation". This is the spec whose failure mode motivated item 1 and whose world carries the reference
open()wait for item 2; it proves the harness change is inert on a green path (the miss branch only runs on a miss, which the unit tests cover). No spec or world file was touched, so there is no other "touched spec" to run. Verdict: Passed.Noticed, not touched
cloud-provider-sync-contract.e2e.test.ts:602go(/session)→selectModelright after leavingsession/<id>(see table).background-desktop-updateassertssee("Restart to update")right afteropenSettings(); the text lives in the shell titlebar, so the line does not actually prove Settings shows it. Claim wording, not a race.worlds/library.tsworlds andalphaUpdateEligibilityWorld/updaterChannelWorld/ vault-recoveryopenMcpSettingsinfirst-run.tsare not imported by any spec.pnpm --dir evals typecheckreports ~399 pre-existing errors in files this PR does not touch (same as fix(app): apply per-session bash 'ask' permission and withdraw orphaned permission requests after Stop #4798 / test(evals): saved-app world.open waits for the destination page before the spec acts on it #4794); none inpackages/cdp.