Repository navigation
Complete browser tasks with site tools, page controls and user takeover - #4468
Conversation
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
abba20e to
b7e8fce
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
b7e8fce to
d7e1518
Compare
341854f to
334b418
Compare
Test evidence — a-transcript-link-replaces-the-selected-artifact-with-its-own-live-sidebar-tab-n — ✅ passedSHA 590b055 · engine v1 ✅ passed · 0 expectations passed · 0 failed · 0 pending
[world] desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788745859559) · session("Browser project") · [seed:raw] evalIn() · desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788745978829) · session("New task") · [seed:raw] evalIn() · session("Unrelated browser research") · [seed:raw] evalIn() ×4 · desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788746111451) · session("New task") · [seed:raw] evalIn() Test run created 2026-09-07T01:49:15.211Z · Source: Historical receipt on f57e309 (superseded)Test evidence — a-transcript-link-replaces-the-selected-artifact-with-its-own-live-sidebar-tab-n — ✅ passedSHA f57e309 · engine v1 ✅ passed · 0 expectations passed · 0 failed · 0 pending
[world] desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788744033550) · session("Browser project") · [seed:raw] evalIn() · desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788744164158) · session("New task") · [seed:raw] evalIn() · session("Unrelated browser research") · [seed:raw] evalIn() ×4 · desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788744297131) · session("New task") · [seed:raw] evalIn() Test run created 2026-09-07T01:18:48.659Z · Source: |
![diff-warden[bot]](https://avatars.githubusercontent.com/u/118544361?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (590b05592f5c1da3429eea9f90a956b2b7c77cb6). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
Test evidence — a conversation signs in, uses site tools and page controls with consent, isolation and recovery — ✅ passedSHA fba1b02 · engine v1 ✅ passed · 7 expectations passed · 0 failed · 0 pending
[world] desktop(daytona) · workspace(/tmp/openwork-browser-tools-1788912621563) · session("Browser project") · [seed:raw] evalIn() ×2 ℹ️ ASSERTION — 1. Explicit in-tab sign-in survives human takeover and resumeThe initial GET and filled-but-unsubmitted form recorded zero sign-ins. Only trusted form submission established the fixture session. Paused operations were refused and the same owned tab resumed with Session active.
ℹ️ ASSERTION — 2. No action before approval, no disclosure before sharing, no success before observationThe fixture saw no save at execution review. It saw exactly one authenticated save at result review, while neither the model nor the transcript had its result. Sharing released the receipt; browser_observe then verified Saved 1 before the final answer.
ℹ️ ASSERTION — 3. Injected webpage input cannot authorize navigation after takeoverCDP mouse and keyboard input followed by navigation to the redirect left the destination request count at zero. Explicit app address-bar navigation restored the signed-in page while agent operations remained paused until Resume browser.
ℹ️ ASSERTION — 4. Popup ownership, inherited sign-in, and isolation are independently witnessedA PNG-derived coordinate opened the popup only after approval. Its request carried the existing session without another sign-in. Hostile popup features exposed no Node globals and could not read the controlled cross-origin response.
ℹ️ ASSERTION — 5. Frame permissions and visual fallback preserve isolationNested fallback iframe markup did not grant the undelegated sibling tools. Both cross-origin frames reported no Node globals. The approved PNG-derived click reached the child control once, changed a fresh image, and could not reuse the consumed observation.
ℹ️ ASSERTION — 6. Origin-keying decisions do not trust the website's JavaScript worldBoth main-world getters returned eligible values and the page directly supplied forged policy booleans. Native opt-out and retained site-keying still refused registration and execution. A forged modelContext did not enter host discovery; the delegated sibling stayed available and no unsafe callback ran.
ℹ️ ASSERTION — 7. Current Den execution policy blocks origin, redirect, upload and deny-all attemptsThe desktop reported exact browserOrigins and blockBrowserUploads from real administrator PATCHes. Blocked requests produced no fixture writes. The identical upload succeeded only after its restriction was removed. An empty origin list denied all; clearing it restored the same signed-in tab. Closed and disabled handles remained refused.
Test run created 2026-09-09T00:10:21.540Z · Source: |
There was a problem hiding this comment.
Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (fba1b02aa74f159521f8fc31b72a694844a645e6). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run
This branch was successfully deployed
2 participants
A browser task keeps its conversation-owned, signed-in tab across website tools, page controls and visual input. Navigation consent, reading access, action approval and result sharing remain separate. Take over pauses automation until the user resumes; dispatch is not a claim that the task succeeded.
Current candidate
Head:
fba1b02aa74f159521f8fc31b72a694844a645e6.Base:
dev@314adea7957c79abae165e9ccb3cf6bde72be08d.Reconciled onto current dev, preserving explicit suspension/reload, task protection, first-document background emulation, retained-tab limits, cleanup, conversation deletion, login sync, artifact navigation, and managed execution policy. All 19 feature commits have verified signatures.
Review and runtime corrections
Takeover input correction
Addresses HDK-ZNW: webpage key and mouse events no longer enable manual navigation after takeover. Queued CDP/Electron input is not evidence of a human navigation decision, even when the browser marks it trusted.
Only explicit navigation through the app's address bar, Back, Forward, or Reload can enable manual browsing while paused. Those IPC handlers validate the main-window sender and main frame before changing navigation authority. Page/child-frame messages cannot grant it. Resume restores fresh task navigation consent; managed policy remains enforced throughout.
For a sign-in that needs navigation or redirects after takeover, use the address bar or Reload first. Same-page interaction remains available; clicking in the webpage alone does not lift the navigation guard.
Extended the existing native takeover regression for queued key/mouse input, forged toolbar messages, zero destination requests, valid toolbar navigation, and fresh consent on resume. Extended the existing browser journey with injected input, a blocked redirect witness, and explicit address-bar recovery. No new test files or exemptions.
Navigation security boundary
This is main-frame navigation consent, not a complete egress or DNS-rebinding sandbox. Frames, images, scripts, fetches and other subresources remain governed by the existing managed request policy. The browser retains its persistent signed-in profile.
Safety boundaries retained
The existing server-owned execution policy remains authoritative, including exact origins, intersected restrictions and upload blocking. No new Den API fields, migration or older host-only policy layer; #4481 remains unnecessary. No accounting branch dependency.
Task tools cannot read external profiles or enroll login-sync sites. Users sign in during takeover or use separately approved login sync. Cancellation precedes website discovery; callbacks require approval and bounded results stay local until Share result. Observations expire and are consumed before input. Popups preserve ownership, partition and sandboxing; cross-origin tools require frame delegation verified outside page-controlled globals.
Verification — Passed (selected browser journey)
Passed on this candidate:
node --test apps/desktop/electron/browser-panel.test.mjs: 45 passed, zero failed/skipped, exit 0.pnpm --filter @openwork/desktop typecheck:electron: exit 0.pnpm evals:check-browser: 751 callbacks checked; no captured variables, raw code strings or browser type errors, exit 0.git diff --check: exit 0.pnpm evals:e2e webmcp-browser-agent: 1 passed, 0 failed, 0 skipped, exit 0. Printedplacement: daytona (daytona CLI authenticated). Both Electron and Den were cold-booted fromfba1b02aa74f159521f8fc31b72a694844a645e6and disposed after the run.2026-09-09T00-10-21-539Z-a-conversation-signs-in-uses-site-tools-and-page-controls-with-consent-isolationthrough the existing PR evidence publisher. See the current test-evidence comment for the assertions and trace.Repaired failures: missing standalone eval dependencies, the discovery-resumption assumption, the reproduced background native crash, and lifecycle integration mismatches during the latest rebase. The final selected run has no failures or skips. Earlier red receipts remain historical, not current verdicts.
Review gates: both reported findings have code corrections and resolved response threads. Required build/core checks, SDK generation and CodeQL passed on this head at the latest check. Fresh Warden review is still in progress; no approval or merge is claimed. The separate viewport, transcript-link and full suspension journey suites were not rerun in this focused pass; their old receipts below remain historical.
Reproduce:
Historical runtime evidence
On
590b05592f5c1da3429eea9f90a956b2b7c77cb6, three journeys passed five tests with zero failures/skips in Daytona. These are not current-head proof:No merge, release or deployment. External-browser automation, declarative WebMCP, closed-shadow DOM references, file transfer and restoration of live handles after restart remain unsupported. Controlled fixtures do not establish open-ended task-planning quality on arbitrary sites or providers.