Skip to content

feat(browser): give each conversation its own built-in browser tabs - #4434

Merged
benjaminshafii merged 3 commits into
devfrom
feature/thread-owned-browser-tabs
Sep 4, 2026
Merged

benjaminshafii merged 3 commits into
devfrom
feature/thread-owned-browser-tabs

Conversation

@reachjalil

Copy link
Copy Markdown
Member

Problem

You read one conversation while another conversation's agent needs a web page. The browser pops open in the conversation you are reading, showing a page that has nothing to do with it, and the working conversation shows nothing. The built-in browser kept one global tab list with no owner, openwork_execute dropped the agent's session id before the UI command bridge, and the side panel mirrored every native tab into whichever conversation was mounted. Any navigation of a hidden tab also auto-selected it onto the screen.

What changes

  • Tabs belong to the conversation that opened them. The bridge stamps UI commands with the requesting session (origin, set from the tool context, never by the agent), the control bus hands that origin to the action, and the Electron browser keeps an owner per tab. Each side panel shows only its own tabs.
  • Background conversations browse silently. Only the on-screen conversation's tabs may take the screen. A tab opened for a background conversation loads there, its panel state is marked open, and it is waiting when the user switches. The navigation auto-select is owner-aware; panel-opened / panel-closed name the owner.
  • Hidden tabs stay real pages for the agent. Chromium only paints a page it considers visible, so a hidden tab keeps a one-pixel presence in the window corner (under the rounded-corner mask) while a main-process DevTools session emulates a 1280×800 viewport and focus. Reads, clicks, typing, and CDP screenshots work before anyone looks; both are undone when the tab returns to the screen and the existing viewport-reset path takes over.
  • New @openwork/browser-tabs package holds the ownership, surfacing, and background-rendering policy, shared by the main process and the renderer and unit-tested on its own.

What stays the same

Tabs opened without a conversation (no session on screen) stay shared and behave exactly as before. The visible conversation's own tabs surface as before. browser_* tools and the CDP handle contract are unchanged; open_url additionally returns owner_session_id and visible.

Proof

New journey evals/specs/browser-tabs-owned-by-thread.e2e.test.ts: a background conversation opens a page — the on-screen conversation's tab, panel, and viewport are untouched; the hidden page lays out at 1280×800, reports focus, accepts a dispatched click and typed text, and screenshots at full size; switching to the owner shows its tab at the panel's viewport; switching back restores the first conversation's own tab only.

  • OPENWORK_EVAL_REF=fb940de88f0acef07a99cf2641f7215d9170a518 pnpm evals:e2e browser-tabs-owned-by-thread — placement: daytona (daytona CLI authenticated), checkout resolved fb940de88f0a, 1 passed / 0 failed / 0 skipped.
  • pnpm evals:e2e browser-panel-viewport-recovery --local — 1 passed (the existing built-in browser journey, same world).
  • packages/browser-tabs: 12 policy tests; apps/desktop/electron/browser-panel.test.mjs: 9 (5 new, stubbed WebContentsView); apps/server plugin tests: 26 (1 new: origin stamping cannot be spoofed by the agent).
  • @openwork/app, openwork-server, typecheck:electron, check:electron, evals lint:layers, boundary and channel ratchets: clean.

@vercel

vercel Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

4 Skipped Deployments
Project Deployment Actions Updated
openwork-app Ignored Ignored Preview Sep 4, 2026 8:49pm UTC
openwork-den Ignored Ignored Preview Sep 4, 2026 8:49pm UTC
openwork-diagnostics Ignored Ignored Preview Sep 4, 2026 8:49pm UTC
openwork-landing Ignored Ignored Preview Sep 4, 2026 8:49pm UTC

Comment thread evals/specs/browser-tabs-owned-by-thread.e2e.test.ts
diff-warden[bot]
diff-warden Bot previously approved these changes Sep 4, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (fb940de88f0acef07a99cf2641f7215d9170a518). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run

Comment thread evals/specs/browser-tabs-owned-by-thread.e2e.test.ts
@reachjalil
reachjalil enabled auto-merge (squash) September 4, 2026 20:14
A conversation working in the background opened its browser tab in whatever
conversation the user was reading: the native browser kept one global tab
list with no owner, the agent's session id was dropped before the UI command
bridge, and the side panel mirrored every native tab into whichever
conversation was mounted.

Tabs now belong to the conversation whose agent opened them. The bridge
stamps UI commands with the requesting session, the control bus hands that
origin to the action, and the Electron browser keeps an owner per tab. Only
the on-screen conversation's tabs may take the screen; a tab opened for a
background conversation loads silently there and is waiting when the user
switches to it. Hidden tabs stay real pages for the agent driving them: a
one-pixel on-window presence keeps Chromium painting while a main-process
DevTools session emulates a full viewport and focus, so reads, clicks,
typing, and screenshots all work before anyone looks. Both are undone when
the tab returns to the screen.

The ownership, surfacing, and background-rendering policy lives in the new
@openwork/browser-tabs package, shared by the main process and the renderer
and tested on its own.
The Den API image copies apps/desktop/package.json for version metadata and
the gateway image builds apps/app; both now depend on @openwork/browser-tabs,
so the pruned workspace installs need the package present.
diff-warden[bot]
diff-warden Bot previously approved these changes Sep 4, 2026

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (6b4a3c01d4a370b057046f1f973593221de81edb). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run

…tabs journey

The claim that a person can switch to the background conversation and find
its page waiting was driven through the app's control rail. Click the
conversation in the sidebar instead, as a user does, and give the first
conversation a stable title so the click has a target.
@reachjalil

Copy link
Copy Markdown
Member Author

Test evidence — a-background-conversation-s-agent-browses-silently-and-its-page-is-waiting-when- — ✅ passed

✅ passed · 0 expectations passed · 0 failed · 0 pending

screenshots not attached (gh < 2.99; run brew upgrade gh)

[world] desktop(daytona) · workspace(/tmp/openwork-builtin-browser-1788555078931) · session("New task") · [seed:raw] evalIn()
[user] click(text=Reading the news) · see(label=/^Select tab: .*viewport-probe=reading$/, timeoutMs=30000) · see(label=/^Select tab: .*viewport-probe=reading$/) · notSee(label=/^Select tab: .*viewport-probe=research$/) · click(text=Background research) · notSee(label=/^Select tab: .*viewport-probe=reading$/) · click(text=Reading the news) · see(label=/^Select tab: .*viewport-probe=reading$/, timeoutMs=30000)
steps 1 ✅ The background conversation opens a page without touching the screen (4.6s) · 2 ✅ The hidden page is real for the agent: viewport, focus, clicks, typing, screenshot (3.7s) · 3 ✅ Switching to the background conversation shows its page at the panel's size (4.4s) · 4 ✅ Returning to the first conversation brings back only its own tab (1.4s)
verdict passed · 5 user observations (see ×3, notSee ×2) · 0 probes · steps 4/4


Test run created 2026-09-04T20:49:42.394Z · Source: evals/results/test-runs/2026-09-04T20-49-42-393Z-a-background-conversation-s-agent-browses-silently-and-its-page-is-waiting-when/test-run.json · Repro: pnpm --dir evals artifacts:publish -- --pr 4434 --test-run 2026-09-04T20-49-42-393Z-a-background-conversation-s-agent-browses-silently-and-its-page-is-waiting-when

@benjaminshafii benjaminshafii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the conversation ownership registry, origin stamping, and latest journey changes. The outstanding switching comment is addressed: the test now clicks named conversations through user.click, and the fresh published Daytona run passes all four steps. Cleared that resolved finding; merge remains subject to required checks.

@benjaminshafii
benjaminshafii merged commit 1d2fc5b into dev Sep 4, 2026
31 checks passed
auto-merge was automatically disabled September 4, 2026 20:55

Pull Request is not mergeable

@diff-warden diff-warden Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warden security clearance: clear. No blocking findings (security or desktop↔den sync) in this diff (68254accdefff4668c86846c2d775e44eb5e86b7). Automated clearance satisfies the required-review gate only — a human still reviews and merges. Analysis run

@reachjalil

Copy link
Copy Markdown
Member Author

Heads-up from the stacked PR #4452: after this branch moved onto newer dev (68254accd), the upstream journey browser-panel-viewport-recovery fails locally on this head without any login-import change (expected 0 to be greater than 0 — the tab's innerWidth is 0 right after its tab button appears). Reproduced twice on a clean checkout of 68254accd. Worth a re-proof of this head before merge.

yomgui added a commit that referenced this pull request Sep 11, 2026
… person's tab, not agent browser control (#4779)

* fix(app): seed a created session's snapshot so its composer can send immediately

A freshly created session had no cached snapshot, so deriveSessionRenderModel
reported "switching" while the first snapshot round trip ran and the composer
disabled Run task. On Daytona that window is ~0.9 s and a click or Enter inside
it was silently dropped with the draft intact. Seed the snapshot cache from the
create response (stale, so the real fetch still runs as a background refresh).

* fix(app): let a send that settled with a server failure recover instead of wedging

A prompt_async answered with 5xx/408 is an uncertain admission (#4635), but
the hold it created could never end: the acceptance check treated absence as
no evidence, Stop refused while the admission was unknown, and the failure body
was dropped, so the person saw a paused composer with no explanation.

- Keep the settled response body on PromptAdmissionUnknownError and show it as
  the session error card right away; the hold still prevents any resend.
- readPromptAdmission: after the exact-message GET misses, list the idle
  conversation; when native lists it without the message the POST is settled
  and did not admit, so the drain receives admission_rejected and halts like
  any definite failure (explicit retry only, never an automatic resend).
- On release the pending row is removed and the submitted composer is kept as
  an unsent message to restore; on confirmed acceptance the stale failure card
  is cleared.

* test(evals): read an empty Lexical composer as an empty draft

Lexical keeps one empty paragraph in an empty editor, which innerText reports
as "\n". readComposerState now drops that trailing line break so draftText
matches what the person sees; user.see("composer", { text }) already trimmed.

* test(evals): open the fresh conversation through the sidebar's New session button

The sidebar button was renamed from "New task" to "New session" in #4491
(Sep 5) and the "new" scenario added in #4631 (Sep 8) still targeted the old
label, so it could never find the button. Same user action, current label.

* test(evals): open the saved-app verification return tab as a person's new tab

returnVerification drove the agent browser entry point (browser.openUrl, the
IPC behind browser.open_url) with no requesting conversation. Since #4434 the
task host refuses that, and it never accepted an about:blank destination.
The tab only stands in for the person's own browser so the openwork:// return
link can be navigated without OS protocol registration, so create it the way a
person opens a new tab and find its CDP target by diffing the page targets.
yomgui added a commit that referenced this pull request Sep 11, 2026
…re the spec acts on it (#4794)

* fix(app): seed a created session's snapshot so its composer can send immediately

A freshly created session had no cached snapshot, so deriveSessionRenderModel
reported "switching" while the first snapshot round trip ran and the composer
disabled Run task. On Daytona that window is ~0.9 s and a click or Enter inside
it was silently dropped with the draft intact. Seed the snapshot cache from the
create response (stale, so the real fetch still runs as a background refresh).

* fix(app): let a send that settled with a server failure recover instead of wedging

A prompt_async answered with 5xx/408 is an uncertain admission (#4635), but
the hold it created could never end: the acceptance check treated absence as
no evidence, Stop refused while the admission was unknown, and the failure body
was dropped, so the person saw a paused composer with no explanation.

- Keep the settled response body on PromptAdmissionUnknownError and show it as
  the session error card right away; the hold still prevents any resend.
- readPromptAdmission: after the exact-message GET misses, list the idle
  conversation; when native lists it without the message the POST is settled
  and did not admit, so the drain receives admission_rejected and halts like
  any definite failure (explicit retry only, never an automatic resend).
- On release the pending row is removed and the submitted composer is kept as
  an unsent message to restore; on confirmed acceptance the stale failure card
  is cleared.

* test(evals): read an empty Lexical composer as an empty draft

Lexical keeps one empty paragraph in an empty editor, which innerText reports
as "\n". readComposerState now drops that trailing line break so draftText
matches what the person sees; user.see("composer", { text }) already trimmed.

* test(evals): open the fresh conversation through the sidebar's New session button

The sidebar button was renamed from "New task" to "New session" in #4491
(Sep 5) and the "new" scenario added in #4631 (Sep 8) still targeted the old
label, so it could never find the button. Same user action, current label.

* test(evals): open the saved-app verification return tab as a person's new tab

returnVerification drove the agent browser entry point (browser.openUrl, the
IPC behind browser.open_url) with no requesting conversation. Since #4434 the
task host refuses that, and it never accepted an about:blank destination.
The tab only stands in for the person's own browser so the openwork:// return
link can be navigated without OS protocol registration, so create it the way a
person opens a new tab and find its CDP target by diffing the page targets.

* test(evals): temporary Z2 diagnostics around dashboard delete route change

* test(evals): saved-apps world.open returns once the destination page root has rendered

* test(evals): remove temporary Z2 diagnostics from saved-app-creation
yomgui added a commit that referenced this pull request Sep 11, 2026
… route-commit wait in world open() (#4810)

* fix(app): seed the delegated-task eval with a running parent thread

The eval.task_activity.seed hook rendered an in-flight child under a parent
whose thread status stayed "ready": it only marked the activity store busy.
Since #4635 MessageList reads that as a stopped parent and the unobserved
child shows "Waiting for task result" instead of the Working shimmer, so
task-activity-shimmer and parent-child-permission-approval lost their
[data-subagent-activity="shimmer"] row. A delegating parent is mid-run;
mark the eval thread streaming like the chat-loading seed already does.

* fix(app): seed a created session's snapshot so its composer can send immediately

A freshly created session had no cached snapshot, so deriveSessionRenderModel
reported "switching" while the first snapshot round trip ran and the composer
disabled Run task. On Daytona that window is ~0.9 s and a click or Enter inside
it was silently dropped with the draft intact. Seed the snapshot cache from the
create response (stale, so the real fetch still runs as a background refresh).

* fix(app): let a send that settled with a server failure recover instead of wedging

A prompt_async answered with 5xx/408 is an uncertain admission (#4635), but
the hold it created could never end: the acceptance check treated absence as
no evidence, Stop refused while the admission was unknown, and the failure body
was dropped, so the person saw a paused composer with no explanation.

- Keep the settled response body on PromptAdmissionUnknownError and show it as
  the session error card right away; the hold still prevents any resend.
- readPromptAdmission: after the exact-message GET misses, list the idle
  conversation; when native lists it without the message the POST is settled
  and did not admit, so the drain receives admission_rejected and halts like
  any definite failure (explicit retry only, never an automatic resend).
- On release the pending row is removed and the submitted composer is kept as
  an unsent message to restore; on confirmed acceptance the stale failure card
  is cleared.

* test(evals): read an empty Lexical composer as an empty draft

Lexical keeps one empty paragraph in an empty editor, which innerText reports
as "\n". readComposerState now drops that trailing line break so draftText
matches what the person sees; user.see("composer", { text }) already trimmed.

* test(evals): open the fresh conversation through the sidebar's New session button

The sidebar button was renamed from "New task" to "New session" in #4491
(Sep 5) and the "new" scenario added in #4631 (Sep 8) still targeted the old
label, so it could never find the button. Same user action, current label.

* test(evals): create the empty-session workspace at its declared tmp path

The emptySession world declared /tmp/openwork-empty-session-<ts> but
seed.workspace() adopted the existing first-launch default workspace
instead ("OpenWork Chat" under the dev profile home). On Daytona that
profile lives inside the repo checkout, so the engine merged
/workspace/.opencode/opencode.json ("permission": "allow") over the
workspace's own catch-all and composer-run-mode never saw the workspace
MCP rule take effect.

* test(evals): name the server error when a run-mode read is not 200

* fix(app): reuse the running OpenWork server when the selected workspace is server-owned

Workspaces created from the app live only in the server registry, so a
renderer reload reaches startServerWithoutDesktopWorkspace and restarted
the embedded server (and its engine) although it was healthy and already
serving that workspace. Reuse it when it is ready with the preferred
remote-access mode; only restart when nothing usable is running.

* test(evals): open the saved-app verification return tab as a person's new tab

returnVerification drove the agent browser entry point (browser.openUrl, the
IPC behind browser.open_url) with no requesting conversation. Since #4434 the
task host refuses that, and it never accepted an about:blank destination.
The tab only stands in for the person's own browser so the openwork:// return
link can be navigated without OS protocol registration, so create it the way a
person opens a new tab and find its CDP target by diffing the page targets.

* fix(app): re-read pending questions when a session reaches a terminal edge

An interrupted question tool leaves the engine's pending list through its
finalizer without a question.rejected event, so Stop on a delegated child
left its question (and option buttons) in the parent view. Reconcile
questions alongside permissions on terminal edges, for both engines.

* Revert "fix(app): re-read pending questions when a session reaches a terminal edge"

This reverts commit 9e70982.

* fix(app): withdraw the stopped tree's unanswered questions after Stop

OpenCode v1 runs tools on detached fibers: aborting a session marks the
question tool part as aborted but leaves its request in GET /question, so a
parent that stopped a delegated child kept showing the child's question and
its option buttons. After Stop confirms the tree idle, reject the pending
questions owned by the stopped sessions; other roots are left untouched.

* test(evals): temporary Z2 diagnostics around dashboard delete route change

* test(evals): saved-apps world.open returns once the destination page root has rendered

* test(evals): create the permission-stop-recovery workspace at its declared tmp path

Test 2 of parent-child-permission-approval ran bash without asking although the
world wrote permission.bash="ask" to the workspace opencode.json. The world
called seed.workspace() without create, so it adopted the first-launch default
workspace under the dev profile, which on Daytona lives inside the repo
checkout; the engine merges /workspace/.opencode/opencode.json
("permission": "allow") after the workspace file and the agent ruleset ends
as [* allow, bash ask, * allow] (last match wins; permissions/effective reported
shell allow, source engine, workspace file .../home/OpenWork Chat/opencode.jsonc).

Same trap and same remedy as #4747 for emptySession; opt-in so the other
splitPaneQuestions worlds keep their arrangement.

* fix(app): withdraw the stopped tree's unanswered permission requests after Stop

OpenCode v1 runs tools on detached fibers: aborting a session marks the tool
part as aborted but leaves its Permission.ask pending in /permission, exactly
as #4786 found for questions. The stopped tree would keep showing Allow once /
Deny in every pane and after reload although nothing can consume the answer.

After stopForegroundTree confirms every stopped session idle, also list pending
permissions and reject the ones owned by the stopped tree; other roots are
never touched and each withdrawal kind is best-effort on its own.

* test(evals): remove temporary Z2 diagnostics from saved-app-creation

* test(evals): make locate() misses list every candidate of the requested role plus route and page roots

A miss used to quote only the first 8 visible buttons in DOM order, which
are always the shell rail, so every miss read as "the page has no
buttons" (#4794's diagnosis detour). The browser-side report now lists
every rendered element of the requested role (buttons/links for
role-less targets), the current route/hash, and whether the
[data-app-header] / [data-dashboard-page] roots are mounted. The Node
side caps the list at 40 and says how many the page really had.
Matching is unchanged.

* test(evals): run the serialized locate() miss callback against a fake page DOM

Proves the browser-side branch the way Chromium will execute it: the
report lists every rendered menuitem on a dashboard whose shell rail has
nine buttons, names the route and mounted page roots, and a role-less
miss lists all nine buttons instead of the first eight.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants