Repository navigation
Snapshots: App Capture turns on Chrome/Electron accessibility itself - #1803
Merged
Merged
Conversation
… itself AXManualAccessibility is written on the application element before the walk for Chromium browsers and Electron apps (after Codex's Appshots), once per process, with a bounded 600 ms settle inside the 2 s budget. Other apps are asked only when their tree comes back empty. AXEnhancedUserInterface only as a per-capture fallback for a Chromium that refuses AXManualAccessibility. No text: the picture goes on its own. Unit tests via swift test; --selftest checks the decisions and gains --ax-app.
Every AX message gets its own messaging timeout clamped to the remaining budget (children did not inherit the app element's 0.25 s), and the work stops 100 ms early so the AXEnhancedUserInterface reset fits in the 2 s. That reset now always runs unless the attribute read true beforehand. The walk is generic over a tree source; a fake tree tests the secure-field skip and the deadline. Documents the persistent AXManualAccessibility cost.
backnotprop
marked this pull request as ready for review
October 8, 2026 19:44
backnotprop
added a commit
that referenced
this pull request
Oct 8, 2026
…o snapshots-review-fixes Conflicts resolved as a union: - mod snapshots.ts: the mod's own lease and claim kept, with #1804's older-CLI answer, the quiet loop end, and nothing spawned (no git, claim folder or lease) until a hub exists. - mod snapshots.test.ts: both sets of tests; the off-switch test follows the new on-by-default rule. - SelfTest.swift: the security checks and #1803's enablement checks both count toward the exit status. Package.swift: both test targets. - packages/shared/package.json: both exports. AGENTS.md: #1804's text with the review fixes applied (install rules, URL scheme, hub resend, message format, one lease and claim per host).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
App Capture (⌥⇧⌘5, toolbar App Capture) no longer needs Chrome's accessibility mode turned on by hand.
What it does (
AXEnablement.swift, after Codex's AppshotsenableAccessibilityIfNeeded):* Framework.framework) and Electron apps (Electron Framework.framework: Slack, VS Code, Discord, Notion, Cursor, …):AXManualAccessibility = trueon the application element BEFORE the walk, then poll up to 600 ms for content.AXManualAccessibilityonly when the tree comes back empty (an unknown attribute is refused harmlessly).AXEnhancedUserInterfaceis not written by default: AppKit animates every window frame change while it is on (window managers feel it) and Chromium treats it as a screen reader. Only a known Chromium that refusesAXManualAccessibilitygets it, for that capture only: it is reset to false afterwards whether it read as off or could not be read, and left alone only when it already readtrue(a screen reader set it).textUnavailable: "this app gave no text, so the picture is sent on its own".Restore policy:
AXManualAccessibilityis written once per process (pid + launch date) and never toggled per capture.For users: after the first App Capture of Chrome (or Slack, VS Code, …), that app keeps its accessibility tree until it quits. That costs it a little CPU and memory while it runs, the same as when VoiceOver, a password manager or any other assistive app is running. Nothing is saved; quitting the app ends it.
Why not restore: Turning it off after each capture would make Chrome drop and rebuild the tree (seconds on a big page) every time; nothing persists to disk, and it is the state any assistive app leaves Chrome in. If the tree is empty again later, writing
trueagain is not a toggle.Tests:
swift testinapps/snapshots-macos(new test target, 16 tests: bundle/framework detection, step, fallback, the reset rule, messaging timeout clamp, settle time inside budget, pid reuse, exclusion never asked; the walk is generic over a tree source, so a fake tree checks the secure-field skip (only role and subrole are queried, nothing of its children) and that a slow tree stops at the deadline).--selftestchecks the decisions, printstree: …, exits 1 on a failed check or a budget overrun, and gains--ax-app <bundle id>. The first revision's--selftestrun here wroteAXManualAccessibilityto the frontmost app (Cursor), so it is not run against the owner's apps again; the review fixes are covered byswift test. The full-capture budget with the reset is bounded by construction (per-message clamp + 100 ms reserve); a live timing against an unresponsive app was not measured.Owner live check: quit and reopen Chrome (so a11y is off), open a page with text (e.g. example.com), press ⌥⇧⌘5; the snapshot's text should hold the page text (
page …/heading Example Domain). The line in~/.plannotator/snapshots/app.logreadsapp text com.google.Chrome: chromium: AXManualAccessibility set, tree ready in N ms. A second capture shows no write. Terminal alternative:PlannotatorSnapshots --selftest /tmp/x --ax-app com.google.Chromefrom a terminal with Accessibility.