Repository navigation
Plannotator Snapshots: native macOS capture HUD (launch) - #1762
Draft
backnotprop wants to merge 27 commits into
Draft
backnotprop wants to merge 27 commits into
backnotprop wants to merge 27 commits into
Conversation
… app logs permission requests
…s, Reopen only when needed, revoked states); icon slots
…ransparent) and the HUD's mark; icon scripts
…a drawn from the mascot's
The two capture modes are now Screenshot (⌥⇧⌘4, the picture) and Snapshot (⌥⇧⌘5, the picture plus the window's accessibility text). Renamed in the menu, HUD, permission card, CLI help and flag (plannotator screenshot --snapshot, also the /plannotator-screenshot argument), docs, and the internal identifiers: shot kind "snapshot", settings.snapshots, the URL scheme kind=snapshot, Swift enum cases and selectors. Also fixes 'plannotator screenshot --app' never reaching the Shots command: the annotate live-app --app flag was stripped from argv before the screenshot dispatch. --snapshot is not.
The URL command that launches the app is handled before didFinishLaunching, so begin() saved pendingShot and resumeAfterLaunch() then read that same entry as a shot from before a relaunch and switched the card to Waiting: the Allow button was gone and macOS was never asked. pendingShot now records the pid that saved it, and only an earlier process's shot resumes. Found in the signed permission test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The URL command that launches the app is handled before didFinishLaunching, so Permissions.begin() drew the card through a show hook that was not wired yet and the card never appeared: the app sat with no window. Permissions keeps the last card state and draws it the moment the hook is set. The panel now logs page load, ready, layout changes and a web-content crash, and has a navigation delegate (its didFail handlers were never called). Found in the signed permission test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ring had one side cut away, so it read as a "U" (and stood still under reduced motion). It is now a faint full ring with one bright quarter turning on it; under reduced motion it fades slowly instead of standing still. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The note on the whole image is now a field under the shot (N or the rail
button focuses it), saved with the shot, removable, and sent before the
numbered boxes ('Note on this image: …'); the headline and send summary
count notes apart from comments, and Ask includes the note.
The open-shot panel resizes from its top-left corner and top/left edges,
anchored bottom-right. The drag runs natively (PanelResizer over the web
view) so frames follow the pointer with no web round trip; the size is
kept in UserDefaults, clamped between 760x540 and the visible screen less
16 pt, and a double-click resets it.
…r-snapshot skill Product: Plannotator Snapshots (app bundle, menu bar, permission cards, HUD, docs). Command: plannotator snapshot, /plannotator-snapshot. Modes: Screen Capture (opt-shift-cmd-4, a box, picture only) and App Capture (opt-shift-cmd-5, a window plus its accessibility text, --app). Internal names follow: bundle id ai.plannotator.snapshots, URL scheme plannotator-snapshots://, data dir snapshots/, PLANNOTATOR_SNAPSHOTS and the snapshots config key, the hub, folders, Swift target, release job. --app and --static are taken from annotate's argv only (live-flags.ts), so snapshot --app reaches its command. New core skill plannotator-snapshot (core, Claude bang copy, OpenCode, Gemini, Kiro, Vibe, Copilot and Droid stubs) runs plannotator snapshot --wait.
backnotprop
changed the title
Granting Screen Recording often makes macOS quit and reopen the app. The pending capture was cleared at the grant, so the reopened app sat silent with nothing on screen and a waiting /plannotator-snapshot never returned. The pending capture now stays saved until the capture actually opens, and a relaunch that finds the permission already granted goes straight to the capture. Found in the owner's first run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The capture overlay gets a toolbar like macOS's own ⌘⇧5 bar (Cancel, Screen Capture, Window, Full Screen, App Capture, each with its key), bottom center of the display with the pointer, in its own panel above the overlay: never key, never in a capture, out of the way while a box is dragged. App Capture there (or A) picks a window and takes it with its text; without Accessibility it closes the overlay, shows the card, and reopens on App Capture. The HUD's strip and panel get a + beside a remembered Screen / App toggle (hub setting captureMode), replacing the icon-only ◫ button; "App Capture for ⌥⇧⌘4" stays in the menu bar. Co-Authored-By: Claude <noreply@anthropic.com>
…pple-signing environment Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # AGENTS.md # apps/hook/hooks/mod/controller.ts # apps/hook/hooks/mod/enabled.ts # apps/hook/hooks/mod/register.ts # apps/hook/package.json # apps/hook/server/cli.ts # package.json # packages/server/uninstall.ts
…1803) * Snapshots: App Capture turns on Chromium/Electron accessibility trees itself AXManualAccessibility is written on the application element before the walk for Chromium browsers and Electron apps (after Codex's Appshots), once per process, with a bounded 600 ms settle inside the 2 s budget. Other apps are asked only when their tree comes back empty. AXEnhancedUserInterface only as a per-capture fallback for a Chromium that refuses AXManualAccessibility. No text: the picture goes on its own. Unit tests via swift test; --selftest checks the decisions and gains --ax-app. * Snapshots: review fixes for App Capture's accessibility enablement Every AX message gets its own messaging timeout clamped to the remaining budget (children did not inherit the app element's 0.25 s), and the work stops 100 ms early so the AXEnhancedUserInterface reset fits in the 2 s. That reset now always runs unless the attribute read true beforehand. The walk is generic over a tree source; a fake tree tests the secure-field skip and the deadline. Documents the persistent AXManualAccessibility cost.
backnotprop
had a problem deploying
to
apple-signing
October 8, 2026 19:44 — with 
GitHub Actions
Failure
GitHub Actions
Failure
This was referenced Oct 8, 2026
) * Snapshots for every agent: Pi and OpenCode 2 link to the hub, on by default - packages/shared/snapshots/agent-link.ts: the Pi / OpenCode session link (summon, hub hello, pull-bridge poll, deliver claimed once across processes, Ask through the host's session bridge); vendored to Pi. - packages/ai/session-bridge-pull-client.ts: custom poll/event paths, poll extras, extra commands with an outbox post, superseded back-off. - Pi: native /plannotator-snapshot (non-blocking), Send as a followUp user message, Ask as a real turn through the Pi session bridge. - OpenCode 2: native /plannotator-snapshot beside the review commands, Send as a queued session.prompt into the root session, Ask through the OpenCode session bridge; OpenCode 1 keeps the blocking stub. - One switch, resolveSnapshotsEnabled in packages/shared/config.ts, default on; the mod's mirror defaults on too (enabled.test.ts parity). - Tests against a real in-process hub; AGENTS.md Snapshots hosts section. * Snapshots hosts: review fixes (lazy OpenCode, no lone notice, session lease) - OpenCode: no event subscription, bridge or probe until a running hub is found (macOS 5 s check); the hub going away, a deleted session and the plugin's cleanup tear it down. - OpenCode: a successful /plannotator-snapshot posts no notice; a failure's notice is tracked and a Send while it is pending is steered with it (#1515). - Shared link: the Inbox wake's session lease; only the holder says hello, polls and delivers. Nothing spawned before a hub (link and mod). - macOS only links at start; elsewhere the command links on demand. - Pi: the Send is a followUp custom message confirmed by details.sendId. - OpenCode: one question per session across bridges. - An older plannotator answers 'update Plannotator' (link and mod). * Snapshots link: a stale hub.json costs a pid check, not a fetch, lease write or git spawn
backnotprop
had a problem deploying
to
apple-signing
October 8, 2026 20:14 — with
GitHub Actions
Failure
GitHub Actions
Failure
…very once, hub input) (#1805) * Snapshots: fix the review findings (URL-scheme trust, signing secrets, delivery once, hub input) - The app's URL scheme carries only an action and a capture kind; the data dir and the CLI argv come from the app's defaults (written by the CLI with `defaults write`), and the argv runs only when every element is a trusted file. The hub registry must name loopback, the HUD web view navigates only on the hub's origin, and the message handler takes the main frame on that origin only (SnapshotsSecurity, unit tested; --selftest checks it too). - release.yml enters the apple-signing environment only on tag runs; PR and branch runs reference no secret. - Delivery is one turn across two Claude Code processes on one session: the session lease plus a per-send claim (the Inbox link's rules); the hub stops re-sending once a send is accepted and hands it out again after a hello. - `snapshot --wait` re-points an open collection that already has a destination. - install-app compares build stamps by order, keeps a newer or same build, and waits for a running app to quit before replacing it. - Hub input: send ids are plain names inside sends/, captures are verified as images by their bytes, window text only from incoming/, malformed marks are refused with 400, HUD tokens are bounded. - The agent message puts window titles and URLs on their own JSON-quoted lines and drops URL queries. - Tests for the hub, store, compose, connections, SnapshotsLink, install-app and the uninstall additions. * Snapshots review minors: 127.0.0.1-only hub origin, defaults written only by the installed CLI, signing after the eligibility gate - HubOrigin accepts only http://127.0.0.1:<port> (localhost can resolve to ::1). - The CLI writes the app's dataDir/cli defaults only from the compiled binary the install script manages (__CLI_VERSION__ set and isManagedBinary); a run from source or a test never moves the installed app's data dir. - release.yml: snapshots-macos needs release-eligibility and signs only after it passed; build and the jobs below it keep running when the gate is skipped (PRs, branches, dry runs) and stop when it failed.
- Remove snapshot from HIDDEN_SUBCOMMANDS (the set is empty again); the top-level --help lists plannotator snapshot, add and the other forms; snapshot --help describes stop, install-app, hub, --session, --no-capture - plannotator skill: a ## plannotator snapshot section, a table row and the PLANNOTATOR_SNAPSHOTS env row; the freshness test pins it released - Marketing: /docs/commands/snapshot, the env var row, quickstart and installation command lists - Pi and OpenCode READMEs: /plannotator-snapshot, on by default, PLANNOTATOR_SNAPSHOTS=0 - AGENTS.md: the Snapshots section reads as shipped - Snapshots sign check defaults to main
- tests/helpers/snapshots-world.ts: a real hub from plannotator snapshot hub --background in the world's temp HOME and data dir (no native app runs); snapshotsHubClient drives any hub as the HUD does - Pi (real Pi in RPC mode), OpenCode 2 (the packed plugin in real opencode2) and the Claude Code mod (its own code on real processes): /plannotator-snapshot, one Send delivered exactly once with the image path and the note, past the hub's re-send; Ask from the hub answered as a turn. Scripted model only - CI: the Inbox Pi / Claude Code jobs and the OpenCode 2 installed-package job run them against the compiled binary; path filters cover the hub
backnotprop
changed the title
…stall it /plannotator-snapshot answered "could not start" when the CLI summoned the session and then found no app, though the hub was up and the session linked. The mod, Pi and OpenCode now say one sentence (SNAPSHOTS_APP_MISSING_TEXT, the mod's copy pinned equal): linked, run `plannotator snapshot install-app`. The CLI's own line names install-app too. A test per host, the live proofs assert it on macOS, and a test ties the stub to the CLI's text.
# Conflicts: # AGENTS.md
…l; docs say add --screen needs macOS - The mod's /plannotator-snapshot now checks macOS (the file only macOS has) before it runs the CLI, and off macOS gives the text Pi and OpenCode give (SNAPSHOTS_MACOS_ONLY_TEXT, copy pinned equal), runs nothing and links on demand. A 0.28.8 binary on Linux no longer reads as "update Plannotator". - "Not installed" is reached only by a plannotator built without the embedded app, where install-app fails too: every host now says the session is linked and to reinstall with the install script, or use snapshot open. The CLI's line says the same. - Skill and docs: add <file>, open, status, stop and hub run off macOS; add --screen uses screencapture and needs macOS.
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plannotator Snapshots for macOS, the launch PR. Press ⌥⇧⌘4 (Screen Capture). The screen freezes; drag a box, mark it with numbered boxes and comments, and press ⌘↩. The agent session gets one message with each image's path, your notes and each box's rectangle and comment. Nothing blocks the session, and Ask (⌘J) gets an answer from the session itself, streamed into the HUD. ⌥⇧⌘5 takes an App Capture: the frontmost window plus its accessibility text.
Merging this ships it:
plannotator snapshotis out ofHIDDEN_SUBCOMMANDS(the set is empty again) and documented.What ships
apps/snapshots-macos, "Plannotator Snapshots.app", Swift/AppKit, macOS 14+, universal). It is embedded in the darwin binaries (index-darwin.ts). The firstplannotator snapshotinstalls it into~/Applicationsand opens it through LaunchServices, so the app holds Screen Recording (and, for App Capture, Accessibility) itself. Nothing in your terminal inherits those permissions. Carbon hotkeys need no permission. The URL scheme carries only an action and a capture kind.packages/server/snapshots,plannotator snapshot hub): the store under<data dir>/snapshots/, routing (the summoning session, then the one you typed into in the last 15 minutes, then the only one), delivery and Ask, over the pull-bridge protocol.packages/snapshots-hud): the strip, the panel, the marking canvas, View text, the Ask pane, the ⌘K picker and the no-session fallbacks.plannotator snapshot [--app] [--wait] [--session] [--no-capture], plusadd,open,status,stop,install-appandhub. Capture is macOS only. The other subcommands run anywhere./plannotator-snapshot: a core skill (plannotator snapshot --wait) with command stubs for the other hosts. The installer ships it.Defaults and hosts
On by default for every agent. One switch turns it off:
PLANNOTATOR_SNAPSHOTS=0or{ "snapshots": false }inconfig.json. It is read once per session start. Theplannotator snapshotCLI is unaffected./plannotator-snapshotfollowUpmessageplannotator snapshot --waitSigning
snapshots-macosjob builds the app on macOS. Onv*tag pushes it signs with the Developer ID, enables the hardened runtime, then notarizes and staples the app. Other runs build it ad-hoc signed.apple-signing, which admits onlymainandv*tags. The job enters it only afterrelease-eligibilitypasses, so pull request and branch runs reference no secret:APPLE_DEVELOPER_ID_P12APPLE_DEVELOPER_ID_P12_PASSWORDAPPLE_DEVELOPER_ID_IDENTITYAPPLE_NOTARY_KEY_P8APPLE_NOTARY_KEY_IDAPPLE_NOTARY_ISSUER_IDSnapshots sign check(snapshots-sign-check.yml, run it by hand) proves the secrets without making a release. It builds, signs, notarizes, staples and verifies. Its default ref is nowmain. Its last run passed.Messages a host shows
plannotatorbuilt without the Mac app: the session is linked, so the message says to reinstall with the install script, or useplannotator snapshot open.install-appcannot help in that case.Docs
plannotator --helpandplannotator snapshot --help.plannotatorskill has a## plannotator snapshotsection, kept fresh byplannotator-skill-reference.test.ts./docs/commands/snapshot/, plus the env var row and the quickstart and installation command lists.Proof
tests/helpers/scripted-model.ts) and a real hub started byplannotator snapshot hub --backgroundin a temp HOME and data dir (tests/helpers/snapshots-world.ts). The native app never runs. The test plays the HUD through the hub API: capture a PNG, add a note, Send, Ask.apps/pi-extension/snapshots-live.test.ts, the real Pi in RPC mode):/plannotator-snapshot, then one Send delivered exactly once, still once after the hub's 5 s re-send, carrying the image path and the note. The hub reports it delivered. Ask runs as a turn and returns the model's text.apps/opencode-plugin/snapshots-v2.test.ts, the packed plugin in realopencode22.0.19): the native command links the session. One Send arrives as one queued turn, byte for byte the composed text, still once after the re-send. Ask runs as a turn.apps/hook/hooks/mod/snapshots-live.test.ts, the mod's own code on real processes):/plannotator-snapshotthrough the mod. One plugin turn is byte-identical to the composed text, still once after 7 s. Ask runs as a turn./plannotator-snapshotlinks the session on demand and the test summons it as the CLI would). Their path filters now cover the hub. They also passed locally on macOS with the CLI from source.vendor.sh'ssnapshots/loop. It had been failing since the Pi host landed, and became visible once the Pi job ran on this branch.bun run typecheck,bun test, the mod engine harness,build:hook, the Pi parity gate (vendor.sh+bun pm pack --dry-run),check-release-versionand actionlint all pass.--selftest, and the compiled darwin binary installing the embedded app.How to try it
/plannotator-snapshot. Anywhere else, runplannotator snapshot --wait.plannotator snapshot statusto see the hub and sessions.Not built yet