From a50f303efa590c5ed628efeefdbb2087c198457f Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 14:36:04 -0400 Subject: [PATCH 01/12] Add subtext-ios-sim skill: local iOS Simulator loop (preview) Drive and inspect an iOS app in the local Simulator the way sightmap browser does for web: an FBSimulatorControl-based helper for the accessibility tree and HID input, a dependency-free Node CLI and daemon (subtext-sim), sightmap component matching on iOS trees, live console from the device log, and a run log for producer/consumer sightmap authoring. Includes a SwiftUI fixture app and an e2e script. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- .changeset/ios-sim-loop.md | 5 + docs/ios-sim-loop/PLAN.md | 104 + skills/subtext-ios-sim/README.md | 102 + skills/subtext-ios-sim/SKILL.md | 128 + skills/subtext-ios-sim/bin/subtext-sim | 4 + .../fixtures/SampleApp/.sightmap/.gitignore | 2 + .../SampleApp/.sightmap/components.yaml | 9 + .../SampleApp/.sightmap/views/login.yaml | 12 + .../SampleApp/.sightmap/views/products.yaml | 15 + .../fixtures/SampleApp/Info.plist | 19 + .../SampleApp/Sources/FullstoryBridge.swift | 26 + .../SampleApp/Sources/SampleApp.swift | 209 + .../fixtures/SampleApp/build.sh | 70 + skills/subtext-ios-sim/helper/.gitignore | 2 + skills/subtext-ios-sim/helper/Package.swift | 45 + skills/subtext-ios-sim/helper/README.md | 50 + .../Sources/SubtextSimHelper/Describe.swift | 89 + .../Sources/SubtextSimHelper/Keyboard.swift | 75 + .../Sources/SubtextSimHelper/Protocol.swift | 83 + .../SubtextSimHelper/SimulatorRegistry.swift | 96 + .../Sources/SubtextSimHelper/main.swift | 134 + skills/subtext-ios-sim/helper/build.sh | 74 + .../helper/testdata/describe-bench-eager.json | 6065 +++++++++++++++++ .../helper/testdata/describe-bench-heavy.json | 305 + .../helper/testdata/describe-bench.json | 167 + skills/subtext-ios-sim/lib/BRIDGE.md | 62 + skills/subtext-ios-sim/lib/cli.mjs | 444 ++ skills/subtext-ios-sim/lib/client.mjs | 84 + skills/subtext-ios-sim/lib/corpus.mjs | 53 + skills/subtext-ios-sim/lib/daemon.mjs | 567 ++ skills/subtext-ios-sim/lib/doctor.mjs | 154 + skills/subtext-ios-sim/lib/errors.mjs | 36 + skills/subtext-ios-sim/lib/helper-client.mjs | 109 + skills/subtext-ios-sim/lib/logs.mjs | 165 + skills/subtext-ios-sim/lib/normalize.mjs | 153 + skills/subtext-ios-sim/lib/runlog.mjs | 66 + skills/subtext-ios-sim/lib/selector.mjs | 264 + .../subtext-ios-sim/lib/sightmap-bridge.mjs | 389 ++ skills/subtext-ios-sim/lib/sightmap-cli.mjs | 79 + skills/subtext-ios-sim/lib/simctl.mjs | 95 + skills/subtext-ios-sim/test/axe-to-helper.mjs | 39 + skills/subtext-ios-sim/test/bridge.test.mjs | 261 + skills/subtext-ios-sim/test/cli.test.mjs | 185 + skills/subtext-ios-sim/test/e2e.sh | 129 + .../test/fakes/fake-helper.mjs | 36 + .../subtext-ios-sim/test/fixtures/corpus.json | 214 + .../test/fixtures/describe-deep.json | 1 + .../test/fixtures/describe-eager.json | 1 + .../test/fixtures/describe-heavy.json | 1 + .../test/fixtures/describe-simple.json | 1 + .../test/fixtures/parity-corpus.json | 101 + .../fixtures/sightmap-parity/components.yaml | 33 + .../test/fixtures/sightmap/components.yaml | 19 + .../test/fixtures/sightmap/views/eager.yaml | 10 + .../test/fixtures/sightmap/views/heavy.yaml | 33 + .../test/fixtures/sightmap/views/simple.yaml | 20 + 56 files changed, 11694 insertions(+) create mode 100644 .changeset/ios-sim-loop.md create mode 100644 docs/ios-sim-loop/PLAN.md create mode 100644 skills/subtext-ios-sim/README.md create mode 100644 skills/subtext-ios-sim/SKILL.md create mode 100755 skills/subtext-ios-sim/bin/subtext-sim create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/.gitignore create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/Info.plist create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/Sources/FullstoryBridge.swift create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/Sources/SampleApp.swift create mode 100755 skills/subtext-ios-sim/fixtures/SampleApp/build.sh create mode 100644 skills/subtext-ios-sim/helper/.gitignore create mode 100644 skills/subtext-ios-sim/helper/Package.swift create mode 100644 skills/subtext-ios-sim/helper/README.md create mode 100644 skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift create mode 100644 skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift create mode 100644 skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Protocol.swift create mode 100644 skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/SimulatorRegistry.swift create mode 100644 skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/main.swift create mode 100755 skills/subtext-ios-sim/helper/build.sh create mode 100644 skills/subtext-ios-sim/helper/testdata/describe-bench-eager.json create mode 100644 skills/subtext-ios-sim/helper/testdata/describe-bench-heavy.json create mode 100644 skills/subtext-ios-sim/helper/testdata/describe-bench.json create mode 100644 skills/subtext-ios-sim/lib/BRIDGE.md create mode 100644 skills/subtext-ios-sim/lib/cli.mjs create mode 100644 skills/subtext-ios-sim/lib/client.mjs create mode 100644 skills/subtext-ios-sim/lib/corpus.mjs create mode 100644 skills/subtext-ios-sim/lib/daemon.mjs create mode 100644 skills/subtext-ios-sim/lib/doctor.mjs create mode 100644 skills/subtext-ios-sim/lib/errors.mjs create mode 100644 skills/subtext-ios-sim/lib/helper-client.mjs create mode 100644 skills/subtext-ios-sim/lib/logs.mjs create mode 100644 skills/subtext-ios-sim/lib/normalize.mjs create mode 100644 skills/subtext-ios-sim/lib/runlog.mjs create mode 100644 skills/subtext-ios-sim/lib/selector.mjs create mode 100644 skills/subtext-ios-sim/lib/sightmap-bridge.mjs create mode 100644 skills/subtext-ios-sim/lib/sightmap-cli.mjs create mode 100644 skills/subtext-ios-sim/lib/simctl.mjs create mode 100644 skills/subtext-ios-sim/test/axe-to-helper.mjs create mode 100644 skills/subtext-ios-sim/test/bridge.test.mjs create mode 100644 skills/subtext-ios-sim/test/cli.test.mjs create mode 100755 skills/subtext-ios-sim/test/e2e.sh create mode 100644 skills/subtext-ios-sim/test/fakes/fake-helper.mjs create mode 100644 skills/subtext-ios-sim/test/fixtures/corpus.json create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-deep.json create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-eager.json create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-heavy.json create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-simple.json create mode 100644 skills/subtext-ios-sim/test/fixtures/parity-corpus.json create mode 100644 skills/subtext-ios-sim/test/fixtures/sightmap-parity/components.yaml create mode 100644 skills/subtext-ios-sim/test/fixtures/sightmap/components.yaml create mode 100644 skills/subtext-ios-sim/test/fixtures/sightmap/views/eager.yaml create mode 100644 skills/subtext-ios-sim/test/fixtures/sightmap/views/heavy.yaml create mode 100644 skills/subtext-ios-sim/test/fixtures/sightmap/views/simple.yaml diff --git a/.changeset/ios-sim-loop.md b/.changeset/ios-sim-loop.md new file mode 100644 index 0000000..67edb7c --- /dev/null +++ b/.changeset/ios-sim-loop.md @@ -0,0 +1,5 @@ +--- +"subtext": minor +--- + +Add the `subtext-ios-sim` skill (preview): drive and inspect an iOS app in the local Simulator — annotated accessibility snapshots, taps and typing by identifier or sightmap component, live console, and a run log for producer/consumer sightmap authoring. diff --git a/docs/ios-sim-loop/PLAN.md b/docs/ios-sim-loop/PLAN.md new file mode 100644 index 0000000..135e7db --- /dev/null +++ b/docs/ios-sim-loop/PLAN.md @@ -0,0 +1,104 @@ +# iOS Simulator live local loop: build plan + +**Goal:** an agent drives and inspects an iOS app in the local Simulator the same way `sightmap browser` does for web. It snapshots the UI tree, acts on components by id or component query, reads console output, and authors `.sightmap/` from what it sees. Everything runs on the developer's Mac: no Appium, no cloud, Simulator only. + +**Status:** prototype on branch `chandler/ios-live-loop`. It ships as a first-party skill in the Subtext plugin, so it can be tested on any Mac by installing the plugin from this branch. The intent is to move the command surface upstream as `sightmap device …` once it's proven. + +## Shape + +``` +agent ──► subtext-sim CLI ──► daemon ──► sim helper (FBSimulatorControl) ──► iOS Simulator + │ └──► simctl (lifecycle, screenshot, log stream) + └──► .sightmap/ matcher (sightmap CLI, offline) +network (v1): app's Fullstory SDK session ──► Subtext MCP review-open / review-zoom +``` + +| Piece | Path | What it is | +|---|---|---| +| Skill | `skills/subtext-ios-sim/SKILL.md` | Teaches agents the workflow and commands | +| CLI | `skills/subtext-ios-sim/bin/subtext-sim` + `lib/*.mjs` | Node ≥18, **no npm dependencies**. It mirrors `sightmap browser` commands. | +| Daemon | `lib/daemon.mjs` | Long-lived; owns the helper process and the device log stream. The CLI talks to it over a Unix socket recorded in `.sightmap/.sim-session`. | +| Helper | `skills/subtext-ios-sim/helper/` | Swift package built on FBSimulatorControl through idb's released binary xcframeworks (pinned). It builds on first use into `~/Library/Caches/subtext-sim//`. | +| Fixture app | `skills/subtext-ios-sim/fixtures/SampleApp/` | A small SwiftUI app with `accessibilityIdentifier`s, a long list, a form and network calls, plus `build.sh`. It's used for the demo and for testing. | + +## Helper protocol (stable contract between CLI and helper) + +The helper is a long-lived process speaking JSON lines over stdin/stdout, one request per line: + +```json +{"id": 1, "cmd": "describe", "args": {"udid": "…"}} +{"id": 1, "ok": true, "result": {…}} +{"id": 2, "ok": false, "error": {"code": "not_booted", "message": "…"}} +``` + +| cmd | args | result | +|---|---|---| +| `hello` | — | `{version, xcode, fbsimctl}` | +| `describe` | `udid` | `{screen: {w, h}, root: Node}` where `Node = {role, type, label, identifier, value, frame: {x, y, w, h}, enabled, children: [Node]}`. `identifier` is `AXUniqueId`; points are top-left origin. | +| `tap` | `udid, x, y` | `{}` | +| `type` | `udid, text` | `{}` | +| `swipe` | `udid, x1, y1, x2, y2, duration?` | `{}` | +| `button` | `udid, name` (`home`, `lock`, `siri`) | `{}` | +| `key` | `udid, key` (`return`, `delete`, `tab`…) | `{}` | + +Lifecycle (boot, install, launch, terminate, openurl), screenshots and logs go through `xcrun simctl` from Node, not the helper. + +## CLI surface (v1) + +| Command | Behavior | +|---|---| +| `subtext-sim doctor` | Checks Xcode (not just the CLT), simctl, iOS runtimes, the Swift toolchain, Node, the sightmap CLI and the helper build; prints exact fixes. Builds the helper if needed. | +| `subtext-sim start [--device NAME\|UDID] [--app PATH] [--bundle-id ID] [--detach]` | Boots the simulator, enables accessibility, installs and launches the app, starts the daemon, writes `.sightmap/.sim-session`. | +| `subtext-sim status` / `stop` | Reports session health / shuts everything down. | +| `subtext-sim snapshot [--out F] [--tree-out F] [--json]` | describe → normalized tree → sightmap matching → annotated tree with stable per-snapshot ids. | +| `subtext-sim click ` | Resolves to an element, **refuses off-screen targets** (suggests scrolling), taps its center. | +| `subtext-sim fill ` | Taps, then types. | +| `subtext-sim scroll [--in id\|query]` / `swipe` / `tap-xy` / `button` / `key` | Lower-level input. | +| `subtext-sim openurl ` / `screenshot --out F` | Deep links / PNG. | +| `subtext-sim console list [--limit N] [--grep S]` / `console get ` | App process logs from the device log stream, live. | +| `subtext-sim session-url` | The Fullstory session URL from the SDK's log output, when the SDK is present. | + +**Network (v1):** read through the Subtext MCP. The agent passes `session-url` to `review-open`, then calls `review-zoom` with `{"network": "standard"}` (or `"detail"` for headers and bodies). That data arrives on the SDK's upload interval, roughly 20 s, so the skill tells agents to batch: run the flow, then read once. Faster or live delivery depends on SDK work outside this branch. + +**Producer / consumer roles:** the skill splits the loop into two subagent roles, so the upload delay never blocks anyone. +- **Producer:** drives the app with `subtext-sim` (start, snapshot, click/fill/scroll, openurl). It works through flows quickly and writes a run log to `.sightmap/.sim-runs/.jsonl`. Each entry records a step: timestamp, action, target component or id, screen and view. The log also records the session URL. +- **Consumer:** follows the producer's run log. It reads that session's captured data through the Subtext MCP (`review-open`, then `review-zoom` on network, console and interaction), lining the data up against the producer's steps. With it, the consumer authors or cleans up `.sightmap/`: + - names `requests:` and `properties:`; + - adds `messages:`; + - fixes components and views the producer found unnamed or ambiguous. + + It re-reads as new data lands, roughly every upload interval. + +The orchestrating agent runs both, or the producer runs alone when only UI work is needed. Data the consumer reads depends on the org's mobile capture settings: network capture must be on for network data; resource uploading isn't needed. + +**Element model:** `role ← type`, `identifier ← AXUniqueId`, `text ← label`, `value`, `bounds`. These map onto sightmap selectors as `[identifier="…"]`, `[text="…"]`, `[value="…"]`, so `.sightmap/` YAML stays platform-neutral. + +**Deviations found while building:** +- Enabling app accessibility only takes effect at boot, so the first `start` on a fresh simulator reboots it once. +- `stop` keeps the simulator running unless `--shutdown` is passed. +- Rows in long lists that haven't been rendered yet report "not found"; `scroll --until` handles it. + +## Build phases + +1. **In parallel:** + - **A, helper:** the Swift helper and `build.sh`, tested against a booted simulator. + - **B, sightmap bridge:** the normalizer and sightmap bridge (the offline tree format, matching, annotated output), developed against recorded `describe` fixtures. + - **C, app, skill and install path:** the fixture app, the skill draft, `doctor`, and installing the plugin from a branch. +2. **D, CLI and daemon:** wire up the helper, simctl, the log stream and the bridge into the commands above. +3. **E, verification:** + - local end to end on the fixture app; + - a clean-machine dry run: fresh clone of the branch outside any dev checkout, isolated plugin config and cache, plugin installed from the clone, `doctor` → `start` → `snapshot` → `click` → `console`; + - after the branch is pushed, a test on a second Mac from the branch. + +## Testing on another Mac + +1. Prerequisites: macOS 15+, Xcode 26+ with an iOS Simulator runtime, Node 18+. +2. Install the plugin from the branch (exact command is in the skill's README once verified), then ask the agent to run `subtext-sim doctor`. +3. Try the fixture app: `fixtures/SampleApp/build.sh`, then `subtext-sim start --app `. + +## Known limits (v1) + +- Simulator only; Apple Silicon only (idb's prebuilt frameworks are arm64). +- No hover, file upload or arbitrary viewport sizes. +- Network data isn't live, and depends on the app's Fullstory SDK and org network-capture settings. +- The helper relies on Apple private frameworks through FBSimulatorControl, so new Xcode releases may need a helper update. diff --git a/skills/subtext-ios-sim/README.md b/skills/subtext-ios-sim/README.md new file mode 100644 index 0000000..5ab5ee8 --- /dev/null +++ b/skills/subtext-ios-sim/README.md @@ -0,0 +1,102 @@ +# subtext-ios-sim (prototype) + +Drive and inspect an iOS app in the local iOS Simulator from a coding agent. The agent-facing instructions are in [`SKILL.md`](SKILL.md); this file is for people testing the prototype from the `chandler/ios-live-loop` branch. + +## Requirements + +- Apple Silicon Mac, macOS 15 or later +- Xcode 26 or later, opened once, with an iOS Simulator runtime (Xcode > Settings > Components) +- `xcode-select -p` pointing at Xcode, not the Command Line Tools +- Node 18 or later + +## Install from the branch (Claude Code) + +The plugin's marketplace (`subtext-marketplace`) is this repo, so add it pinned to the branch and install `subtext` from it: + +```sh +claude plugin marketplace add 'fullstorydev/subtext#chandler/ios-live-loop' +claude plugin install subtext@subtext-marketplace +``` + +Or in one step: + +```sh +claude plugin install subtext --marketplace 'fullstorydev/subtext#chandler/ios-live-loop' +``` + +`https://github.com/fullstorydev/subtext.git#chandler/ios-live-loop` works too. Restart Claude Code afterwards. + +**Already have Subtext installed from `main`?** The marketplace name is already taken by the `main` source, and `add` refuses a different source for it. Remove it first (this uninstalls the plugin), then run the commands above: + +```sh +claude plugin marketplace remove subtext-marketplace +``` + +To go back to the released plugin later: remove the marketplace again, then `claude plugin marketplace add fullstorydev/subtext` and `claude plugin install subtext@subtext-marketplace`. + +**Picking up new commits on the branch.** The plugin version doesn't change between branch commits, so `claude plugin update` reports "already at the latest version". Refresh the marketplace and reinstall instead: + +```sh +claude plugin marketplace update subtext-marketplace +claude plugin uninstall subtext@subtext-marketplace +claude plugin install subtext@subtext-marketplace +``` + +**Where the files land:** `~/.claude/plugins/cache/subtext-marketplace/subtext//` (this skill is under `skills/subtext-ios-sim/` there); the marketplace clone is in `~/.claude/plugins/marketplaces/subtext-marketplace/`. `~/.claude/plugins/installed_plugins.json` records the installed commit (`gitCommitSha`). + +## Other harnesses + +These were not tested against the branch. + +- **Gemini CLI:** `gemini extensions install https://github.com/fullstorydev/subtext --ref chandler/ios-live-loop` +- **Codex, Cursor:** the plugin manifests point at the repo's `skills/` directory. Clone the branch (`git clone -b chandler/ios-live-loop https://github.com/fullstorydev/subtext`) and install the plugin from that local checkout. + +## Try it + +Ask the agent: + +1. "Run the subtext-sim doctor." Every failing check prints the exact fix. The first run builds the simulator helper, which takes a minute or two. +2. "Build the subtext-ios-sim sample app and start it in the simulator." The agent runs `fixtures/SampleApp/build.sh`, then `subtext-sim start --app `. +3. "Sign in with any email and password, open Product 150, and add it to the cart. Then show me the app's console output." + +You can also run the pieces yourself from the installed skill directory: + +```sh +cd ~/.claude/plugins/cache/subtext-marketplace/subtext/*/skills/subtext-ios-sim +node bin/subtext-sim doctor +app="$(fixtures/SampleApp/build.sh)" # prints the .app path +cd fixtures/SampleApp # its .sightmap/ annotates the snapshots +node ../../bin/subtext-sim start --app "$app" --detach +node ../../bin/subtext-sim snapshot +node ../../bin/subtext-sim stop +``` + +## Tests + +```sh +node --test test/*.test.mjs # unit tests; a fake helper stands in for the simulator +test/e2e.sh --device "" # drives the sample app on that simulator, then shuts it down +``` + +`e2e.sh` boots the simulator if needed, signs in, scrolls to Product 150, adds it to the cart, and checks the console, a screenshot and the run log. Set `E2E_KEEP_BOOTED=1` to leave the simulator running. + +## Sample app + +`fixtures/SampleApp/` is a small SwiftUI app, "Subtext Sim Sample" (`dev.subtext.SimSample`), with: + +- a login form (any email containing `@` and a non-empty password signs in); +- a tab bar with Products, Network and Account tabs; +- a list of 200 products, each opening a detail screen with an Add to Cart button; +- a Send Request button that POSTs JSON to `https://httpbin.org/anything/…` and logs the result through `print` and `os_log`. + +Every control has an `accessibilityIdentifier` (`login-email`, `login-submit`, `tab-network`, `product-row-42`, `product-detail-add`, `network-send`…). + +`build.sh` needs only Xcode: it compiles with `swiftc` and assembles the bundle by hand, so there's no Xcode project. It writes to `~/Library/Caches/subtext-sim/SampleApp/` (set `OUT_DIR` to change that) and prints the `.app` path on stdout. + +To capture sessions with Fullstory, build with your org ID: + +```sh +FS_ORG_ID= fixtures/SampleApp/build.sh +``` + +This downloads the public Fullstory iOS SDK (the binary from [`fullstorydev/fullstory-swift-package-ios`](https://github.com/fullstorydev/fullstory-swift-package-ios), checksum-verified; set `FS_SDK_VERSION` to pick a release), embeds it, adds a `Fullstory` dictionary with `OrgId` and `RecordOnStart` to `Info.plist`, and logs `FULLSTORY_SESSION_URL ` when a session starts. That line is what `subtext-sim session-url` reads. diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md new file mode 100644 index 0000000..21f83a6 --- /dev/null +++ b/skills/subtext-ios-sim/SKILL.md @@ -0,0 +1,128 @@ +--- +name: subtext-ios-sim +description: Drive and inspect an iOS app in the local iOS Simulator — boot a simulator, install and launch the app, snapshot the UI tree, tap and type by accessibility identifier or sightmap component, read the app's console, and read its network traffic through the Subtext MCP. Use when the user wants to run, try, test, or reproduce a flow in an iOS app on the Simulator, inspect an iOS screen, or author `.sightmap/` definitions for an iOS app. +--- + +# iOS Simulator loop + +> **PREREQUISITE:** Read `subtext-shared` for MCP conventions. For `.sightmap/` schema, read `sightmap-authoring`. + +**Type:** Workflow — observe, act, observe, on a local simulator. + +`subtext-sim` drives an app in the iOS Simulator on this Mac the way `sightmap browser` drives Chrome: snapshot the accessibility tree (annotated with your `.sightmap/` components), act on elements by id or component query, read console output. Simulator only; nothing leaves the Mac except the app's own traffic. + +## Prerequisites + +Apple Silicon Mac, macOS 15+, Xcode 26+ selected with `xcode-select` (not just the Command Line Tools), an iOS Simulator runtime, Node 18+. Run `doctor` first; it prints the exact fix for anything missing and builds the simulator helper on first use (a minute or two, cached afterwards). + +## Invoking the CLI + +The CLI is `bin/subtext-sim` inside **this skill's base directory** (the path you were given when this skill loaded). Always run it with `node` and the absolute path: + +```bash +node /bin/subtext-sim doctor +``` + +Below, `subtext-sim` is shorthand for that. Run commands from the project root: the session file lives in `.sightmap/.sim-session`. + +## The loop + +```bash +subtext-sim start --app path/to/MyApp.app --detach # or --bundle-id for an installed app; --device "iPhone 17 Pro" or a UDID +subtext-sim snapshot # annotated tree; each element gets an id for this snapshot +subtext-sim click login-submit # by accessibility identifier, snapshot id, or component query +subtext-sim fill login-email "a@example.com" +subtext-sim snapshot # re-snapshot after every action that changes the screen +subtext-sim console list --grep error # app logs (print / os_log), live +subtext-sim stop # --shutdown also shuts the simulator down +``` + +1. **Start** once per session, always with `--detach` (without it, `start` holds the shell as a foreground daemon). `status` reports health; `start` again reinstalls (with `--app`) and relaunches the app. The first `start` on a simulator that has never had app accessibility enabled reboots it once. +2. **Snapshot** before acting. Ids are per-snapshot, so act on the latest one. Prefer accessibility identifiers or component names over snapshot ids in anything you write down. +3. **Act** with `click` / `fill`. Both re-read the screen just before tapping, so a component query or identifier never goes stale. Lower level: `scroll`, `swipe`, `tap-xy x y`, `type`, `key return`, `button home`, `openurl `. +4. **Check** with `snapshot` (state), `console list` / `console get ` (logs), `screenshot --out shot.png` (visual evidence). + +**Targets** are a snapshot id (`12`), an exact accessibility identifier (`login-submit`), or a component query: `SignInButton`, `ProductRow[title^="Product 15"]`, `LoginForm SignInButton`, or a role with attributes for unmapped apps, `button[identifier="login-submit"]`. A query matching several elements errors and lists them; add a predicate or append `#N`. + +**Off-screen elements:** `click` refuses targets whose center is off the screen or under the tab bar, toolbar or keyboard, and says what to do. Long SwiftUI/UIKit lists only expose the rows near the screen, so a row further down is "not found" until you scroll. `scroll down --until ` scrolls (slowly enough not to fling past it) until the target is tappable, then refreshes the snapshot ids. + +**Waiting:** actions return as soon as the tap lands; navigation animates for a moment after. If a snapshot still shows the old screen, snapshot again. + +## Commands + +| Command | What it does | +|---|---| +| `doctor [--fix]` | Checks Xcode, simulators, Swift, Node, the sightmap CLI; builds the helper. | +| `start [--device NAME\|UDID] [--app PATH] [--bundle-id ID] [--label L] [--detach]` | Boots, enables app accessibility, installs, launches, starts the daemon and a new run log. | +| `status` / `stop [--shutdown]` | Session health / stop the daemon and the app (and the simulator). | +| `launch` / `relaunch` | Foreground the app / restart it (re-enables accessibility first). | +| `snapshot [--view V] [--include-offscreen] [--out F] [--tree-out F] [--json]` | Annotated tree. `--out` writes the text plus a `.tree.json` that `sightmap coverage` / `explain --snap` read. | +| `click ` | Re-resolves, checks it's on screen, uncovered and enabled, taps its center. | +| `fill [--clear]` | Taps, types (appends unless `--clear`), and errors if the field doesn't show the text. | +| `scroll [--in ] [--until ] [--max N]` | One page, or until a target is tappable. `--in` scrolls inside that element. | +| `swipe x1 y1 x2 y2 [--duration S]`, `tap-xy x y`, `type `, `key `, `button `, `openurl ` | Raw input in screen points. Keys: `return delete tab escape space` and arrows. Buttons: `home lock siri side`. | +| `screenshot --out F.png` | PNG of the screen. | +| `console list [--limit N] [--grep S] [--level L]` / `console get ` | The app's os_log/Logger output (Apple frameworks only at error/fault) and its stdout/stderr (levels `stdout`, `stderr`). | +| `session-url` | The Fullstory session URL from the app's logs. | +| `run new [--label L]` / `run current` | Start a new run log / print the current run's id and path. | +| `run tail [--follow] [--since SEQ] [--run ID]` | Print `.sightmap/.sim-runs/.jsonl` entries; `--follow` keeps streaming. | + +Every command takes `--json`. Errors are one line with a `Next:` step. + +## Network, and the producer / consumer split + +Network and Fullstory-captured console data come from the app's Fullstory SDK session, read through the Subtext MCP. It needs an app built with the Fullstory iOS SDK and network capture enabled for the org. That data arrives on the SDK's upload interval, about **20 s** behind the app. So don't make one agent tap and then wait; split the work into two subagents. + +**Producer:** drives the app and never waits on network data. +- `start` (opens a run), then work through flows with `snapshot` / `click` / `fill` / `scroll` / `openurl`. +- Every action is appended to the run log `.sightmap/.sim-runs/.jsonl` with its timestamp, target component and view. The session URL is logged as a `session` entry once the SDK reports it. +- It also fixes UI-side gaps as it goes: elements it can't target cleanly are candidates for new `components:`. +- When done, it reports the run id. + +**Consumer:** turns captured traffic into sightmap. +1. `run tail --follow` (or `--since SEQ`) to follow the producer's steps and pick up the `session` URL. +2. `review-open` with that URL, then `review-zoom` with `resolution={"network": "standard", "console": "standard"}`. Use `"detail"` on specific requests for headers and bodies. +3. Line up each request and log with the producer step just before it (by timestamp), then author or clean up: + - `requests:` (route glob + method) for endpoints that mean something; + - `properties:` for outcomes hidden in bodies; + - `messages:` for meaningful logs and errors; + - corrections to components and views the producer flagged. +4. Re-read about every 20 s, or once after the producer finishes. Each `review-open` costs credits, so don't poll faster than data can arrive. + +The orchestrating agent starts the producer, then the consumer once the run has a `session` entry. Run the producer alone when only UI or components work is needed. + +Local, live alternatives that need no SDK: `console list` (the app's os_log / print output) and `snapshot`. + +## Authoring `.sightmap/` for iOS + +Use the `sightmap-authoring` workflow; only the selectors differ. Elements expose `role` (from the accessibility type), `identifier` (the `accessibilityIdentifier`), `text` (the label) and `value`: + +```yaml +components: + - name: SignInButton + selector: 'button[identifier="login-submit"]' + - name: ProductRow + selector: 'button[identifier^="product-row-"]' + properties: + - name: title + extract: text +``` + +Views have no URL to route on: the snapshot infers the view whose own components match the most elements (`[View: Login] (inferred)`), or pass `--view`. `/fixtures/SampleApp/.sightmap/` is a small worked example. + +Prefer `[identifier=…]`; fall back to `[text=…]` only for copy that won't change or be localized. Snapshot after editing to confirm the components resolve. + +**SwiftUI gotcha:** `.accessibilityIdentifier` on a container overwrites the identifiers of its children. Put identifiers on the leaf controls (buttons, fields, rows), not on the stack around them. If every child of a view shows the same identifier, that's why. Tab bar items take the identifier set on the `Label` inside `.tabItem`. + +## Sample app + +`/fixtures/SampleApp/build.sh` builds a small SwiftUI app (login form, 200-row product list, tab bar, a JSON POST button) and prints the `.app` path. Set `FS_ORG_ID=` to build it with the Fullstory SDK, which enables `session-url` and network. + +## Known limits + +- Simulator only, Apple Silicon only. No physical devices. +- No hover, file upload, or arbitrary screen sizes: pick a different `--device`. +- Network isn't live, and needs the Fullstory SDK in the app plus network capture enabled for the org. +- The helper uses Apple private frameworks; a new Xcode release may need a plugin update. `doctor` says so when the helper fails to build. +- On iOS 26 a SwiftUI tab bar shows up as one `other "Tab Bar"` element without its tabs; switch tabs with `tap-xy` from a screenshot. +- Scrolling a long list to a far row takes about 3 s per screenful. diff --git a/skills/subtext-ios-sim/bin/subtext-sim b/skills/subtext-ios-sim/bin/subtext-sim new file mode 100755 index 0000000..dd483fe --- /dev/null +++ b/skills/subtext-ios-sim/bin/subtext-sim @@ -0,0 +1,4 @@ +#!/usr/bin/env node +import('../lib/cli.mjs').then(async ({ main }) => { + process.exitCode = await main(); +}); diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/.gitignore b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/.gitignore new file mode 100644 index 0000000..a47a5b8 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/.gitignore @@ -0,0 +1,2 @@ +.sim-session +.sim-runs/ diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml new file mode 100644 index 0000000..9bb8fb1 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml @@ -0,0 +1,9 @@ +version: 1 +memory: + - Any email containing "@" and any non-empty password signs in. +components: + - name: TabBarItem + selector: 'tab' + properties: + - name: title + extract: text diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml new file mode 100644 index 0000000..8fa150d --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml @@ -0,0 +1,12 @@ +version: 1 +views: + - name: Login + components: + - name: EmailField + selector: '[identifier="login-email"]' + - name: PasswordField + selector: '[identifier="login-password"]' + - name: SignInButton + selector: 'button[identifier="login-submit"]' + - name: LoginError + selector: '[identifier="login-error"]' diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml new file mode 100644 index 0000000..7892dab --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml @@ -0,0 +1,15 @@ +version: 1 +views: + - name: Products + components: + - name: ProductRow + selector: '[identifier^="product-row-"]' + properties: + - name: title + extract: text + - name: ProductDetail + components: + - name: ProductName + selector: '[identifier="product-detail-name"]' + - name: AddToCartButton + selector: 'button[identifier="product-detail-add"]' diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/Info.plist b/skills/subtext-ios-sim/fixtures/SampleApp/Info.plist new file mode 100644 index 0000000..bc8935c --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/Info.plist @@ -0,0 +1,19 @@ + + + + + CFBundleIdentifierdev.subtext.SimSample + CFBundleNameSimSample + CFBundleDisplayNameSubtext Sim Sample + CFBundleExecutableSimSample + CFBundlePackageTypeAPPL + CFBundleShortVersionString1.0 + CFBundleVersion1 + CFBundleInfoDictionaryVersion6.0 + MinimumOSVersion17.0 + CFBundleSupportedPlatformsiPhoneSimulator + UIDeviceFamily1 + UILaunchScreen + UIApplicationSceneManifestUIApplicationSupportsMultipleScenes + + diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/Sources/FullstoryBridge.swift b/skills/subtext-ios-sim/fixtures/SampleApp/Sources/FullstoryBridge.swift new file mode 100644 index 0000000..873e214 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/Sources/FullstoryBridge.swift @@ -0,0 +1,26 @@ +import Foundation +#if SUBTEXT_FULLSTORY +import FullStory +#endif + +/// Logs the session URL as `FULLSTORY_SESSION_URL ` so `subtext-sim session-url` can find it. +enum FullstoryBridge { + #if SUBTEXT_FULLSTORY + private final class Delegate: NSObject, FSDelegate { + func fullstoryDidStartSession(_ sessionUrl: String) { + appLog.info("FULLSTORY_SESSION_URL \(sessionUrl, privacy: .public)") + print("FULLSTORY_SESSION_URL \(sessionUrl)") + } + } + + private static let delegate = Delegate() + + static func start() { + FS.delegate = delegate + } + #else + static func start() { + appLog.info("built without the Fullstory SDK") + } + #endif +} diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/Sources/SampleApp.swift b/skills/subtext-ios-sim/fixtures/SampleApp/Sources/SampleApp.swift new file mode 100644 index 0000000..6ad3046 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/Sources/SampleApp.swift @@ -0,0 +1,209 @@ +import SwiftUI +import os + +let appLog = Logger(subsystem: "dev.subtext.SimSample", category: "app") + +final class Session: ObservableObject { + @Published var signedInEmail: String? +} + +@main +struct SampleApp: App { + @StateObject private var session = Session() + + init() { + FullstoryBridge.start() + appLog.info("launched") + print("Subtext Sim Sample launched") + } + + var body: some Scene { + WindowGroup { + if let email = session.signedInEmail { + MainTabs(email: email) + .environmentObject(session) + } else { + LoginView() + .environmentObject(session) + } + } + } +} + +struct LoginView: View { + @EnvironmentObject private var session: Session + @State private var email = "" + @State private var password = "" + @State private var error: String? + + var body: some View { + VStack(spacing: 16) { + Text("Subtext Sim Sample") + .font(.largeTitle.bold()) + .accessibilityIdentifier("login-title") + TextField("Email", text: $email) + .textContentType(.emailAddress) + .keyboardType(.emailAddress) + .textInputAutocapitalization(.never) + .autocorrectionDisabled() + .textFieldStyle(.roundedBorder) + .accessibilityIdentifier("login-email") + SecureField("Password", text: $password) + .textFieldStyle(.roundedBorder) + .accessibilityIdentifier("login-password") + if let error { + Text(error) + .foregroundStyle(.red) + .accessibilityIdentifier("login-error") + } + Button("Sign In", action: signIn) + .buttonStyle(.borderedProminent) + .accessibilityIdentifier("login-submit") + } + .padding(24) + } + + private func signIn() { + guard email.contains("@"), !password.isEmpty else { + error = "Enter an email and password" + appLog.error("sign-in rejected: missing email or password") + print("sign-in rejected") + return + } + appLog.info("signed in as \(email, privacy: .public)") + print("signed in as \(email)") + session.signedInEmail = email + } +} + +struct MainTabs: View { + let email: String + + var body: some View { + TabView { + ProductList() + .tabItem { + Label("Products", systemImage: "bag") + .accessibilityIdentifier("tab-products") + } + NetworkView() + .tabItem { + Label("Network", systemImage: "network") + .accessibilityIdentifier("tab-network") + } + AccountView(email: email) + .tabItem { + Label("Account", systemImage: "person.crop.circle") + .accessibilityIdentifier("tab-account") + } + } + } +} + +struct Product: Identifiable, Hashable { + let id: Int + var name: String { "Product \(id)" } + var price: String { String(format: "$%d.%02d", 5 + id % 95, (id * 37) % 100) } + + static let all = (1...200).map(Product.init) +} + +struct ProductList: View { + var body: some View { + NavigationStack { + List(Product.all) { product in + NavigationLink(value: product) { + HStack { + Text(product.name) + Spacer() + Text(product.price).foregroundStyle(.secondary) + } + } + .accessibilityIdentifier("product-row-\(product.id)") + } + .navigationTitle("Products") + .navigationDestination(for: Product.self) { ProductDetail(product: $0) } + } + } +} + +struct ProductDetail: View { + let product: Product + @State private var inCart = false + + var body: some View { + VStack(spacing: 16) { + Text(product.name) + .font(.title.bold()) + .accessibilityIdentifier("product-detail-name") + Text(product.price) + .font(.title2) + .accessibilityIdentifier("product-detail-price") + Button(inCart ? "Added" : "Add to Cart") { + inCart = true + appLog.info("added \(product.name, privacy: .public) to cart") + print("added \(product.name) to cart") + } + .buttonStyle(.borderedProminent) + .disabled(inCart) + .accessibilityIdentifier("product-detail-add") + } + .padding() + .navigationTitle(product.name) + .navigationBarTitleDisplayMode(.inline) + } +} + +struct NetworkView: View { + @State private var status = "Idle" + @State private var requestCount = 0 + + var body: some View { + VStack(spacing: 16) { + Button("Send Request", action: send) + .buttonStyle(.borderedProminent) + .accessibilityIdentifier("network-send") + Text(status) + .accessibilityIdentifier("network-status") + } + .padding() + } + + private func send() { + requestCount += 1 + let path = "subtext-sim/order-\(requestCount)" + var request = URLRequest(url: URL(string: "https://httpbin.org/anything/\(path)")!) + request.httpMethod = "POST" + request.setValue("application/json", forHTTPHeaderField: "Content-Type") + request.httpBody = try? JSONSerialization.data(withJSONObject: ["order": requestCount, "items": ["Product 1", "Product 2"]]) + status = "Sending…" + appLog.info("POST \(path, privacy: .public)") + print("POST /anything/\(path)") + URLSession.shared.dataTask(with: request) { _, response, error in + let code = (response as? HTTPURLResponse)?.statusCode ?? -1 + let result = error.map { "failed: \($0.localizedDescription)" } ?? "HTTP \(code)" + appLog.info("POST \(path, privacy: .public) -> \(result, privacy: .public)") + print("POST /anything/\(path) -> \(result)") + DispatchQueue.main.async { status = result } + }.resume() + } +} + +struct AccountView: View { + @EnvironmentObject private var session: Session + let email: String + + var body: some View { + VStack(spacing: 16) { + Text(email) + .accessibilityIdentifier("account-email") + Button("Sign Out") { + appLog.info("signed out") + print("signed out") + session.signedInEmail = nil + } + .accessibilityIdentifier("account-sign-out") + } + .padding() + } +} diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/build.sh b/skills/subtext-ios-sim/fixtures/SampleApp/build.sh new file mode 100755 index 0000000..25f08a7 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/build.sh @@ -0,0 +1,70 @@ +#!/bin/bash +# Builds the Subtext Sim Sample app for the iOS Simulator with only Xcode installed +# (swiftc plus a hand-assembled bundle, no .xcodeproj) and prints the .app path. +# +# ./build.sh # plain build +# FS_ORG_ID= ./build.sh # embed the public Fullstory iOS SDK and record on start +# +# Env: OUT_DIR (default ~/Library/Caches/subtext-sim/SampleApp), +# FS_SDK_VERSION (tag of fullstorydev/fullstory-swift-package-ios, default 1.76.0). +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +out_dir="${OUT_DIR:-$HOME/Library/Caches/subtext-sim/SampleApp}" +fs_version="${FS_SDK_VERSION:-1.76.0}" +app="$out_dir/SimSample.app" +log() { echo "build.sh: $*" >&2; } + +arch="$(uname -m)" +target="${arch}-apple-ios17.0-simulator" +sdk="$(xcrun --sdk iphonesimulator --show-sdk-path)" + +rm -rf "$app" +mkdir -p "$app" +cp "$here/Info.plist" "$app/Info.plist" + +swift_flags=() +if [[ -n "${FS_ORG_ID:-}" ]]; then + fs_cache="$HOME/Library/Caches/subtext-sim/fullstory-$fs_version" + framework="$fs_cache/FullStory.xcframework/ios-arm64_x86_64-simulator/FullStory.framework" + if [[ ! -d "$framework" ]]; then + log "fetching Fullstory iOS SDK $fs_version" + manifest="$(curl -fsSL "https://raw.githubusercontent.com/fullstorydev/fullstory-swift-package-ios/$fs_version/Package.swift")" + zip_url="$(sed -nE 's/.*url: *"([^"]+)".*/\1/p' <<<"$manifest")" + checksum="$(sed -nE 's/.*checksum: *"([^"]+)".*/\1/p' <<<"$manifest")" + mkdir -p "$fs_cache" + curl -fsSL -o "$fs_cache/sdk.zip" "$zip_url" + actual="$(shasum -a 256 "$fs_cache/sdk.zip" | cut -d' ' -f1)" + if [[ "$actual" != "$checksum" ]]; then + log "checksum mismatch for $zip_url (expected $checksum, got $actual)" + exit 1 + fi + unzip -q -o "$fs_cache/sdk.zip" -d "$fs_cache" + rm "$fs_cache/sdk.zip" + fi + swift_flags+=(-D SUBTEXT_FULLSTORY -F "$(dirname "$framework")" -framework FullStory + -Xlinker -rpath -Xlinker @executable_path/Frameworks) + mkdir -p "$app/Frameworks" + cp -R "$framework" "$app/Frameworks/" + plist="$app/Info.plist" + /usr/libexec/PlistBuddy \ + -c "Add :Fullstory dict" \ + -c "Add :Fullstory:OrgId string $FS_ORG_ID" \ + -c "Add :Fullstory:RecordOnStart bool true" \ + "$plist" >/dev/null + log "including Fullstory SDK $fs_version" +fi + +log "compiling for $target" +xcrun --sdk iphonesimulator swiftc \ + -target "$target" -sdk "$sdk" -O -parse-as-library \ + "${swift_flags[@]+"${swift_flags[@]}"}" \ + -o "$app/SimSample" \ + "$here"/Sources/*.swift + +if [[ -d "$app/Frameworks" ]]; then + codesign --force --sign - "$app"/Frameworks/*.framework >/dev/null 2>&1 +fi +codesign --force --sign - "$app" >/dev/null 2>&1 + +echo "$app" diff --git a/skills/subtext-ios-sim/helper/.gitignore b/skills/subtext-ios-sim/helper/.gitignore new file mode 100644 index 0000000..2d9f16e --- /dev/null +++ b/skills/subtext-ios-sim/helper/.gitignore @@ -0,0 +1,2 @@ +.build/ +.swiftpm/ diff --git a/skills/subtext-ios-sim/helper/Package.swift b/skills/subtext-ios-sim/helper/Package.swift new file mode 100644 index 0000000..0b52d97 --- /dev/null +++ b/skills/subtext-ios-sim/helper/Package.swift @@ -0,0 +1,45 @@ +// swift-tools-version:6.0 +import PackageDescription + +// FBSimulatorControl and friends come from idb's released, prebuilt xcframeworks. +// URLs and checksums are copied verbatim from the Package.swift attached to the idb v1.7.7 release. +let idbRelease = "https://github.com/facebook/idb/releases/download/v1.7.7" + +let idbFrameworks: [(name: String, checksum: String)] = [ + ("FBControlCore", "887a26ab93c9f54cddd6c32f21b8cc5586f1396644759706036c02653af47903"), + ("FBControlCoreObjC", "8f2f37fd6a74b98f516aabf4c27881a81e44831a0fa7f01895cced10de2469d9"), + ("FBSimulatorAX", "1331b557dc775847a4e46522b1bfb0b393181a28646621a3271169c06f2829eb"), + ("FBSimulatorBridgeProtocol", "150761f6905edc5aa56783931f0006f9d43c844b94d0a424db2cd5b83de22fe7"), + ("FBSimulatorControl", "b2c88f22ca55b3da252d8a7e33a98be911347a2d78bd429038c38ecbf48f3077"), + ("FBSimulatorServices", "e97f47bd119f639c4bd3c86fe38ac1989c0bd47f00e035d9a5f07d8996117608"), + ("IDBPrivateHeaders", "eb78edff346bc38b7392a5243235c6d42827b7036371dc8a5e166fa6ee3b9461"), + ("SimulatorIPC", "4419875068465239711bcf296578a7259f6fa9db00c5dfadb5e31a57a99b2ab7"), +] + +let package = Package( + name: "SubtextSimHelper", + platforms: [.macOS(.v15)], + products: [ + .executable(name: "subtext-sim-helper", targets: ["SubtextSimHelper"]) + ], + targets: idbFrameworks.map { + .binaryTarget(name: $0.name, url: "\(idbRelease)/\($0.name).xcframework.zip", checksum: $0.checksum) + } + [ + .executableTarget( + name: "SubtextSimHelper", + dependencies: idbFrameworks.map { .target(name: $0.name) }, + swiftSettings: [.swiftLanguageMode(.v5)], + linkerSettings: [ + // idb's CoreSimulatorStub / AccessibilityPlatformTranslationStub frameworks only re-export these, + // and would have to ship beside the binary. Weak-linking the real private frameworks at their + // install paths keeps the helper a single self-contained file. + .unsafeFlags([ + "-Xlinker", "-F/Library/Developer/PrivateFrameworks", + "-Xlinker", "-weak_framework", "-Xlinker", "CoreSimulator", + "-Xlinker", "-F/System/Library/PrivateFrameworks", + "-Xlinker", "-weak_framework", "-Xlinker", "AccessibilityPlatformTranslation", + ]) + ] + ) + ] +) diff --git a/skills/subtext-ios-sim/helper/README.md b/skills/subtext-ios-sim/helper/README.md new file mode 100644 index 0000000..76b0d88 --- /dev/null +++ b/skills/subtext-ios-sim/helper/README.md @@ -0,0 +1,50 @@ +# subtext-sim-helper + +A long-lived Swift process that reads the iOS Simulator's accessibility tree and sends touch, keyboard and +button input. It is built on Meta's FBSimulatorControl, consumed as idb v1.7.7's prebuilt xcframeworks. +URLs and checksums are pinned in `Package.swift`. + +## Requirements + +- An Apple Silicon Mac on macOS 15 or later (idb's prebuilt frameworks are arm64 only). +- Full Xcode, not just the Command Line Tools, with an iOS Simulator runtime. Open Xcode once so that it + installs CoreSimulator. +- Network access for the first build, to download the frameworks from GitHub. + +No Homebrew, idb install or other dependency is needed. + +## Build + +```sh +helper/build.sh # prints the binary path on its last stdout line +``` + +The binary goes to `${SUBTEXT_SIM_CACHE:-~/Library/Caches/subtext-sim}//subtext-sim-helper`. +When nothing in `Package.swift`, `Sources/` or `build.sh` has changed, the script skips the build and prints +the cached path. A warm build takes about 15 s. On a cold machine, add the time to download about 10 MB of +frameworks. + +## Protocol + +JSON lines over stdin/stdout, one request per line. The supported commands are `hello`, `describe`, `tap`, +`type`, `swipe`, `button` and `key`. See "Helper protocol" in `docs/ios-sim-loop/PLAN.md`. Diagnostics go +to stderr only. Set `SUBTEXT_SIM_HELPER_DEBUG=1` for verbose framework logging. + +Additions to the plan's contract: + +- Each `describe` node also carries `subrole` (for example `AXSwitch`) and `traits` (for example + `["Button", "Toggle"]`). +- `value` is always a string or `null`. A switch reports `"1"` or `"0"`. +- Error codes: + - `bad_request` and `unknown_cmd` for malformed requests. + - `device_not_found` and `not_booted` for simulator state. + - `ax_disabled` when app accessibility is off. The message includes the `defaults write` fix. + - `ax_unavailable` and `ax_error` for accessibility failures. + - `hid_error` for input failures. + - `unsupported_text` when `type` receives characters outside printable ASCII. + - `internal` for anything else. +- `button` takes `home`, `lock`, `siri`, `side`, `apple_pay`, `volume_up` or `volume_down`. +- `key` takes a name (`return`, `delete`, `tab`, `escape`, `space`, the arrow keys, `home`, `end`, + `page_up`, `page_down`, `forward_delete`), a single character, or a numeric HID usage. + +`testdata/describe-bench*.json` holds recorded `describe` results that can be used as fixtures. diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift new file mode 100644 index 0000000..b15aacc --- /dev/null +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift @@ -0,0 +1,89 @@ +import FBSimulatorAX +import FBSimulatorControl +import Foundation + +enum Describe { + static let keys: Set = [.type, .role, .subrole, .traits, .label, .uniqueID, .value, .frameDict, .enabled] + + static let accessibilityDisabledMessage = + "the app exposes no accessibility elements; enable app accessibility with " + + "`xcrun simctl spawn %@ defaults write com.apple.Accessibility ApplicationAccessibilityEnabled -bool true` " + + "and relaunch the app" + + static func run(_ connection: SimulatorConnection, udid: String) async throws -> [String: Any] { + let options = AccessibilityRequestOptions(format: .nested, keys: keys) + let response = try await connection.automation.describe(.frontmost, options: options) + let elements = response.elements.elements + let root: [String: Any] + if elements.count == 1 { + root = node(elements[0]) + } else { + root = syntheticRoot(children: elements.map(node)) + } + if (root["children"] as? [Any])?.isEmpty ?? true { + throw HelperError(code: "ax_disabled", message: String(format: accessibilityDisabledMessage, udid)) + } + return ["screen": screen(connection.simulator, root: root), "root": root] + } + + private static func node(_ element: AccessibilityDocumentElement) -> [String: Any] { + [ + "role": json(element.role), + "subrole": json(element.subrole), + "type": json(element.type), + "label": json(element.label), + "identifier": json(element.identifier), + "value": json(element.value.flatMap { $0 }.flatMap(string)), + "frame": frame(element.frame.flatMap { $0 }?.rect), + "enabled": element.enabled.flatMap { $0 } ?? true, + "traits": element.traits.flatMap { $0 } ?? [], + "children": (element.children ?? []).map(node), + ] + } + + private static func syntheticRoot(children: [[String: Any]]) -> [String: Any] { + [ + "role": "AXGroup", "subrole": NSNull(), "type": "Group", "traits": [String](), "label": NSNull(), "identifier": NSNull(), "value": NSNull(), + "frame": frame(nil), "enabled": true, "children": children, + ] + } + + private static func screen(_ simulator: Simulator, root: [String: Any]) -> [String: Any] { + if let info = simulator.screenInfo, info.scale > 0 { + return [ + "w": Double(info.widthPixels) / Double(info.scale), + "h": Double(info.heightPixels) / Double(info.scale), + ] + } + let rootFrame = root["frame"] as? [String: Any] + return ["w": rootFrame?["w"] ?? 0, "h": rootFrame?["h"] ?? 0] + } + + private static func frame(_ rect: CGRect?) -> [String: Any] { + let rect = rect ?? .zero + return ["x": rect.origin.x, "y": rect.origin.y, "w": rect.size.width, "h": rect.size.height] + } + + private static func json(_ value: String??) -> Any { + value.flatMap { $0 }.map { $0 as Any } ?? NSNull() + } + + private static func json(_ value: String?) -> Any { + value.map { $0 as Any } ?? NSNull() + } + + private static func string(_ value: AccessibilityAttributeValue) -> String? { + switch value { + case .string(let string): return string + case .bool(let bool): return bool ? "1" : "0" + case .int(let int): return String(int) + case .double(let double): return String(double) + case .null: return nil + case .array, .object: + guard let data = try? JSONEncoder().encode(value) else { return nil } + return String(decoding: data, as: UTF8.self) + @unknown default: + return nil + } + } +} diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift new file mode 100644 index 0000000..552d1d1 --- /dev/null +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift @@ -0,0 +1,75 @@ +import FBSimulatorControl + +/// USB HID usage IDs (keyboard page 0x07) for a US keyboard layout. +enum Keyboard { + static let leftShift: UInt32 = 225 + + static let namedKeys: [String: UInt32] = [ + "return": 40, "enter": 40, + "escape": 41, "esc": 41, + "delete": 42, "backspace": 42, + "tab": 43, + "space": 44, + "forward_delete": 76, "forwarddelete": 76, + "right": 79, "left": 80, "down": 81, "up": 82, + "home": 74, "end": 77, "page_up": 75, "pageup": 75, "page_down": 78, "pagedown": 78, + ] + + private static let unshifted: [Character: UInt32] = { + var map: [Character: UInt32] = [:] + for (offset, scalar) in "abcdefghijklmnopqrstuvwxyz".enumerated() { map[scalar] = 4 + UInt32(offset) } + for (offset, scalar) in "1234567890".enumerated() { map[scalar] = 30 + UInt32(offset) } + let punctuation: [Character: UInt32] = [ + "\n": 40, "\t": 43, " ": 44, "-": 45, "=": 46, "[": 47, "]": 48, "\\": 49, + ";": 51, "'": 52, "`": 53, ",": 54, ".": 55, "/": 56, + ] + map.merge(punctuation) { $1 } + return map + }() + + private static let shifted: [Character: UInt32] = { + var map: [Character: UInt32] = [:] + for (offset, scalar) in "ABCDEFGHIJKLMNOPQRSTUVWXYZ".enumerated() { map[scalar] = 4 + UInt32(offset) } + for (offset, scalar) in "!@#$%^&*()".enumerated() { map[scalar] = 30 + UInt32(offset) } + let punctuation: [Character: UInt32] = [ + "_": 45, "+": 46, "{": 47, "}": 48, "|": 49, ":": 51, "\"": 52, "~": 53, "<": 54, ">": 55, "?": 56, + ] + map.merge(punctuation) { $1 } + return map + }() + + static func events(typing text: String) throws -> [SimulatorHIDEvent] { + var events: [SimulatorHIDEvent] = [] + for character in text { + if let code = unshifted[character] { + events.append(.keyboard(direction: .down, keyCode: code)) + events.append(.keyboard(direction: .up, keyCode: code)) + } else if let code = shifted[character] { + events.append(.keyboard(direction: .down, keyCode: leftShift)) + events.append(.keyboard(direction: .down, keyCode: code)) + events.append(.keyboard(direction: .up, keyCode: code)) + events.append(.keyboard(direction: .up, keyCode: leftShift)) + } else { + throw HelperError( + code: "unsupported_text", + message: "character \(String(reflecting: String(character))) has no US-keyboard key; only printable ASCII, newline and tab can be typed") + } + } + return events + } + + static func keyCode(named name: String) throws -> UInt32 { + let normalized = name.lowercased() + if let code = namedKeys[normalized] { + return code + } + if normalized.count == 1, let code = unshifted[Character(normalized)] { + return code + } + if let numeric = UInt32(normalized) { + return numeric + } + throw HelperError.badRequest( + "unknown key '\(name)'; use one of \(namedKeys.keys.sorted().joined(separator: ", ")), a single character, or a numeric HID usage") + } +} diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Protocol.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Protocol.swift new file mode 100644 index 0000000..d71400c --- /dev/null +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Protocol.swift @@ -0,0 +1,83 @@ +import Foundation + +let helperVersion = "0.1.0" +let fbSimulatorControlVersion = "idb v1.7.7" + +struct HelperError: Error { + let code: String + let message: String + + static func badRequest(_ message: String) -> HelperError { HelperError(code: "bad_request", message: message) } +} + +struct Request { + let id: Any + let cmd: String + let args: [String: Any] + + init(line: String) throws { + guard let data = line.data(using: .utf8), + let object = try? JSONSerialization.jsonObject(with: data) as? [String: Any] + else { + throw HelperError.badRequest("request is not a JSON object") + } + id = object["id"] ?? NSNull() + guard let cmd = object["cmd"] as? String else { + throw HelperError.badRequest("missing string field 'cmd'") + } + self.cmd = cmd + args = object["args"] as? [String: Any] ?? [:] + } + + func string(_ key: String) throws -> String { + guard let value = args[key] as? String, !value.isEmpty else { + throw HelperError.badRequest("missing string arg '\(key)'") + } + return value + } + + func number(_ key: String) throws -> Double { + guard let value = optionalNumber(key) else { + throw HelperError.badRequest("missing numeric arg '\(key)'") + } + return value + } + + func optionalNumber(_ key: String) -> Double? { + (args[key] as? NSNumber)?.doubleValue + } +} + +/// Protocol output goes to a private duplicate of the original stdout; fd 1 itself is pointed at stderr +/// so that nothing the linked frameworks print can corrupt the JSON-lines stream. +final class ResponseWriter { + private let handle: FileHandle + + init() { + let protocolFD = dup(STDOUT_FILENO) + dup2(STDERR_FILENO, STDOUT_FILENO) + handle = FileHandle(fileDescriptor: protocolFD, closeOnDealloc: true) + } + + func success(id: Any, result: [String: Any]) { + write(["id": id, "ok": true, "result": result]) + } + + func failure(id: Any, error: HelperError) { + write(["id": id, "ok": false, "error": ["code": error.code, "message": error.message]]) + } + + private func write(_ object: [String: Any]) { + do { + var data = try JSONSerialization.data(withJSONObject: object, options: [.withoutEscapingSlashes]) + data.append(0x0A) + handle.write(data) + } catch { + log("failed to encode response: \(error)") + } + } +} + +func log(_ message: String) { + FileHandle.standardError.write(Data("[subtext-sim-helper] \(message)\n".utf8)) +} diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/SimulatorRegistry.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/SimulatorRegistry.swift new file mode 100644 index 0000000..fcc95ba --- /dev/null +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/SimulatorRegistry.swift @@ -0,0 +1,96 @@ +import FBControlCore +import FBSimulatorAX +import FBSimulatorControl +import Foundation + +/// One long-lived connection per simulator: the FBSimulatorControl handle, its UI-automation reader +/// and its HID channel are built on first use and reused until a call through them fails. +final class SimulatorConnection { + let simulator: Simulator + let automation: any UIAutomation + private var hid: SimulatorHID? + + init(simulator: Simulator) throws { + self.simulator = simulator + automation = try simulator.uiAutomation(backend: .accessibility) + } + + func hidChannel() async throws -> SimulatorHID { + if let hid { + return hid + } + let connected = try await simulator.hid.connect() + hid = connected + return connected + } + + func send(_ event: SimulatorHIDEvent, logger: any ControlCoreLogger) async throws { + try await hidChannel().send(event: event, logger: logger) + } +} + +final class SimulatorRegistry { + let logger: any ControlCoreLogger + private var bootstrap: SimulatorControlBootstrap? + private var connections: [String: SimulatorConnection] = [:] + + init(logger: any ControlCoreLogger) { + self.logger = logger + } + + func connection(udid: String) throws -> SimulatorConnection { + if let cached = connections[udid] { + try requireBooted(cached.simulator) + return cached + } + guard let simulator = try simulatorSet().simulator(withUDID: udid) else { + throw HelperError(code: "device_not_found", message: "no simulator with UDID \(udid) in the default device set") + } + try requireBooted(simulator) + let connection = try SimulatorConnection(simulator: simulator) + connections[udid] = connection + log("connected to \(simulator.name) (\(udid))") + return connection + } + + func invalidate(udid: String) { + if connections.removeValue(forKey: udid) != nil { + log("dropped cached connection for \(udid)") + } + } + + /// Runs `body` against the cached connection. A failure drops the connection so the next call + /// rebuilds it, which is what a simulator reboot under the long-lived helper needs. Only idempotent + /// work (reads) is retried immediately; an input event may have partially landed. + func withConnection(udid: String, retry: Bool, _ body: (SimulatorConnection) async throws -> T) async throws -> T { + let first = try connection(udid: udid) + do { + return try await body(first) + } catch let error as HelperError { + throw error + } catch { + invalidate(udid: udid) + guard retry else { throw error } + log("\(udid): \(error.localizedDescription); reconnecting and retrying once") + return try await body(try connection(udid: udid)) + } + } + + private func simulatorSet() throws -> SimulatorSet { + if let bootstrap { + return bootstrap.set + } + let configuration = SimulatorControlConfiguration(deviceSetPath: nil, logger: logger) + let created = try SimulatorControlBootstrap(configuration: configuration) + bootstrap = created + return created.set + } + + private func requireBooted(_ simulator: Simulator) throws { + guard simulator.state == .booted else { + throw HelperError( + code: "not_booted", + message: "simulator \(simulator.name) (\(simulator.udid)) is \(simulator.stateString.rawValue), not booted; run `xcrun simctl boot \(simulator.udid)`") + } + } +} diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/main.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/main.swift new file mode 100644 index 0000000..dd65d02 --- /dev/null +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/main.swift @@ -0,0 +1,134 @@ +import FBControlCore +import FBSimulatorAX +import FBSimulatorControl +import Foundation + +let writer = ResponseWriter() +let logger = ControlCoreGlobalConfiguration.systemLogger( + writingToStderr: true, debugLogging: ProcessInfo.processInfo.environment["SUBTEXT_SIM_HELPER_DEBUG"] != nil) +ControlCoreGlobalConfiguration.defaultLogger = logger +let registry = SimulatorRegistry(logger: logger) + +let buttons: [String: SimulatorHIDButton] = [ + "home": .homeButton, + "lock": .lock, + "siri": .siri, + "side": .sideButton, + "apple_pay": .applePay, + "volume_up": .volumeUp, + "volume_down": .volumeDown, +] + +func handle(_ request: Request) async throws -> [String: Any] { + switch request.cmd { + case "hello": + return ["version": helperVersion, "xcode": xcodeVersion() as Any? ?? NSNull(), "fbsimctl": fbSimulatorControlVersion] + + case "describe": + let udid = try request.string("udid") + return try await registry.withConnection(udid: udid, retry: true) { try await Describe.run($0, udid: udid) } + + case "tap": + let udid = try request.string("udid") + let event = SimulatorHIDEvent.tapAt(x: try request.number("x"), y: try request.number("y")) + try await send(event, udid: udid) + + case "type": + let udid = try request.string("udid") + guard let text = request.args["text"] as? String else { + throw HelperError.badRequest("missing string arg 'text'") + } + try await send(.composite(try Keyboard.events(typing: text)), udid: udid) + + case "swipe": + let udid = try request.string("udid") + let event = SimulatorHIDEvent.swipe( + try request.number("x1"), yStart: try request.number("y1"), + xEnd: try request.number("x2"), yEnd: try request.number("y2"), + delta: SimulatorHIDEvent.defaultSwipeDelta, + duration: request.optionalNumber("duration") ?? 0.3) + try await send(event, udid: udid) + + case "button": + let udid = try request.string("udid") + let name = try request.string("name") + guard let button = buttons[name.lowercased()] else { + throw HelperError.badRequest("unknown button '\(name)'; use one of \(buttons.keys.sorted().joined(separator: ", "))") + } + try await send(.shortButtonPress(button), udid: udid) + + case "key": + let udid = try request.string("udid") + try await send(.shortKeyPress(try Keyboard.keyCode(named: try request.string("key"))), udid: udid) + + default: + throw HelperError(code: "unknown_cmd", message: "unknown cmd '\(request.cmd)'") + } + return [:] +} + +func send(_ event: SimulatorHIDEvent, udid: String) async throws { + try await registry.withConnection(udid: udid, retry: false) { try await $0.send(event, logger: logger) } +} + +func xcodeVersion() -> String? { + let developerDir = + ProcessInfo.processInfo.environment["DEVELOPER_DIR"] + ?? (try? FileManager.default.destinationOfSymbolicLink(atPath: "/var/db/xcode_select_link")) + ?? "/Applications/Xcode.app/Contents/Developer" + let plist = URL(fileURLWithPath: developerDir).deletingLastPathComponent().appendingPathComponent("Info.plist") + guard let info = NSDictionary(contentsOf: plist) else { return nil } + return info["CFBundleShortVersionString"] as? String +} + +func classify(_ error: Error) -> HelperError { + if let error = error as? HelperError { + return error + } + let message = (error as? LocalizedError)?.errorDescription ?? String(describing: error) + switch error { + case AccessibilityError.simulatorNotBooted: + return HelperError(code: "not_booted", message: message) + case AccessibilityError.accessibilityUnavailable, AccessibilityError.springBoardNotRunning, + AccessibilityError.springBoardRestarting, AccessibilityError.serviceNotAnswering: + return HelperError(code: "ax_unavailable", message: message) + case is AccessibilityError, is UIAutomationError: + return HelperError(code: "ax_error", message: message) + case is SimulatorHIDError: + return HelperError(code: "hid_error", message: message) + default: + return HelperError(code: "internal", message: message) + } +} + +let lines = AsyncStream { continuation in + Thread.detachNewThread { + while let line = readLine(strippingNewline: true) { + continuation.yield(line) + } + continuation.finish() + } +} + +Task { + log("ready (v\(helperVersion), \(fbSimulatorControlVersion))") + for await line in lines where !line.trimmingCharacters(in: .whitespaces).isEmpty { + let started = Date() + var id: Any = NSNull() + var command = "?" + do { + let request = try Request(line: line) + id = request.id + command = request.cmd + writer.success(id: id, result: try await handle(request)) + } catch { + let failure = classify(error) + log("\(command) failed [\(failure.code)]: \(failure.message)") + writer.failure(id: id, error: failure) + } + log(String(format: "%@ %.0fms", command, Date().timeIntervalSince(started) * 1000)) + } + exit(0) +} + +dispatchMain() diff --git a/skills/subtext-ios-sim/helper/build.sh b/skills/subtext-ios-sim/helper/build.sh new file mode 100755 index 0000000..5c356dc --- /dev/null +++ b/skills/subtext-ios-sim/helper/build.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env bash +# Builds subtext-sim-helper (release) into ${SUBTEXT_SIM_CACHE:-~/Library/Caches/subtext-sim}// +# and prints the binary's path as the last line of stdout. Everything else goes to stderr. +# Skips the build when the cached binary was built from identical sources. +set -euo pipefail + +HELPER_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +CACHE_ROOT="${SUBTEXT_SIM_CACHE:-$HOME/Library/Caches/subtext-sim}" + +fail() { + echo "build.sh: $*" >&2 + exit 1 +} + +[[ "$(uname -s)" == "Darwin" ]] || fail "macOS is required" +[[ "$(uname -m)" == "arm64" ]] || fail "Apple Silicon is required (idb's prebuilt frameworks are arm64 only)" + +DEVELOPER_DIR_PATH="$(xcode-select -p 2>/dev/null || true)" +if [[ "$DEVELOPER_DIR_PATH" != *.app/Contents/Developer ]]; then + fail "full Xcode is required, not just the Command Line Tools (xcode-select -p: '${DEVELOPER_DIR_PATH:-unset}'). Install Xcode, then run: sudo xcode-select -s /Applications/Xcode.app" +fi +[[ -d /Library/Developer/PrivateFrameworks/CoreSimulator.framework ]] \ + || fail "CoreSimulator is missing; open Xcode once so it installs its components (or run: xcodebuild -runFirstLaunch)" +xcrun --find swift >/dev/null 2>&1 || fail "the Swift toolchain was not found through xcrun" + +VERSION="$(sed -n 's/^let helperVersion = "\(.*\)"$/\1/p' "$HELPER_DIR/Sources/SubtextSimHelper/Protocol.swift")" +[[ -n "$VERSION" ]] || fail "could not read helperVersion from Sources/SubtextSimHelper/Protocol.swift" + +SOURCE_HASH="$( + cd "$HELPER_DIR" + { echo build.sh; echo Package.swift; find Sources -type f -name '*.swift'; } | LC_ALL=C sort \ + | while read -r file; do printf '%s\n' "$file"; shasum -a 256 "$file"; done \ + | shasum -a 256 | cut -d' ' -f1 +)" + +OUT_DIR="$CACHE_ROOT/$VERSION" +BINARY="$OUT_DIR/subtext-sim-helper" +STAMP="$OUT_DIR/source.sha256" + +if [[ -x "$BINARY" && -f "$STAMP" && "$(cat "$STAMP")" == "$SOURCE_HASH" ]]; then + echo "$BINARY" + exit 0 +fi + +mkdir -p "$CACHE_ROOT" +LOCK="$CACHE_ROOT/.build.lock" +for _ in $(seq 1 600); do + mkdir "$LOCK" 2>/dev/null && break + sleep 1 +done +[[ -d "$LOCK" ]] || fail "timed out waiting for another build to release $LOCK" +trap 'rmdir "$LOCK" 2>/dev/null || true' EXIT + +# Another process may have finished the same build while this one waited for the lock. +if [[ -x "$BINARY" && -f "$STAMP" && "$(cat "$STAMP")" == "$SOURCE_HASH" ]]; then + echo "$BINARY" + exit 0 +fi + +SCRATCH="$CACHE_ROOT/.build" +echo "build.sh: building subtext-sim-helper $VERSION (first build downloads idb v1.7.7 frameworks, ~10 MB)" >&2 +started=$SECONDS +xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" >&2 \ + || fail "swift build failed; see the output above" + +BUILT="$(xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --show-bin-path)/subtext-sim-helper" +[[ -x "$BUILT" ]] || fail "build finished but $BUILT is missing" + +mkdir -p "$OUT_DIR" +cp "$BUILT" "$BINARY.tmp" +mv -f "$BINARY.tmp" "$BINARY" +echo "$SOURCE_HASH" > "$STAMP" +echo "build.sh: built in $((SECONDS - started))s" >&2 +echo "$BINARY" diff --git a/skills/subtext-ios-sim/helper/testdata/describe-bench-eager.json b/skills/subtext-ios-sim/helper/testdata/describe-bench-eager.json new file mode 100644 index 0000000..cf61ee5 --- /dev/null +++ b/skills/subtext-ios-sim/helper/testdata/describe-bench-eager.json @@ -0,0 +1,6065 @@ +{ + "screen": { + "w": 402, + "h": 874 + }, + "root": { + "subrole": null, + "identifier": null, + "value": null, + "frame": { + "x": 0, + "y": 0, + "w": 402, + "h": 874 + }, + "role": "AXApplication", + "label": "Bench", + "enabled": true, + "traits": [ + "None" + ], + "type": "Application", + "children": [ + { + "type": "StaticText", + "subrole": null, + "label": "last:none n:0", + "identifier": "lastTapped", + "frame": { + "y": 62, + "x": 152, + "w": 98.33333333333331, + "h": 20.33333333333333 + }, + "enabled": true, + "children": [], + "traits": [ + "StaticText", + "Scrollable" + ], + "role": "AXStaticText", + "value": null + }, + { + "label": "Eager 0", + "role": "AXButton", + "identifier": "eager.0", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "type": "Button", + "frame": { + "h": 20.33333333333333, + "y": 84.33333333333333, + "w": 58.66666666666666, + "x": 172 + }, + "enabled": true, + "subrole": null, + "value": null + }, + { + "enabled": true, + "value": null, + "role": "AXButton", + "identifier": "eager.1", + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "subrole": null, + "label": "Eager 1", + "frame": { + "w": 55.99999999999997, + "y": 106.66666666666666, + "x": 173.33333333333334, + "h": 20.333333333333343 + } + }, + { + "label": "Eager 2", + "frame": { + "x": 172, + "w": 58.333333333333314, + "y": 129, + "h": 20.333333333333314 + }, + "children": [], + "value": null, + "type": "Button", + "role": "AXButton", + "identifier": "eager.2", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null + }, + { + "children": [], + "subrole": null, + "value": null, + "type": "Button", + "identifier": "eager.3", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 3", + "frame": { + "h": 20.333333333333343, + "y": 151.33333333333331, + "w": 58.66666666666666, + "x": 172 + }, + "role": "AXButton" + }, + { + "frame": { + "x": 171.66666666666666, + "w": 59, + "y": 173.66666666666666, + "h": 20.333333333333343 + }, + "identifier": "eager.4", + "label": "Eager 4", + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "role": "AXButton", + "value": null, + "children": [], + "subrole": null + }, + { + "children": [], + "frame": { + "x": 172, + "w": 58.333333333333314, + "y": 196, + "h": 20.333333333333343 + }, + "label": "Eager 5", + "role": "AXButton", + "identifier": "eager.5", + "enabled": true, + "value": null, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button" + }, + { + "value": null, + "role": "AXButton", + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "label": "Eager 6", + "identifier": "eager.6", + "children": [], + "frame": { + "w": 58.66666666666666, + "y": 218.33333333333331, + "h": 20.333333333333343, + "x": 172 + } + }, + { + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 7", + "enabled": true, + "role": "AXButton", + "value": null, + "frame": { + "x": 172.33333333333334, + "y": 240.66666666666666, + "h": 20.333333333333343, + "w": 57.66666666666666 + }, + "children": [], + "identifier": "eager.7", + "subrole": null + }, + { + "type": "Button", + "value": null, + "identifier": "eager.8", + "enabled": true, + "label": "Eager 8", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "children": [], + "subrole": null, + "frame": { + "w": 59, + "x": 171.66666666666666, + "h": 20.33333333333337, + "y": 263 + } + }, + { + "children": [], + "type": "Button", + "role": "AXButton", + "enabled": true, + "frame": { + "y": 285.3333333333333, + "x": 172, + "w": 58.66666666666666, + "h": 20.333333333333314 + }, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 9", + "subrole": null, + "identifier": "eager.9" + }, + { + "subrole": null, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "y": 307.66666666666663, + "x": 168.33333333333334, + "w": 65.99999999999997, + "h": 20.33333333333337 + }, + "type": "Button", + "children": [], + "label": "Eager 10", + "role": "AXButton", + "identifier": "eager.10", + "enabled": true + }, + { + "subrole": null, + "identifier": "eager.11", + "value": null, + "role": "AXButton", + "children": [], + "frame": { + "y": 330, + "h": 20.333333333333314, + "x": 169.66666666666666, + "w": 63.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "type": "Button", + "label": "Eager 11" + }, + { + "subrole": null, + "frame": { + "y": 352.3333333333333, + "h": 20.333333333333314, + "x": 168.33333333333334, + "w": 65.66666666666666 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "label": "Eager 12", + "role": "AXButton", + "identifier": "eager.12", + "children": [], + "value": null + }, + { + "identifier": "eager.13", + "label": "Eager 13", + "type": "Button", + "value": null, + "frame": { + "y": 374.66666666666663, + "w": 65.99999999999997, + "h": 20.333333333333314, + "x": 168.33333333333334 + }, + "role": "AXButton", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "children": [] + }, + { + "role": "AXButton", + "subrole": null, + "value": null, + "frame": { + "x": 168, + "w": 66.33333333333331, + "y": 397, + "h": 20.333333333333314 + }, + "label": "Eager 14", + "type": "Button", + "enabled": true, + "identifier": "eager.14", + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "type": "Button", + "role": "AXButton", + "subrole": null, + "frame": { + "h": 20.333333333333314, + "y": 419.3333333333333, + "w": 65.99999999999997, + "x": 168.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 15", + "children": [], + "value": null, + "identifier": "eager.15", + "enabled": true + }, + { + "subrole": null, + "identifier": "eager.16", + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "role": "AXButton", + "frame": { + "x": 168, + "y": 441.66666666666663, + "w": 66.33333333333331, + "h": 20.333333333333314 + }, + "enabled": true, + "children": [], + "label": "Eager 16", + "type": "Button" + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "value": null, + "enabled": true, + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 17", + "identifier": "eager.17", + "frame": { + "h": 20.333333333333314, + "w": 65, + "x": 168.66666666666666, + "y": 464 + } + }, + { + "value": null, + "children": [], + "role": "AXButton", + "subrole": null, + "label": "Eager 18", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "w": 66.33333333333331, + "x": 168, + "h": 20.333333333333314, + "y": 486.3333333333333 + }, + "enabled": true, + "type": "Button", + "identifier": "eager.18" + }, + { + "value": null, + "label": "Eager 19", + "subrole": null, + "identifier": "eager.19", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "type": "Button", + "frame": { + "x": 168, + "y": 508.66666666666663, + "w": 66.33333333333331, + "h": 20.33333333333337 + }, + "enabled": true, + "role": "AXButton" + }, + { + "subrole": null, + "role": "AXButton", + "type": "Button", + "enabled": true, + "frame": { + "w": 68.66666666666666, + "x": 167, + "y": 531, + "h": 20.333333333333258 + }, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "label": "Eager 20", + "identifier": "eager.20", + "children": [] + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "value": null, + "enabled": true, + "frame": { + "x": 168.33333333333334, + "w": 65.66666666666666, + "h": 20.33333333333337, + "y": 553.3333333333333 + }, + "role": "AXButton", + "label": "Eager 21", + "identifier": "eager.21", + "children": [], + "type": "Button" + }, + { + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "frame": { + "y": 575.6666666666666, + "w": 68.33333333333331, + "h": 20.33333333333337, + "x": 167 + }, + "label": "Eager 22", + "identifier": "eager.22", + "role": "AXButton", + "value": null + }, + { + "identifier": "eager.23", + "role": "AXButton", + "value": null, + "label": "Eager 23", + "frame": { + "h": 20.33333333333337, + "x": 167, + "y": 598, + "w": 68.33333333333331 + }, + "children": [], + "enabled": true, + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "children": [], + "identifier": "eager.24", + "label": "Eager 24", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "frame": { + "x": 167, + "w": 68.33333333333331, + "h": 20.33333333333337, + "y": 620.3333333333333 + } + }, + { + "label": "Eager 25", + "frame": { + "x": 167, + "y": 642.6666666666666, + "w": 68.33333333333331, + "h": 20.33333333333337 + }, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "subrole": null, + "identifier": "eager.25", + "enabled": true, + "children": [], + "value": null + }, + { + "value": null, + "children": [], + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 26", + "role": "AXButton", + "frame": { + "w": 68.66666666666666, + "y": 665, + "h": 20.33333333333337, + "x": 167 + }, + "identifier": "eager.26", + "enabled": true, + "type": "Button" + }, + { + "identifier": "eager.27", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "type": "Button", + "enabled": true, + "subrole": null, + "children": [], + "value": null, + "frame": { + "x": 167.66666666666666, + "w": 67.33333333333334, + "h": 20.33333333333337, + "y": 687.3333333333333 + }, + "label": "Eager 27" + }, + { + "enabled": true, + "type": "Button", + "role": "AXButton", + "label": "Eager 28", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 167, + "w": 68.66666666666666, + "h": 20.33333333333337, + "y": 709.6666666666666 + }, + "children": [], + "subrole": null, + "identifier": "eager.28" + }, + { + "identifier": "eager.29", + "value": null, + "frame": { + "w": 68.66666666666666, + "x": 167, + "y": 732, + "h": 20.33333333333337 + }, + "role": "AXButton", + "subrole": null, + "label": "Eager 29", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "type": "Button" + }, + { + "subrole": null, + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "type": "Button", + "label": "Eager 30", + "identifier": "eager.30", + "role": "AXButton", + "frame": { + "y": 754.3333333333333, + "x": 166.66666666666666, + "w": 69.00000000000003, + "h": 20.33333333333337 + } + }, + { + "type": "Button", + "subrole": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "frame": { + "x": 168.33333333333334, + "h": 20.33333333333337, + "y": 776.6666666666666, + "w": 65.99999999999997 + }, + "identifier": "eager.31", + "label": "Eager 31", + "children": [], + "role": "AXButton" + }, + { + "enabled": true, + "subrole": null, + "children": [], + "type": "Button", + "identifier": "eager.32", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "label": "Eager 32", + "frame": { + "x": 167, + "y": 799, + "h": 20.33333333333337, + "w": 68.33333333333331 + } + }, + { + "frame": { + "x": 166.66666666666666, + "y": 821.3333333333333, + "h": 20.33333333333337, + "w": 69.00000000000003 + }, + "type": "Button", + "value": null, + "children": [], + "role": "AXButton", + "subrole": null, + "label": "Eager 33", + "identifier": "eager.33", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true + }, + { + "label": "Eager 34", + "children": [], + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.34", + "value": null, + "frame": { + "h": 20.33333333333337, + "y": 843.6666666666666, + "w": 69.00000000000003, + "x": 166.66666666666666 + }, + "subrole": null, + "enabled": true, + "type": "Button" + }, + { + "frame": { + "y": 866, + "x": 167, + "w": 68.66666666666666, + "h": 20.33333333333337 + }, + "enabled": true, + "subrole": null, + "value": null, + "identifier": "eager.35", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "type": "Button", + "label": "Eager 35" + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "subrole": null, + "label": "Eager 36", + "value": null, + "frame": { + "h": 20.33333333333337, + "w": 69.00000000000003, + "y": 888.3333333333333, + "x": 166.66666666666666 + }, + "identifier": "eager.36", + "children": [], + "role": "AXButton", + "enabled": true + }, + { + "value": null, + "children": [], + "label": "Eager 37", + "identifier": "eager.37", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "frame": { + "w": 68.33333333333331, + "h": 20.33333333333337, + "x": 167, + "y": 910.6666666666666 + }, + "enabled": true, + "subrole": null + }, + { + "value": null, + "label": "Eager 38", + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "x": 166.66666666666666, + "y": 933, + "h": 20.33333333333337, + "w": 69.00000000000003 + }, + "subrole": null, + "identifier": "eager.38", + "role": "AXButton", + "enabled": true + }, + { + "children": [], + "label": "Eager 39", + "identifier": "eager.39", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "value": null, + "subrole": null, + "frame": { + "y": 955.3333333333333, + "w": 69.00000000000003, + "h": 20.33333333333337, + "x": 166.66666666666666 + }, + "enabled": true + }, + { + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "type": "Button", + "children": [], + "label": "Eager 40", + "role": "AXButton", + "subrole": null, + "identifier": "eager.40", + "frame": { + "y": 977.6666666666666, + "x": 166.66666666666666, + "h": 20.33333333333337, + "w": 69.33333333333334 + } + }, + { + "frame": { + "y": 1000, + "x": 168, + "w": 66.33333333333331, + "h": 20.33333333333337 + }, + "enabled": true, + "subrole": null, + "children": [], + "label": "Eager 41", + "identifier": "eager.41", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "value": null + }, + { + "value": null, + "frame": { + "y": 1022.3333333333333, + "x": 167, + "h": 20.333333333333258, + "w": 68.66666666666666 + }, + "enabled": true, + "role": "AXButton", + "label": "Eager 42", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.42", + "subrole": null + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "label": "Eager 43", + "role": "AXButton", + "enabled": true, + "identifier": "eager.43", + "type": "Button", + "frame": { + "w": 69.00000000000003, + "y": 1044.6666666666665, + "x": 166.66666666666666, + "h": 20.333333333333485 + }, + "subrole": null, + "children": [] + }, + { + "label": "Eager 44", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "frame": { + "h": 20.333333333333258, + "x": 166.66666666666666, + "y": 1067, + "w": 69.33333333333334 + }, + "subrole": null, + "type": "Button", + "value": null, + "role": "AXButton", + "identifier": "eager.44" + }, + { + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 45", + "enabled": true, + "identifier": "eager.45", + "type": "Button", + "value": null, + "frame": { + "y": 1089.3333333333333, + "w": 69.00000000000003, + "h": 20.333333333333258, + "x": 166.66666666666666 + }, + "children": [], + "subrole": null + }, + { + "label": "Eager 46", + "subrole": null, + "frame": { + "x": 166.66666666666666, + "w": 69.33333333333334, + "y": 1111.6666666666665, + "h": 20.333333333333258 + }, + "enabled": true, + "role": "AXButton", + "value": null, + "identifier": "eager.46", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "type": "Button" + }, + { + "subrole": null, + "enabled": true, + "role": "AXButton", + "value": null, + "frame": { + "h": 20.333333333333258, + "w": 68.33333333333331, + "y": 1134, + "x": 167 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.47", + "label": "Eager 47", + "children": [] + }, + { + "identifier": "eager.48", + "value": null, + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "frame": { + "x": 166.66666666666666, + "w": 69.33333333333334, + "h": 20.333333333333258, + "y": 1156.3333333333333 + }, + "label": "Eager 48", + "subrole": null, + "type": "Button", + "enabled": true + }, + { + "subrole": null, + "label": "Eager 49", + "identifier": "eager.49", + "type": "Button", + "value": null, + "frame": { + "h": 20.333333333333258, + "x": 166.66666666666666, + "w": 69.33333333333334, + "y": 1178.6666666666665 + }, + "enabled": true, + "role": "AXButton", + "children": [], + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "value": null, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "identifier": "eager.50", + "frame": { + "y": 1201, + "h": 20.333333333333258, + "w": 68.66666666666666, + "x": 167 + }, + "enabled": true, + "label": "Eager 50", + "children": [] + }, + { + "value": null, + "identifier": "eager.51", + "enabled": true, + "role": "AXButton", + "label": "Eager 51", + "subrole": null, + "type": "Button", + "frame": { + "h": 20.333333333333258, + "w": 65.99999999999997, + "x": 168.33333333333334, + "y": 1223.3333333333333 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "enabled": true, + "identifier": "eager.52", + "role": "AXButton", + "subrole": null, + "label": "Eager 52", + "type": "Button", + "value": null, + "frame": { + "y": 1245.6666666666665, + "w": 68.33333333333331, + "x": 167, + "h": 20.333333333333258 + }, + "children": [], + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "enabled": true, + "label": "Eager 53", + "value": null, + "frame": { + "y": 1268, + "x": 167, + "w": 68.66666666666666, + "h": 20.333333333333258 + }, + "type": "Button", + "subrole": null, + "identifier": "eager.53", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "children": [], + "subrole": null, + "label": "Eager 54", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "type": "Button", + "frame": { + "y": 1290.3333333333333, + "w": 69.00000000000003, + "x": 166.66666666666666, + "h": 20.333333333333258 + }, + "enabled": true, + "value": null, + "identifier": "eager.54" + }, + { + "value": null, + "role": "AXButton", + "subrole": null, + "frame": { + "w": 68.66666666666666, + "x": 167, + "y": 1312.6666666666665, + "h": 20.333333333333258 + }, + "enabled": true, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.55", + "label": "Eager 55", + "children": [] + }, + { + "role": "AXButton", + "label": "Eager 56", + "identifier": "eager.56", + "frame": { + "x": 166.66666666666666, + "y": 1335, + "h": 20.333333333333258, + "w": 69.00000000000003 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "subrole": null, + "value": null, + "enabled": true, + "type": "Button" + }, + { + "type": "Button", + "value": null, + "enabled": true, + "role": "AXButton", + "frame": { + "w": 67.66666666666666, + "y": 1357.3333333333333, + "h": 20.333333333333258, + "x": 167.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.57", + "label": "Eager 57", + "children": [], + "subrole": null + }, + { + "value": null, + "frame": { + "y": 1379.6666666666665, + "h": 20.333333333333258, + "w": 69.00000000000003, + "x": 166.66666666666666 + }, + "identifier": "eager.58", + "label": "Eager 58", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "subrole": null, + "type": "Button", + "role": "AXButton" + }, + { + "label": "Eager 59", + "type": "Button", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "children": [], + "identifier": "eager.59", + "value": null, + "frame": { + "x": 166.66666666666666, + "w": 69.00000000000003, + "h": 20.333333333333258, + "y": 1402 + }, + "enabled": true + }, + { + "subrole": null, + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "label": "Eager 60", + "frame": { + "x": 166.66666666666666, + "h": 20.333333333333258, + "y": 1424.3333333333333, + "w": 69.00000000000003 + }, + "type": "Button", + "identifier": "eager.60" + }, + { + "type": "Button", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "frame": { + "x": 168, + "h": 20.333333333333258, + "y": 1446.6666666666665, + "w": 66.33333333333331 + }, + "enabled": true, + "identifier": "eager.61", + "role": "AXButton", + "children": [], + "label": "Eager 61" + }, + { + "role": "AXButton", + "subrole": null, + "label": "Eager 62", + "identifier": "eager.62", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "w": 68.66666666666666, + "x": 167, + "h": 20.333333333333258, + "y": 1469 + }, + "type": "Button", + "children": [], + "enabled": true + }, + { + "label": "Eager 63", + "subrole": null, + "children": [], + "role": "AXButton", + "type": "Button", + "identifier": "eager.63", + "value": null, + "frame": { + "h": 20.333333333333258, + "w": 69.00000000000003, + "x": 166.66666666666666, + "y": 1491.3333333333333 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "role": "AXButton", + "value": null, + "subrole": null, + "identifier": "eager.64", + "frame": { + "w": 69.33333333333334, + "y": 1513.6666666666665, + "x": 166.66666666666666, + "h": 20.333333333333258 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "enabled": true, + "label": "Eager 64", + "children": [] + }, + { + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 65", + "enabled": true, + "type": "Button", + "frame": { + "w": 69.00000000000003, + "y": 1536, + "h": 20.333333333333258, + "x": 166.66666666666666 + }, + "role": "AXButton", + "subrole": null, + "children": [], + "identifier": "eager.65" + }, + { + "label": "Eager 66", + "role": "AXButton", + "frame": { + "y": 1558.3333333333333, + "w": 69.33333333333334, + "x": 166.66666666666666, + "h": 20.333333333333258 + }, + "type": "Button", + "identifier": "eager.66", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "value": null, + "subrole": null + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "value": null, + "subrole": null, + "frame": { + "h": 20.333333333333258, + "w": 68.33333333333331, + "y": 1580.6666666666665, + "x": 167 + }, + "identifier": "eager.67", + "children": [], + "role": "AXButton", + "label": "Eager 67", + "enabled": true + }, + { + "frame": { + "x": 166.66666666666666, + "y": 1603, + "w": 69.33333333333334, + "h": 20.333333333333258 + }, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "label": "Eager 68", + "enabled": true, + "identifier": "eager.68", + "subrole": null, + "type": "Button", + "children": [], + "value": null + }, + { + "subrole": null, + "identifier": "eager.69", + "value": null, + "type": "Button", + "frame": { + "x": 166.66666666666666, + "h": 20.333333333333258, + "y": 1625.3333333333333, + "w": 69.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "children": [], + "enabled": true, + "label": "Eager 69" + }, + { + "value": null, + "type": "Button", + "identifier": "eager.70", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "frame": { + "h": 20.333333333333258, + "x": 167.33333333333334, + "y": 1647.6666666666665, + "w": 67.66666666666666 + }, + "enabled": true, + "role": "AXButton", + "label": "Eager 70" + }, + { + "value": null, + "children": [], + "frame": { + "h": 20.333333333333258, + "y": 1670, + "x": 168.66666666666666, + "w": 65 + }, + "role": "AXButton", + "label": "Eager 71", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "subrole": null, + "identifier": "eager.71", + "enabled": true + }, + { + "value": null, + "frame": { + "y": 1692.3333333333333, + "w": 67.33333333333334, + "x": 167.66666666666666, + "h": 20.333333333333258 + }, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.72", + "enabled": true, + "children": [], + "subrole": null, + "label": "Eager 72" + }, + { + "label": "Eager 73", + "subrole": null, + "children": [], + "role": "AXButton", + "value": null, + "type": "Button", + "enabled": true, + "frame": { + "h": 20.333333333333258, + "y": 1714.6666666666665, + "w": 67.66666666666666, + "x": 167.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.73" + }, + { + "subrole": null, + "label": "Eager 74", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "type": "Button", + "value": null, + "identifier": "eager.74", + "enabled": true, + "frame": { + "w": 67.33333333333334, + "h": 20.333333333333258, + "x": 167.66666666666666, + "y": 1737 + } + }, + { + "children": [], + "label": "Eager 75", + "subrole": null, + "role": "AXButton", + "type": "Button", + "value": null, + "identifier": "eager.75", + "frame": { + "w": 67.66666666666666, + "x": 167.33333333333334, + "y": 1759.3333333333333, + "h": 20.333333333333258 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "identifier": "eager.76", + "type": "Button", + "label": "Eager 76", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "value": null, + "role": "AXButton", + "frame": { + "x": 167.33333333333334, + "y": 1781.6666666666665, + "h": 20.333333333333258, + "w": 67.66666666666666 + }, + "subrole": null + }, + { + "identifier": "eager.77", + "subrole": null, + "type": "Button", + "label": "Eager 77", + "frame": { + "x": 167.66666666666666, + "w": 67.33333333333334, + "h": 20.333333333333258, + "y": 1804 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "value": null, + "role": "AXButton" + }, + { + "children": [], + "identifier": "eager.78", + "value": null, + "enabled": true, + "frame": { + "h": 20.333333333333258, + "x": 167, + "y": 1826.3333333333333, + "w": 68.33333333333331 + }, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "label": "Eager 78", + "type": "Button", + "subrole": null + }, + { + "type": "Button", + "children": [], + "role": "AXButton", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.79", + "frame": { + "y": 1848.6666666666665, + "x": 167, + "w": 68.33333333333331, + "h": 20.333333333333258 + }, + "enabled": true, + "label": "Eager 79", + "subrole": null + }, + { + "label": "Eager 80", + "role": "AXButton", + "value": null, + "enabled": true, + "identifier": "eager.80", + "frame": { + "x": 166.66666666666666, + "w": 69.00000000000003, + "y": 1871, + "h": 20.333333333333258 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "subrole": null, + "children": [] + }, + { + "label": "Eager 81", + "identifier": "eager.81", + "children": [], + "frame": { + "w": 66.33333333333331, + "h": 20.333333333333258, + "y": 1893.3333333333333, + "x": 168 + }, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "enabled": true, + "value": null, + "role": "AXButton" + }, + { + "subrole": null, + "enabled": true, + "label": "Eager 82", + "identifier": "eager.82", + "value": null, + "frame": { + "h": 20.333333333333258, + "x": 167, + "y": 1915.6666666666665, + "w": 68.66666666666666 + }, + "role": "AXButton", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button" + }, + { + "children": [], + "frame": { + "h": 20.333333333333258, + "x": 166.66666666666666, + "y": 1938, + "w": 69.00000000000003 + }, + "subrole": null, + "identifier": "eager.83", + "role": "AXButton", + "type": "Button", + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 83" + }, + { + "children": [], + "label": "Eager 84", + "identifier": "eager.84", + "value": null, + "role": "AXButton", + "frame": { + "x": 166.66666666666666, + "w": 69.33333333333334, + "h": 20.333333333333258, + "y": 1960.3333333333333 + }, + "subrole": null, + "enabled": true, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "subrole": null, + "frame": { + "w": 69.00000000000003, + "x": 166.66666666666666, + "y": 1982.6666666666665, + "h": 20.333333333333258 + }, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.85", + "value": null, + "role": "AXButton", + "label": "Eager 85", + "enabled": true, + "type": "Button", + "children": [] + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "children": [], + "label": "Eager 86", + "identifier": "eager.86", + "subrole": null, + "value": null, + "frame": { + "w": 69.33333333333334, + "x": 166.66666666666666, + "h": 20.333333333333258, + "y": 2005 + }, + "enabled": true, + "type": "Button" + }, + { + "label": "Eager 87", + "subrole": null, + "type": "Button", + "frame": { + "y": 2027.3333333333333, + "h": 20.333333333333258, + "x": 167, + "w": 68.33333333333331 + }, + "enabled": true, + "value": null, + "children": [], + "identifier": "eager.87", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton" + }, + { + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 88", + "frame": { + "y": 2049.6666666666665, + "x": 166.66666666666666, + "w": 69.33333333333334, + "h": 20.333333333333485 + }, + "identifier": "eager.88", + "enabled": true, + "children": [], + "type": "Button", + "role": "AXButton", + "value": null + }, + { + "identifier": "eager.89", + "frame": { + "y": 2072, + "w": 69.33333333333334, + "h": 20.33333333333303, + "x": 166.66666666666666 + }, + "enabled": true, + "label": "Eager 89", + "value": null, + "role": "AXButton", + "subrole": null, + "type": "Button", + "children": [], + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "children": [], + "identifier": "eager.90", + "type": "Button", + "role": "AXButton", + "subrole": null, + "label": "Eager 90", + "frame": { + "y": 2094.333333333333, + "w": 69.00000000000003, + "h": 20.333333333333485, + "x": 166.66666666666666 + } + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "subrole": null, + "identifier": "eager.91", + "type": "Button", + "enabled": true, + "value": null, + "frame": { + "y": 2116.6666666666665, + "w": 66.33333333333331, + "x": 168, + "h": 20.333333333333485 + }, + "label": "Eager 91" + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "identifier": "eager.92", + "value": null, + "frame": { + "h": 20.333333333333485, + "x": 167, + "y": 2139, + "w": 68.66666666666666 + }, + "type": "Button", + "children": [], + "role": "AXButton", + "label": "Eager 92", + "enabled": true + }, + { + "children": [], + "identifier": "eager.93", + "role": "AXButton", + "type": "Button", + "subrole": null, + "value": null, + "frame": { + "w": 69.00000000000003, + "h": 20.333333333333485, + "y": 2161.333333333333, + "x": 166.66666666666666 + }, + "enabled": true, + "label": "Eager 93", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "frame": { + "x": 166.66666666666666, + "h": 20.333333333333485, + "y": 2183.6666666666665, + "w": 69.33333333333334 + }, + "label": "Eager 94", + "children": [], + "type": "Button", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "enabled": true, + "value": null, + "identifier": "eager.94" + }, + { + "role": "AXButton", + "children": [], + "type": "Button", + "subrole": null, + "value": null, + "frame": { + "x": 166.66666666666666, + "y": 2206, + "h": 20.333333333333485, + "w": 69.00000000000003 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 95", + "identifier": "eager.95" + }, + { + "type": "Button", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 96", + "frame": { + "w": 69.33333333333334, + "x": 166.66666666666666, + "y": 2228.333333333333, + "h": 20.333333333333485 + }, + "children": [], + "enabled": true, + "role": "AXButton", + "identifier": "eager.96", + "subrole": null + }, + { + "subrole": null, + "role": "AXButton", + "frame": { + "y": 2250.6666666666665, + "h": 20.333333333333485, + "x": 167.33333333333334, + "w": 67.66666666666666 + }, + "enabled": true, + "value": null, + "label": "Eager 97", + "children": [], + "type": "Button", + "identifier": "eager.97", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "enabled": true, + "children": [], + "role": "AXButton", + "type": "Button", + "subrole": null, + "value": null, + "frame": { + "y": 2273, + "x": 166.66666666666666, + "h": 20.333333333333485, + "w": 69.33333333333334 + }, + "label": "Eager 98", + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.98" + }, + { + "value": null, + "identifier": "eager.99", + "enabled": true, + "type": "Button", + "label": "Eager 99", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "x": 166.66666666666666, + "y": 2295.333333333333, + "w": 69.33333333333334, + "h": 20.333333333333485 + }, + "role": "AXButton", + "subrole": null + }, + { + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "subrole": null, + "role": "AXButton", + "label": "Eager 100", + "frame": { + "y": 2317.6666666666665, + "w": 76.33333333333331, + "h": 20.333333333333485, + "x": 163 + }, + "children": [], + "identifier": "eager.100", + "value": null + }, + { + "type": "Button", + "label": "Eager 101", + "subrole": null, + "role": "AXButton", + "frame": { + "w": 73.66666666666666, + "y": 2340, + "h": 20.333333333333485, + "x": 164.33333333333334 + }, + "value": null, + "identifier": "eager.101", + "enabled": true, + "children": [], + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "label": "Eager 102", + "subrole": null, + "type": "Button", + "identifier": "eager.102", + "value": null, + "frame": { + "x": 163, + "y": 2362.333333333333, + "h": 20.333333333333485, + "w": 76.33333333333331 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton" + }, + { + "frame": { + "h": 20.333333333333485, + "x": 163, + "y": 2384.6666666666665, + "w": 76.33333333333331 + }, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 103", + "subrole": null, + "identifier": "eager.103", + "value": null, + "enabled": true, + "role": "AXButton", + "children": [], + "type": "Button" + }, + { + "type": "Button", + "enabled": true, + "subrole": null, + "frame": { + "h": 20.333333333333485, + "y": 2407, + "w": 76.66666666666666, + "x": 163 + }, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "label": "Eager 104", + "identifier": "eager.104", + "role": "AXButton", + "children": [] + }, + { + "subrole": null, + "role": "AXButton", + "type": "Button", + "identifier": "eager.105", + "value": null, + "frame": { + "y": 2429.333333333333, + "x": 163, + "h": 20.333333333333485, + "w": 76.33333333333331 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "label": "Eager 105" + }, + { + "value": null, + "role": "AXButton", + "frame": { + "w": 76.66666666666666, + "h": 20.333333333333485, + "y": 2451.6666666666665, + "x": 163 + }, + "children": [], + "enabled": true, + "identifier": "eager.106", + "label": "Eager 106", + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null + }, + { + "frame": { + "h": 20.333333333333485, + "x": 163.66666666666666, + "w": 75.00000000000003, + "y": 2474 + }, + "label": "Eager 107", + "children": [], + "identifier": "eager.107", + "enabled": true, + "subrole": null, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton" + }, + { + "identifier": "eager.108", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 108", + "children": [], + "frame": { + "h": 20.333333333333485, + "y": 2496.333333333333, + "w": 76.66666666666666, + "x": 163 + }, + "value": null, + "role": "AXButton", + "type": "Button", + "enabled": true, + "subrole": null + }, + { + "type": "Button", + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 109", + "frame": { + "h": 20.333333333333485, + "y": 2518.6666666666665, + "x": 163, + "w": 76.66666666666666 + }, + "role": "AXButton", + "subrole": null, + "identifier": "eager.109", + "children": [] + }, + { + "role": "AXButton", + "type": "Button", + "enabled": true, + "children": [], + "label": "Eager 110", + "frame": { + "y": 2541, + "h": 20.333333333333485, + "x": 164.33333333333334, + "w": 73.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "subrole": null, + "identifier": "eager.110" + }, + { + "frame": { + "w": 70.66666666666666, + "h": 20.333333333333485, + "x": 166, + "y": 2563.333333333333 + }, + "role": "AXButton", + "enabled": true, + "children": [], + "identifier": "eager.111", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 111", + "type": "Button", + "subrole": null + }, + { + "frame": { + "y": 2585.6666666666665, + "w": 73.00000000000003, + "x": 164.66666666666666, + "h": 20.333333333333485 + }, + "children": [], + "role": "AXButton", + "type": "Button", + "label": "Eager 112", + "enabled": true, + "subrole": null, + "identifier": "eager.112", + "value": null, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "type": "Button", + "label": "Eager 113", + "identifier": "eager.113", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "enabled": true, + "frame": { + "w": 73.66666666666666, + "h": 20.333333333333485, + "x": 164.33333333333334, + "y": 2608 + }, + "value": null, + "subrole": null + }, + { + "enabled": true, + "type": "Button", + "value": null, + "label": "Eager 114", + "identifier": "eager.114", + "role": "AXButton", + "frame": { + "x": 164.33333333333334, + "y": 2630.333333333333, + "h": 20.333333333333485, + "w": 73.66666666666666 + }, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "role": "AXButton", + "type": "Button", + "frame": { + "y": 2652.6666666666665, + "x": 164.66666666666666, + "h": 20.333333333333485, + "w": 73.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 115", + "children": [], + "subrole": null, + "enabled": true, + "identifier": "eager.115", + "value": null + }, + { + "value": null, + "frame": { + "w": 73.66666666666666, + "h": 20.333333333333485, + "x": 164.33333333333334, + "y": 2675 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "children": [], + "identifier": "eager.116", + "subrole": null, + "role": "AXButton", + "label": "Eager 116" + }, + { + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "type": "Button", + "identifier": "eager.117", + "frame": { + "x": 165, + "w": 72.66666666666666, + "h": 20.333333333333485, + "y": 2697.333333333333 + }, + "enabled": true, + "value": null, + "subrole": null, + "label": "Eager 117" + }, + { + "frame": { + "w": 73.66666666666666, + "x": 164.33333333333334, + "h": 20.333333333333485, + "y": 2719.6666666666665 + }, + "subrole": null, + "label": "Eager 118", + "value": null, + "identifier": "eager.118", + "enabled": true, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "children": [] + }, + { + "role": "AXButton", + "subrole": null, + "label": "Eager 119", + "type": "Button", + "value": null, + "frame": { + "w": 73.66666666666666, + "x": 164.33333333333334, + "y": 2742, + "h": 20.333333333333485 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "identifier": "eager.119" + }, + { + "type": "Button", + "children": [], + "enabled": true, + "identifier": "eager.120", + "value": null, + "frame": { + "w": 76.33333333333331, + "h": 20.333333333333485, + "x": 163, + "y": 2764.333333333333 + }, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "subrole": null, + "label": "Eager 120" + }, + { + "label": "Eager 121", + "value": null, + "frame": { + "w": 73.00000000000003, + "y": 2786.6666666666665, + "x": 164.66666666666666, + "h": 20.333333333333485 + }, + "enabled": true, + "children": [], + "role": "AXButton", + "identifier": "eager.121", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button" + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "type": "Button", + "subrole": null, + "label": "Eager 122", + "identifier": "eager.122", + "frame": { + "w": 75.66666666666666, + "x": 163.33333333333334, + "y": 2809, + "h": 20.333333333333485 + }, + "enabled": true, + "value": null, + "children": [] + }, + { + "label": "Eager 123", + "enabled": true, + "frame": { + "y": 2831.333333333333, + "x": 163, + "w": 76.33333333333331, + "h": 20.333333333333485 + }, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "value": null, + "identifier": "eager.123", + "children": [] + }, + { + "role": "AXButton", + "value": null, + "identifier": "eager.124", + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "label": "Eager 124", + "children": [], + "frame": { + "x": 163, + "y": 2853.6666666666665, + "w": 76.33333333333331, + "h": 20.333333333333485 + } + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Eager 125", + "identifier": "eager.125", + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "x": 163.33333333333334, + "w": 75.66666666666666, + "y": 2876, + "h": 20.333333333333485 + } + }, + { + "role": "AXButton", + "frame": { + "h": 20.333333333333485, + "x": 163, + "y": 2898.333333333333, + "w": 76.33333333333331 + }, + "type": "Button", + "subrole": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "label": "Eager 126", + "children": [], + "identifier": "eager.126" + }, + { + "frame": { + "w": 75.00000000000003, + "y": 2920.6666666666665, + "h": 20.333333333333485, + "x": 163.66666666666666 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 127", + "value": null, + "subrole": null, + "type": "Button", + "identifier": "eager.127", + "children": [], + "role": "AXButton" + }, + { + "value": null, + "type": "Button", + "role": "AXButton", + "subrole": null, + "frame": { + "w": 76.33333333333331, + "x": 163, + "h": 20.333333333333485, + "y": 2943 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "identifier": "eager.128", + "label": "Eager 128" + }, + { + "identifier": "eager.129", + "label": "Eager 129", + "children": [], + "type": "Button", + "value": null, + "frame": { + "y": 2965.333333333333, + "x": 163, + "w": 76.33333333333331, + "h": 20.333333333333485 + }, + "subrole": null, + "role": "AXButton", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "frame": { + "x": 163, + "y": 2987.6666666666665, + "h": 20.333333333333485, + "w": 76.33333333333331 + }, + "identifier": "eager.130", + "label": "Eager 130", + "value": null, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "enabled": true, + "children": [], + "subrole": null + }, + { + "type": "Button", + "value": null, + "children": [], + "frame": { + "y": 3010, + "h": 20.333333333333485, + "w": 73.66666666666666, + "x": 164.33333333333334 + }, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 131", + "role": "AXButton", + "enabled": true, + "identifier": "eager.131" + }, + { + "enabled": true, + "children": [], + "identifier": "eager.132", + "role": "AXButton", + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 132", + "value": null, + "frame": { + "y": 3032.333333333333, + "w": 76.33333333333331, + "h": 20.333333333333485, + "x": 163 + } + }, + { + "type": "Button", + "enabled": true, + "identifier": "eager.133", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "role": "AXButton", + "label": "Eager 133", + "frame": { + "w": 76.33333333333331, + "h": 20.333333333333485, + "x": 163, + "y": 3054.6666666666665 + }, + "children": [], + "value": null + }, + { + "identifier": "eager.134", + "enabled": true, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "label": "Eager 134", + "children": [], + "subrole": null, + "type": "Button", + "frame": { + "x": 163, + "y": 3077, + "w": 76.66666666666666, + "h": 20.333333333333485 + } + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "subrole": null, + "value": null, + "frame": { + "h": 20.333333333333485, + "y": 3099.333333333333, + "x": 163, + "w": 76.33333333333331 + }, + "label": "Eager 135", + "enabled": true, + "children": [], + "identifier": "eager.135" + }, + { + "type": "Button", + "identifier": "eager.136", + "children": [], + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Eager 136", + "role": "AXButton", + "enabled": true, + "frame": { + "x": 163, + "w": 76.33333333333331, + "y": 3121.6666666666665, + "h": 20.333333333333485 + } + }, + { + "frame": { + "x": 163.66666666666666, + "w": 75.33333333333334, + "h": 20.333333333333485, + "y": 3144 + }, + "children": [], + "role": "AXButton", + "subrole": null, + "label": "Eager 137", + "identifier": "eager.137", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "value": null + }, + { + "label": "Eager 138", + "role": "AXButton", + "value": null, + "frame": { + "h": 20.333333333333485, + "w": 76.66666666666666, + "x": 163, + "y": 3166.333333333333 + }, + "identifier": "eager.138", + "subrole": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "type": "Button" + }, + { + "label": "Eager 139", + "subrole": null, + "type": "Button", + "identifier": "eager.139", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "value": null, + "enabled": true, + "children": [], + "frame": { + "w": 76.33333333333331, + "x": 163, + "y": 3188.6666666666665, + "h": 20.333333333333485 + } + }, + { + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 140", + "children": [], + "type": "Button", + "frame": { + "y": 3211, + "h": 20.333333333333485, + "x": 163, + "w": 76.66666666666666 + }, + "role": "AXButton", + "identifier": "eager.140", + "value": null, + "enabled": true + }, + { + "identifier": "eager.141", + "type": "Button", + "children": [], + "value": null, + "label": "Eager 141", + "role": "AXButton", + "frame": { + "y": 3233.333333333333, + "w": 73.66666666666666, + "x": 164.33333333333334, + "h": 20.333333333333485 + }, + "subrole": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "role": "AXButton", + "value": null, + "identifier": "eager.142", + "type": "Button", + "frame": { + "h": 20.333333333333485, + "w": 76.33333333333331, + "x": 163, + "y": 3255.6666666666665 + }, + "enabled": true, + "children": [], + "label": "Eager 142", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null + }, + { + "identifier": "eager.143", + "label": "Eager 143", + "children": [], + "type": "Button", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "frame": { + "y": 3278, + "x": 163, + "w": 76.66666666666666, + "h": 20.333333333333485 + }, + "subrole": null, + "role": "AXButton" + }, + { + "frame": { + "h": 20.333333333333485, + "x": 162.66666666666666, + "y": 3300.333333333333, + "w": 77.00000000000003 + }, + "identifier": "eager.144", + "enabled": true, + "children": [], + "type": "Button", + "role": "AXButton", + "label": "Eager 144", + "subrole": null, + "value": null, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "children": [], + "label": "Eager 145", + "subrole": null, + "type": "Button", + "identifier": "eager.145", + "value": null, + "frame": { + "w": 76.33333333333331, + "x": 163, + "y": 3322.6666666666665, + "h": 20.333333333333485 + }, + "role": "AXButton", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Eager 146", + "frame": { + "w": 76.66666666666666, + "x": 163, + "y": 3345, + "h": 20.333333333333485 + }, + "role": "AXButton", + "type": "Button", + "identifier": "eager.146", + "children": [], + "value": null + }, + { + "enabled": true, + "frame": { + "w": 75.66666666666666, + "x": 163.33333333333334, + "h": 20.333333333333485, + "y": 3367.333333333333 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "label": "Eager 147", + "value": null, + "type": "Button", + "subrole": null, + "identifier": "eager.147", + "role": "AXButton" + }, + { + "label": "Eager 148", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "type": "Button", + "role": "AXButton", + "subrole": null, + "value": null, + "children": [], + "identifier": "eager.148", + "frame": { + "y": 3389.6666666666665, + "x": 163, + "w": 76.66666666666666, + "h": 20.333333333333485 + } + }, + { + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "enabled": true, + "value": null, + "children": [], + "identifier": "eager.149", + "frame": { + "w": 76.66666666666666, + "h": 20.333333333333485, + "x": 163, + "y": 3412 + }, + "label": "Eager 149" + }, + { + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.150", + "enabled": true, + "children": [], + "role": "AXButton", + "type": "Button", + "label": "Eager 150", + "value": null, + "frame": { + "y": 3434.333333333333, + "x": 163, + "w": 76.33333333333331, + "h": 20.333333333333485 + } + }, + { + "label": "Eager 151", + "frame": { + "h": 20.333333333333485, + "w": 73.33333333333334, + "x": 164.66666666666666, + "y": 3456.6666666666665 + }, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.151", + "value": null, + "enabled": true, + "children": [], + "role": "AXButton", + "subrole": null, + "type": "Button" + }, + { + "subrole": null, + "type": "Button", + "label": "Eager 152", + "frame": { + "w": 75.66666666666666, + "x": 163.33333333333334, + "y": 3479, + "h": 20.333333333333485 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "role": "AXButton", + "identifier": "eager.152", + "value": null + }, + { + "label": "Eager 153", + "identifier": "eager.153", + "enabled": true, + "type": "Button", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "w": 76.33333333333331, + "y": 3501.333333333333, + "x": 163, + "h": 20.333333333333485 + }, + "subrole": null, + "role": "AXButton" + }, + { + "label": "Eager 154", + "type": "Button", + "role": "AXButton", + "identifier": "eager.154", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "children": [], + "frame": { + "y": 3523.6666666666665, + "x": 163, + "w": 76.33333333333331, + "h": 20.333333333333485 + }, + "enabled": true + }, + { + "enabled": true, + "children": [], + "frame": { + "x": 163, + "y": 3546, + "w": 76.33333333333331, + "h": 20.333333333333485 + }, + "value": null, + "type": "Button", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "identifier": "eager.155", + "label": "Eager 155" + }, + { + "value": null, + "identifier": "eager.156", + "subrole": null, + "enabled": true, + "label": "Eager 156", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "type": "Button", + "frame": { + "w": 76.33333333333331, + "y": 3568.333333333333, + "x": 163, + "h": 20.333333333333485 + }, + "role": "AXButton" + }, + { + "children": [], + "type": "Button", + "role": "AXButton", + "enabled": true, + "subrole": null, + "value": null, + "label": "Eager 157", + "frame": { + "w": 75.33333333333334, + "h": 20.333333333333485, + "x": 163.66666666666666, + "y": 3590.6666666666665 + }, + "identifier": "eager.157", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "identifier": "eager.158", + "children": [], + "type": "Button", + "frame": { + "y": 3613, + "x": 163, + "w": 76.33333333333331, + "h": 20.333333333333485 + }, + "role": "AXButton", + "value": null, + "label": "Eager 158", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null + }, + { + "subrole": null, + "enabled": true, + "type": "Button", + "identifier": "eager.159", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 159", + "frame": { + "w": 76.33333333333331, + "x": 163, + "y": 3635.333333333333, + "h": 20.333333333333485 + }, + "children": [], + "value": null, + "role": "AXButton" + }, + { + "value": null, + "subrole": null, + "identifier": "eager.160", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "type": "Button", + "enabled": true, + "label": "Eager 160", + "children": [], + "frame": { + "x": 163, + "w": 76.66666666666666, + "h": 20.333333333333485, + "y": 3657.6666666666665 + } + }, + { + "frame": { + "h": 20.333333333333485, + "y": 3680, + "w": 73.66666666666666, + "x": 164.33333333333334 + }, + "enabled": true, + "label": "Eager 161", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "identifier": "eager.161", + "type": "Button", + "role": "AXButton", + "subrole": null, + "value": null + }, + { + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "frame": { + "h": 20.333333333333485, + "w": 76.33333333333331, + "x": 163, + "y": 3702.333333333333 + }, + "children": [], + "subrole": null, + "value": null, + "identifier": "eager.162", + "label": "Eager 162", + "enabled": true + }, + { + "value": null, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "children": [], + "identifier": "eager.163", + "frame": { + "h": 20.333333333333485, + "w": 76.33333333333331, + "y": 3724.6666666666665, + "x": 163 + }, + "enabled": true, + "label": "Eager 163" + }, + { + "subrole": null, + "frame": { + "y": 3747, + "w": 76.66666666666666, + "h": 20.333333333333485, + "x": 163 + }, + "role": "AXButton", + "type": "Button", + "value": null, + "label": "Eager 164", + "enabled": true, + "identifier": "eager.164", + "children": [], + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "identifier": "eager.165", + "label": "Eager 165", + "frame": { + "x": 163, + "w": 76.33333333333331, + "y": 3769.333333333333, + "h": 20.333333333333485 + }, + "type": "Button", + "value": null, + "children": [], + "subrole": null, + "enabled": true + }, + { + "enabled": true, + "children": [], + "type": "Button", + "frame": { + "y": 3791.6666666666665, + "x": 163, + "w": 76.66666666666666, + "h": 20.333333333333485 + }, + "value": null, + "role": "AXButton", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 166", + "identifier": "eager.166" + }, + { + "subrole": null, + "enabled": true, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "label": "Eager 167", + "identifier": "eager.167", + "role": "AXButton", + "type": "Button", + "frame": { + "w": 75.66666666666666, + "h": 20.333333333333485, + "y": 3814, + "x": 163.33333333333334 + } + }, + { + "role": "AXButton", + "value": null, + "subrole": null, + "enabled": true, + "children": [], + "identifier": "eager.168", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 168", + "type": "Button", + "frame": { + "y": 3836.333333333333, + "x": 163, + "w": 76.66666666666666, + "h": 20.333333333333485 + } + }, + { + "enabled": true, + "children": [], + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "label": "Eager 169", + "identifier": "eager.169", + "role": "AXButton", + "frame": { + "y": 3858.6666666666665, + "x": 163, + "w": 76.66666666666666, + "h": 20.333333333333485 + }, + "subrole": null + }, + { + "frame": { + "w": 75.00000000000003, + "x": 163.66666666666666, + "y": 3881, + "h": 20.333333333333485 + }, + "label": "Eager 170", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "value": null, + "enabled": true, + "identifier": "eager.170", + "role": "AXButton", + "children": [] + }, + { + "role": "AXButton", + "children": [], + "identifier": "eager.171", + "enabled": true, + "type": "Button", + "value": null, + "subrole": null, + "label": "Eager 171", + "frame": { + "y": 3903.333333333333, + "h": 20.333333333333485, + "w": 72.66666666666666, + "x": 165 + }, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "label": "Eager 172", + "enabled": true, + "subrole": null, + "identifier": "eager.172", + "children": [], + "value": null, + "role": "AXButton", + "type": "Button", + "frame": { + "w": 75.00000000000003, + "y": 3925.6666666666665, + "x": 163.66666666666666, + "h": 20.333333333333485 + }, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "type": "Button", + "label": "Eager 173", + "subrole": null, + "frame": { + "h": 20.333333333333485, + "x": 163.66666666666666, + "w": 75.00000000000003, + "y": 3948 + }, + "enabled": true, + "children": [], + "identifier": "eager.173", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "value": null + }, + { + "subrole": null, + "value": null, + "frame": { + "h": 20.333333333333485, + "y": 3970.333333333333, + "w": 74.66666666666666, + "x": 164 + }, + "enabled": true, + "children": [], + "role": "AXButton", + "label": "Eager 174", + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.174", + "type": "Button" + }, + { + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "label": "Eager 175", + "children": [], + "subrole": null, + "type": "Button", + "identifier": "eager.175", + "frame": { + "w": 75.00000000000003, + "y": 3992.6666666666665, + "x": 163.66666666666666, + "h": 20.333333333333485 + }, + "enabled": true + }, + { + "enabled": true, + "label": "Eager 176", + "identifier": "eager.176", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "y": 4015, + "x": 163.66666666666666, + "w": 75.33333333333334, + "h": 20.333333333333485 + }, + "type": "Button", + "children": [], + "role": "AXButton", + "subrole": null, + "value": null + }, + { + "subrole": null, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "value": null, + "children": [], + "identifier": "eager.177", + "label": "Eager 177", + "enabled": true, + "frame": { + "x": 164, + "w": 74.66666666666666, + "h": 20.333333333333485, + "y": 4037.333333333333 + } + }, + { + "label": "Eager 178", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.178", + "frame": { + "x": 163.66666666666666, + "w": 75.33333333333334, + "y": 4059.6666666666665, + "h": 20.333333333333485 + }, + "enabled": true, + "type": "Button", + "value": null, + "children": [], + "subrole": null + }, + { + "identifier": "eager.179", + "children": [], + "role": "AXButton", + "frame": { + "h": 20.33333333333394, + "w": 75.33333333333334, + "y": 4082, + "x": 163.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "label": "Eager 179", + "type": "Button", + "subrole": null, + "enabled": true + }, + { + "role": "AXButton", + "type": "Button", + "identifier": "eager.180", + "frame": { + "x": 163, + "y": 4104.333333333333, + "h": 20.33333333333303, + "w": 76.66666666666666 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Eager 180", + "value": null, + "children": [] + }, + { + "label": "Eager 181", + "subrole": null, + "role": "AXButton", + "identifier": "eager.181", + "value": null, + "frame": { + "x": 164.33333333333334, + "h": 20.33333333333394, + "y": 4126.666666666666, + "w": 73.66666666666666 + }, + "enabled": true, + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button" + }, + { + "role": "AXButton", + "label": "Eager 182", + "subrole": null, + "value": null, + "type": "Button", + "identifier": "eager.182", + "children": [], + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 163, + "y": 4149, + "w": 76.33333333333331, + "h": 20.33333333333303 + } + }, + { + "label": "Eager 183", + "enabled": true, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "identifier": "eager.183", + "frame": { + "y": 4171.333333333333, + "x": 163, + "h": 20.33333333333303, + "w": 76.66666666666666 + }, + "subrole": null, + "children": [] + }, + { + "subrole": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "identifier": "eager.184", + "children": [], + "label": "Eager 184", + "value": null, + "frame": { + "h": 20.33333333333303, + "w": 76.66666666666666, + "y": 4193.666666666666, + "x": 163 + }, + "enabled": true + }, + { + "children": [], + "subrole": null, + "value": null, + "role": "AXButton", + "label": "Eager 185", + "frame": { + "y": 4216, + "x": 163, + "h": 20.33333333333303, + "w": 76.33333333333331 + }, + "identifier": "eager.185", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "enabled": true + }, + { + "type": "Button", + "identifier": "eager.186", + "children": [], + "enabled": true, + "label": "Eager 186", + "value": null, + "frame": { + "x": 163, + "h": 20.33333333333303, + "w": 76.66666666666666, + "y": 4238.333333333333 + }, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null + }, + { + "enabled": true, + "label": "Eager 187", + "value": null, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "x": 163.33333333333334, + "w": 75.66666666666666, + "y": 4260.666666666666, + "h": 20.33333333333303 + }, + "identifier": "eager.187", + "subrole": null, + "type": "Button" + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.188", + "frame": { + "h": 20.33333333333303, + "w": 76.66666666666666, + "x": 163, + "y": 4283 + }, + "children": [], + "enabled": true, + "type": "Button", + "value": null, + "label": "Eager 188", + "role": "AXButton", + "subrole": null + }, + { + "subrole": null, + "type": "Button", + "identifier": "eager.189", + "value": null, + "enabled": true, + "role": "AXButton", + "frame": { + "x": 163, + "y": 4305.333333333333, + "h": 20.33333333333303, + "w": 76.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "label": "Eager 189" + }, + { + "enabled": true, + "subrole": null, + "identifier": "eager.190", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "children": [], + "value": null, + "frame": { + "x": 163, + "w": 76.66666666666666, + "h": 20.33333333333303, + "y": 4327.666666666666 + }, + "type": "Button", + "label": "Eager 190" + }, + { + "frame": { + "y": 4350, + "h": 20.33333333333303, + "w": 73.66666666666666, + "x": 164.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "label": "Eager 191", + "value": null, + "enabled": true, + "children": [], + "subrole": null, + "role": "AXButton", + "identifier": "eager.191" + }, + { + "children": [], + "subrole": null, + "identifier": "eager.192", + "value": null, + "type": "Button", + "enabled": true, + "role": "AXButton", + "frame": { + "y": 4372.333333333333, + "x": 163, + "h": 20.33333333333303, + "w": 76.33333333333331 + }, + "label": "Eager 192", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "subrole": null, + "frame": { + "w": 76.33333333333331, + "x": 163, + "h": 20.33333333333303, + "y": 4394.666666666666 + }, + "role": "AXButton", + "identifier": "eager.193", + "label": "Eager 193", + "enabled": true, + "children": [], + "type": "Button", + "value": null, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 163, + "w": 76.66666666666666, + "y": 4417, + "h": 20.33333333333303 + }, + "role": "AXButton", + "label": "Eager 194", + "children": [], + "type": "Button", + "identifier": "eager.194", + "subrole": null, + "enabled": true, + "value": null + }, + { + "subrole": null, + "type": "Button", + "label": "Eager 195", + "role": "AXButton", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 163, + "y": 4439.333333333333, + "h": 20.33333333333303, + "w": 76.33333333333331 + }, + "value": null, + "identifier": "eager.195", + "children": [] + }, + { + "type": "Button", + "label": "Eager 196", + "role": "AXButton", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "x": 163, + "y": 4461.666666666666, + "h": 20.33333333333303, + "w": 76.66666666666666 + }, + "identifier": "eager.196", + "subrole": null, + "value": null + }, + { + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "label": "Eager 197", + "type": "Button", + "identifier": "eager.197", + "subrole": null, + "value": null, + "frame": { + "y": 4484, + "w": 75.33333333333334, + "h": 20.33333333333303, + "x": 163.66666666666666 + }, + "children": [] + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "subrole": null, + "role": "AXButton", + "type": "Button", + "label": "Eager 198", + "value": null, + "identifier": "eager.198", + "frame": { + "w": 76.66666666666666, + "h": 20.33333333333303, + "y": 4506.333333333333, + "x": 163 + }, + "enabled": true + }, + { + "type": "Button", + "label": "Eager 199", + "subrole": null, + "enabled": true, + "role": "AXButton", + "value": null, + "frame": { + "x": 163, + "y": 4528.666666666666, + "w": 76.66666666666666, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "identifier": "eager.199" + }, + { + "value": null, + "children": [], + "type": "Button", + "label": "Eager 200", + "role": "AXButton", + "subrole": null, + "identifier": "eager.200", + "frame": { + "w": 78.66666666666666, + "h": 20.33333333333303, + "x": 162, + "y": 4551 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "value": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "enabled": true, + "subrole": null, + "label": "Eager 201", + "identifier": "eager.201", + "frame": { + "y": 4573.333333333333, + "h": 20.33333333333303, + "x": 163, + "w": 76.33333333333331 + } + }, + { + "enabled": true, + "identifier": "eager.202", + "type": "Button", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 202", + "children": [], + "role": "AXButton", + "frame": { + "x": 162, + "y": 4595.666666666666, + "h": 20.33333333333303, + "w": 78.33333333333331 + }, + "value": null + }, + { + "value": null, + "frame": { + "y": 4618, + "w": 78.66666666666666, + "h": 20.33333333333303, + "x": 162 + }, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.203", + "children": [], + "enabled": true, + "role": "AXButton", + "subrole": null, + "label": "Eager 203", + "type": "Button" + }, + { + "subrole": null, + "label": "Eager 204", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 161.66666666666666, + "y": 4640.333333333333, + "w": 79.00000000000003, + "h": 20.33333333333303 + }, + "children": [], + "type": "Button", + "value": null, + "enabled": true, + "role": "AXButton", + "identifier": "eager.204" + }, + { + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "identifier": "eager.205", + "label": "Eager 205", + "subrole": null, + "enabled": true, + "frame": { + "w": 78.66666666666666, + "y": 4662.666666666666, + "h": 20.33333333333303, + "x": 162 + }, + "type": "Button", + "value": null + }, + { + "value": null, + "subrole": null, + "frame": { + "h": 20.33333333333303, + "x": 161.66666666666666, + "w": 79.00000000000003, + "y": 4685 + }, + "type": "Button", + "label": "Eager 206", + "identifier": "eager.206", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "enabled": true + }, + { + "identifier": "eager.207", + "enabled": true, + "frame": { + "x": 162.66666666666666, + "y": 4707.333333333333, + "h": 20.33333333333303, + "w": 77.33333333333334 + }, + "role": "AXButton", + "subrole": null, + "value": null, + "type": "Button", + "label": "Eager 207", + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "role": "AXButton", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "children": [], + "subrole": null, + "frame": { + "x": 161.66666666666666, + "h": 20.33333333333303, + "y": 4729.666666666666, + "w": 79.00000000000003 + }, + "label": "Eager 208", + "type": "Button", + "identifier": "eager.208" + }, + { + "identifier": "eager.209", + "type": "Button", + "label": "Eager 209", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "children": [], + "value": null, + "frame": { + "x": 161.66666666666666, + "w": 79.00000000000003, + "y": 4752, + "h": 20.33333333333303 + }, + "role": "AXButton" + }, + { + "identifier": "eager.210", + "subrole": null, + "label": "Eager 210", + "frame": { + "x": 163, + "w": 76.33333333333331, + "y": 4774.333333333333, + "h": 20.33333333333303 + }, + "type": "Button", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "children": [], + "value": null + }, + { + "children": [], + "subrole": null, + "type": "Button", + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.211", + "role": "AXButton", + "label": "Eager 211", + "frame": { + "w": 73.00000000000003, + "h": 20.33333333333303, + "x": 164.66666666666666, + "y": 4796.666666666666 + } + }, + { + "label": "Eager 212", + "subrole": null, + "role": "AXButton", + "enabled": true, + "frame": { + "y": 4819, + "h": 20.33333333333303, + "w": 75.66666666666666, + "x": 163.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.212", + "value": null, + "children": [] + }, + { + "subrole": null, + "children": [], + "identifier": "eager.213", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "y": 4841.333333333333, + "w": 76.33333333333331, + "h": 20.33333333333303, + "x": 163 + }, + "value": null, + "label": "Eager 213", + "type": "Button", + "enabled": true, + "role": "AXButton" + }, + { + "children": [], + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.214", + "subrole": null, + "label": "Eager 214", + "enabled": true, + "role": "AXButton", + "type": "Button", + "frame": { + "h": 20.33333333333303, + "w": 76.33333333333331, + "x": 163, + "y": 4863.666666666666 + } + }, + { + "role": "AXButton", + "subrole": null, + "frame": { + "x": 163.33333333333334, + "y": 4886, + "w": 75.66666666666666, + "h": 20.33333333333303 + }, + "label": "Eager 215", + "type": "Button", + "identifier": "eager.215", + "value": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "type": "Button", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.216", + "value": null, + "frame": { + "w": 76.33333333333331, + "x": 163, + "y": 4908.333333333333, + "h": 20.33333333333303 + }, + "children": [], + "enabled": true, + "label": "Eager 216", + "subrole": null + }, + { + "type": "Button", + "value": null, + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 217", + "subrole": null, + "enabled": true, + "identifier": "eager.217", + "role": "AXButton", + "frame": { + "x": 163.66666666666666, + "y": 4930.666666666666, + "h": 20.33333333333303, + "w": 75.00000000000003 + } + }, + { + "identifier": "eager.218", + "children": [], + "label": "Eager 218", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "value": null, + "frame": { + "y": 4953, + "x": 163, + "w": 76.33333333333331, + "h": 20.33333333333303 + }, + "subrole": null, + "enabled": true + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "children": [], + "role": "AXButton", + "type": "Button", + "frame": { + "x": 163, + "y": 4975.333333333333, + "w": 76.33333333333331, + "h": 20.33333333333303 + }, + "value": null, + "label": "Eager 219", + "identifier": "eager.219", + "subrole": null + }, + { + "type": "Button", + "children": [], + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 220", + "subrole": null, + "identifier": "eager.220", + "frame": { + "y": 4997.666666666666, + "x": 162, + "w": 78.33333333333331, + "h": 20.33333333333303 + }, + "enabled": true, + "role": "AXButton" + }, + { + "identifier": "eager.221", + "subrole": null, + "type": "Button", + "enabled": true, + "children": [], + "value": null, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 221", + "frame": { + "y": 5020, + "x": 163.33333333333334, + "h": 20.33333333333303, + "w": 75.66666666666666 + } + }, + { + "role": "AXButton", + "subrole": null, + "value": null, + "children": [], + "label": "Eager 222", + "frame": { + "w": 78.33333333333331, + "x": 162, + "y": 5042.333333333333, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.222", + "type": "Button", + "enabled": true + }, + { + "frame": { + "h": 20.33333333333303, + "x": 162, + "y": 5064.666666666666, + "w": 78.33333333333331 + }, + "children": [], + "value": null, + "subrole": null, + "type": "Button", + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "label": "Eager 223", + "identifier": "eager.223" + }, + { + "type": "Button", + "value": null, + "role": "AXButton", + "subrole": null, + "enabled": true, + "identifier": "eager.224", + "children": [], + "frame": { + "w": 78.33333333333331, + "y": 5087, + "x": 162, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 224" + }, + { + "children": [], + "frame": { + "w": 78.33333333333331, + "x": 162, + "y": 5109.333333333333, + "h": 20.33333333333303 + }, + "enabled": true, + "type": "Button", + "identifier": "eager.225", + "label": "Eager 225", + "role": "AXButton", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "value": null + }, + { + "identifier": "eager.226", + "children": [], + "frame": { + "x": 162, + "h": 20.33333333333303, + "y": 5131.666666666666, + "w": 78.33333333333331 + }, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "value": null, + "enabled": true, + "label": "Eager 226" + }, + { + "enabled": true, + "label": "Eager 227", + "subrole": null, + "children": [], + "value": null, + "frame": { + "h": 20.33333333333303, + "y": 5154, + "x": 162.66666666666666, + "w": 77.33333333333334 + }, + "identifier": "eager.227", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton" + }, + { + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.228", + "value": null, + "children": [], + "frame": { + "h": 20.33333333333303, + "x": 162, + "y": 5176.333333333333, + "w": 78.33333333333331 + }, + "enabled": true, + "label": "Eager 228", + "role": "AXButton" + }, + { + "subrole": null, + "label": "Eager 229", + "frame": { + "y": 5198.666666666666, + "w": 78.33333333333331, + "x": 162, + "h": 20.33333333333303 + }, + "identifier": "eager.229", + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "type": "Button", + "role": "AXButton", + "children": [], + "enabled": true + }, + { + "identifier": "eager.230", + "subrole": null, + "type": "Button", + "children": [], + "role": "AXButton", + "label": "Eager 230", + "enabled": true, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "y": 5221, + "w": 78.66666666666666, + "x": 162, + "h": 20.33333333333303 + } + }, + { + "type": "Button", + "label": "Eager 231", + "enabled": true, + "identifier": "eager.231", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 163, + "w": 76.33333333333331, + "y": 5243.333333333333, + "h": 20.33333333333303 + }, + "children": [], + "value": null, + "role": "AXButton", + "subrole": null + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "role": "AXButton", + "subrole": null, + "label": "Eager 232", + "frame": { + "x": 162, + "y": 5265.666666666666, + "w": 78.33333333333331, + "h": 20.33333333333303 + }, + "identifier": "eager.232", + "value": null, + "enabled": true, + "children": [] + }, + { + "enabled": true, + "identifier": "eager.233", + "type": "Button", + "subrole": null, + "frame": { + "w": 78.66666666666666, + "x": 162, + "y": 5288, + "h": 20.33333333333303 + }, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "label": "Eager 233", + "children": [] + }, + { + "frame": { + "w": 79.00000000000003, + "y": 5310.333333333333, + "x": 161.66666666666666, + "h": 20.33333333333303 + }, + "value": null, + "subrole": null, + "type": "Button", + "label": "Eager 234", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "identifier": "eager.234", + "enabled": true, + "children": [] + }, + { + "label": "Eager 235", + "frame": { + "h": 20.33333333333303, + "w": 78.66666666666666, + "y": 5332.666666666666, + "x": 162 + }, + "value": null, + "role": "AXButton", + "enabled": true, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "subrole": null, + "identifier": "eager.235" + }, + { + "frame": { + "h": 20.33333333333303, + "x": 161.66666666666666, + "w": 79.00000000000003, + "y": 5355 + }, + "children": [], + "type": "Button", + "label": "Eager 236", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "value": null, + "identifier": "eager.236", + "enabled": true, + "subrole": null + }, + { + "subrole": null, + "value": null, + "enabled": true, + "label": "Eager 237", + "frame": { + "x": 162.33333333333334, + "w": 77.66666666666666, + "y": 5377.333333333333, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.237", + "role": "AXButton", + "children": [] + }, + { + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.238", + "frame": { + "w": 79.00000000000003, + "x": 161.66666666666666, + "y": 5399.666666666666, + "h": 20.33333333333303 + }, + "role": "AXButton", + "type": "Button", + "children": [], + "label": "Eager 238", + "subrole": null, + "enabled": true + }, + { + "value": null, + "enabled": true, + "subrole": null, + "children": [], + "type": "Button", + "role": "AXButton", + "label": "Eager 239", + "identifier": "eager.239", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "y": 5422, + "x": 161.66666666666666, + "h": 20.33333333333303, + "w": 79.00000000000003 + } + }, + { + "role": "AXButton", + "type": "Button", + "value": null, + "label": "Eager 240", + "frame": { + "x": 162, + "w": 78.66666666666666, + "h": 20.33333333333303, + "y": 5444.333333333333 + }, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "enabled": true, + "identifier": "eager.240" + }, + { + "enabled": true, + "type": "Button", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Eager 241", + "value": null, + "identifier": "eager.241", + "role": "AXButton", + "frame": { + "y": 5466.666666666666, + "x": 163.33333333333334, + "w": 75.66666666666666, + "h": 20.33333333333303 + } + }, + { + "value": null, + "children": [], + "label": "Eager 242", + "enabled": true, + "type": "Button", + "identifier": "eager.242", + "frame": { + "w": 78.33333333333331, + "h": 20.33333333333303, + "x": 162, + "y": 5489 + }, + "role": "AXButton", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "role": "AXButton", + "frame": { + "x": 162, + "y": 5511.333333333333, + "w": 78.66666666666666, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "identifier": "eager.243", + "type": "Button", + "label": "Eager 243", + "value": null, + "subrole": null, + "enabled": true + }, + { + "frame": { + "y": 5533.666666666666, + "w": 79.00000000000003, + "x": 161.66666666666666, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "subrole": null, + "role": "AXButton", + "label": "Eager 244", + "value": null, + "children": [], + "identifier": "eager.244", + "enabled": true + }, + { + "enabled": true, + "role": "AXButton", + "label": "Eager 245", + "subrole": null, + "identifier": "eager.245", + "type": "Button", + "frame": { + "x": 162, + "h": 20.33333333333303, + "w": 78.66666666666666, + "y": 5556 + }, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button", + "children": [], + "role": "AXButton", + "enabled": true, + "identifier": "eager.246", + "label": "Eager 246", + "value": null, + "frame": { + "x": 161.66666666666666, + "y": 5578.333333333333, + "h": 20.33333333333303, + "w": 79.00000000000003 + } + }, + { + "type": "Button", + "value": null, + "enabled": true, + "children": [], + "role": "AXButton", + "subrole": null, + "label": "Eager 247", + "identifier": "eager.247", + "frame": { + "h": 20.33333333333303, + "x": 162.33333333333334, + "y": 5600.666666666666, + "w": 77.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.248", + "children": [], + "role": "AXButton", + "value": null, + "label": "Eager 248", + "frame": { + "h": 20.33333333333303, + "w": 79.00000000000003, + "y": 5623, + "x": 161.66666666666666 + }, + "enabled": true + }, + { + "enabled": true, + "role": "AXButton", + "subrole": null, + "label": "Eager 249", + "identifier": "eager.249", + "type": "Button", + "frame": { + "h": 20.33333333333303, + "y": 5645.333333333333, + "w": 79.00000000000003, + "x": 161.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "value": null + }, + { + "frame": { + "y": 5667.666666666666, + "x": 162, + "w": 78.66666666666666, + "h": 20.33333333333303 + }, + "identifier": "eager.250", + "label": "Eager 250", + "role": "AXButton", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "type": "Button", + "children": [], + "value": null + }, + { + "type": "Button", + "value": null, + "frame": { + "x": 163.33333333333334, + "w": 75.66666666666666, + "h": 20.33333333333303, + "y": 5690 + }, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Eager 251", + "identifier": "eager.251", + "role": "AXButton", + "enabled": true, + "children": [] + }, + { + "label": "Eager 252", + "enabled": true, + "subrole": null, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "type": "Button", + "identifier": "eager.252", + "frame": { + "y": 5712.333333333333, + "w": 78.33333333333331, + "h": 20.33333333333303, + "x": 162 + } + }, + { + "children": [], + "frame": { + "h": 20.33333333333303, + "y": 5734.666666666666, + "x": 162, + "w": 78.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 253", + "value": null, + "role": "AXButton", + "subrole": null, + "identifier": "eager.253", + "type": "Button", + "enabled": true + }, + { + "subrole": null, + "type": "Button", + "role": "AXButton", + "value": null, + "enabled": true, + "children": [], + "identifier": "eager.254", + "label": "Eager 254", + "frame": { + "w": 78.66666666666666, + "x": 162, + "y": 5757, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "children": [], + "type": "Button", + "role": "AXButton", + "frame": { + "x": 162, + "h": 20.33333333333303, + "y": 5779.333333333333, + "w": 78.33333333333331 + }, + "identifier": "eager.255", + "enabled": true, + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Eager 255" + }, + { + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "role": "AXButton", + "value": null, + "label": "Eager 256", + "type": "Button", + "subrole": null, + "children": [], + "identifier": "eager.256", + "frame": { + "x": 162, + "h": 20.33333333333303, + "y": 5801.666666666666, + "w": 78.66666666666666 + } + }, + { + "subrole": null, + "type": "Button", + "value": null, + "enabled": true, + "label": "Eager 257", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "h": 20.33333333333303, + "x": 162.33333333333334, + "w": 77.66666666666666, + "y": 5824 + }, + "role": "AXButton", + "children": [], + "identifier": "eager.257" + }, + { + "identifier": "eager.258", + "subrole": null, + "enabled": true, + "type": "Button", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 258", + "frame": { + "h": 20.33333333333303, + "y": 5846.333333333333, + "x": 162, + "w": 78.66666666666666 + }, + "value": null, + "role": "AXButton" + }, + { + "frame": { + "h": 20.33333333333303, + "w": 78.66666666666666, + "y": 5868.666666666666, + "x": 162 + }, + "children": [], + "subrole": null, + "identifier": "eager.259", + "value": null, + "type": "Button", + "label": "Eager 259", + "role": "AXButton", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 260", + "subrole": null, + "type": "Button", + "value": null, + "identifier": "eager.260", + "children": [], + "frame": { + "x": 161.66666666666666, + "h": 20.33333333333303, + "w": 79.00000000000003, + "y": 5891 + }, + "enabled": true + }, + { + "subrole": null, + "label": "Eager 261", + "identifier": "eager.261", + "value": null, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true, + "frame": { + "y": 5913.333333333333, + "x": 163, + "h": 20.33333333333303, + "w": 76.33333333333331 + }, + "children": [], + "role": "AXButton" + }, + { + "children": [], + "frame": { + "x": 162, + "y": 5935.666666666666, + "w": 78.33333333333331, + "h": 20.33333333333303 + }, + "label": "Eager 262", + "subrole": null, + "type": "Button", + "role": "AXButton", + "value": null, + "identifier": "eager.262", + "traits": [ + "Button", + "Scrollable" + ], + "enabled": true + }, + { + "type": "Button", + "subrole": null, + "identifier": "eager.263", + "label": "Eager 263", + "value": null, + "frame": { + "y": 5958, + "w": 79.00000000000003, + "h": 20.33333333333303, + "x": 161.66666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "enabled": true + }, + { + "identifier": "eager.264", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "type": "Button", + "subrole": null, + "label": "Eager 264", + "enabled": true, + "children": [], + "value": null, + "frame": { + "y": 5980.333333333333, + "w": 79.00000000000003, + "x": 161.66666666666666, + "h": 20.33333333333303 + } + }, + { + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 265", + "identifier": "eager.265", + "subrole": null, + "value": null, + "frame": { + "y": 6002.666666666666, + "w": 78.66666666666666, + "x": 162, + "h": 20.33333333333303 + }, + "children": [], + "enabled": true, + "type": "Button" + }, + { + "value": null, + "enabled": true, + "role": "AXButton", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 161.66666666666666, + "y": 6025, + "w": 79.00000000000003, + "h": 20.33333333333303 + }, + "type": "Button", + "children": [], + "label": "Eager 266", + "identifier": "eager.266", + "subrole": null + }, + { + "frame": { + "x": 162, + "y": 6047.333333333333, + "w": 78.33333333333331, + "h": 20.33333333333303 + }, + "identifier": "eager.267", + "subrole": null, + "type": "Button", + "enabled": true, + "value": null, + "label": "Eager 267", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "children": [] + }, + { + "frame": { + "y": 6069.666666666666, + "w": 79.00000000000003, + "x": 161.66666666666666, + "h": 20.33333333333303 + }, + "enabled": true, + "identifier": "eager.268", + "role": "AXButton", + "type": "Button", + "subrole": null, + "value": null, + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 268" + }, + { + "identifier": "eager.269", + "type": "Button", + "enabled": true, + "children": [], + "role": "AXButton", + "subrole": null, + "value": null, + "label": "Eager 269", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "y": 6092, + "x": 161.66666666666666, + "w": 79.00000000000003, + "h": 20.33333333333303 + } + }, + { + "children": [], + "subrole": null, + "role": "AXButton", + "identifier": "eager.270", + "type": "Button", + "enabled": true, + "label": "Eager 270", + "value": null, + "frame": { + "h": 20.33333333333303, + "w": 77.66666666666666, + "x": 162.33333333333334, + "y": 6114.333333333333 + }, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "children": [], + "label": "Eager 271", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "subrole": null, + "identifier": "eager.271", + "type": "Button", + "value": null, + "frame": { + "h": 20.33333333333303, + "x": 163.66666666666666, + "y": 6136.666666666666, + "w": 75.00000000000003 + }, + "enabled": true + }, + { + "identifier": "eager.272", + "role": "AXButton", + "type": "Button", + "label": "Eager 272", + "enabled": true, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "value": null, + "frame": { + "x": 162.66666666666666, + "h": 20.33333333333303, + "y": 6159, + "w": 77.33333333333334 + } + }, + { + "role": "AXButton", + "value": null, + "frame": { + "x": 162.66666666666666, + "w": 77.33333333333334, + "h": 20.33333333333303, + "y": 6181.333333333333 + }, + "children": [], + "identifier": "eager.273", + "label": "Eager 273", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "subrole": null + }, + { + "type": "Button", + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "identifier": "eager.274", + "children": [], + "frame": { + "x": 162.66666666666666, + "h": 20.33333333333303, + "y": 6203.666666666666, + "w": 77.00000000000003 + }, + "label": "Eager 274", + "enabled": true, + "role": "AXButton" + }, + { + "value": null, + "role": "AXButton", + "children": [], + "label": "Eager 275", + "enabled": true, + "subrole": null, + "frame": { + "x": 162.66666666666666, + "h": 20.33333333333303, + "y": 6226, + "w": 77.33333333333334 + }, + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "identifier": "eager.275" + }, + { + "value": null, + "type": "Button", + "identifier": "eager.276", + "label": "Eager 276", + "frame": { + "h": 20.33333333333303, + "x": 162.33333333333334, + "y": 6248.333333333333, + "w": 77.66666666666666 + }, + "children": [], + "role": "AXButton", + "subrole": null, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "label": "Eager 277", + "role": "AXButton", + "type": "Button", + "frame": { + "y": 6270.666666666666, + "x": 162.66666666666666, + "h": 20.33333333333303, + "w": 77.00000000000003 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "children": [], + "subrole": null, + "identifier": "eager.277" + }, + { + "frame": { + "x": 162.33333333333334, + "h": 20.33333333333303, + "w": 77.66666666666666, + "y": 6293 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "type": "Button", + "subrole": null, + "identifier": "eager.278", + "label": "Eager 278", + "value": null + }, + { + "role": "AXButton", + "label": "Eager 279", + "subrole": null, + "identifier": "eager.279", + "type": "Button", + "children": [], + "frame": { + "y": 6315.333333333333, + "w": 77.66666666666666, + "x": 162.33333333333334, + "h": 20.33333333333303 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "value": null + }, + { + "label": "Eager 280", + "role": "AXButton", + "identifier": "eager.280", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "w": 79.00000000000003, + "h": 20.33333333333303, + "y": 6337.666666666666, + "x": 161.66666666666666 + }, + "children": [], + "type": "Button", + "value": null, + "enabled": true, + "subrole": null + }, + { + "value": null, + "enabled": true, + "subrole": null, + "role": "AXButton", + "type": "Button", + "children": [], + "frame": { + "y": 6360, + "h": 20.33333333333303, + "w": 76.33333333333331, + "x": 163 + }, + "label": "Eager 281", + "identifier": "eager.281", + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "frame": { + "x": 162, + "h": 20.33333333333303, + "y": 6382.333333333333, + "w": 78.33333333333331 + }, + "identifier": "eager.282", + "enabled": true, + "value": null, + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button", + "label": "Eager 282", + "role": "AXButton" + }, + { + "value": null, + "identifier": "eager.283", + "children": [], + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Eager 283", + "role": "AXButton", + "frame": { + "x": 161.66666666666666, + "h": 20.33333333333303, + "y": 6404.666666666666, + "w": 79.00000000000003 + }, + "subrole": null, + "enabled": true + }, + { + "identifier": "eager.284", + "children": [], + "enabled": true, + "value": null, + "label": "Eager 284", + "subrole": null, + "frame": { + "h": 20.33333333333303, + "y": 6427, + "x": 161.66666666666666, + "w": 79.33333333333334 + }, + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton" + }, + { + "enabled": true, + "type": "Button", + "children": [], + "role": "AXButton", + "label": "Eager 285", + "identifier": "eager.285", + "frame": { + "y": 6449.333333333333, + "h": 20.33333333333303, + "w": 78.66666666666666, + "x": 162 + }, + "subrole": null, + "value": null, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "enabled": true, + "label": "Eager 286", + "identifier": "eager.286", + "role": "AXButton", + "subrole": null, + "type": "Button", + "frame": { + "w": 79.00000000000003, + "x": 161.66666666666666, + "h": 20.33333333333303, + "y": 6471.666666666666 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "value": null + }, + { + "enabled": true, + "subrole": null, + "value": null, + "role": "AXButton", + "label": "Eager 287", + "identifier": "eager.287", + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "frame": { + "x": 162, + "h": 20.33333333333303, + "w": 78.33333333333331, + "y": 6494 + }, + "type": "Button" + }, + { + "label": "Eager 288", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button", + "enabled": true, + "identifier": "eager.288", + "role": "AXButton", + "frame": { + "h": 20.33333333333303, + "x": 161.66666666666666, + "y": 6516.333333333333, + "w": 79.00000000000003 + }, + "children": [], + "value": null + }, + { + "identifier": "eager.289", + "children": [], + "traits": [ + "Button", + "Scrollable" + ], + "value": null, + "frame": { + "x": 161.66666666666666, + "h": 20.33333333333303, + "y": 6538.666666666666, + "w": 79.00000000000003 + }, + "subrole": null, + "role": "AXButton", + "label": "Eager 289", + "enabled": true, + "type": "Button" + }, + { + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "x": 161.66666666666666, + "y": 6561, + "w": 79.00000000000003, + "h": 20.33333333333303 + }, + "role": "AXButton", + "type": "Button", + "enabled": true, + "identifier": "eager.290", + "children": [], + "subrole": null, + "label": "Eager 290" + }, + { + "value": null, + "role": "AXButton", + "frame": { + "y": 6583.333333333333, + "h": 20.33333333333303, + "w": 76.33333333333331, + "x": 163 + }, + "type": "Button", + "label": "Eager 291", + "identifier": "eager.291", + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "subrole": null + }, + { + "value": null, + "identifier": "eager.292", + "enabled": true, + "subrole": null, + "type": "Button", + "role": "AXButton", + "frame": { + "y": 6605.666666666666, + "x": 162, + "w": 78.33333333333331, + "h": 20.33333333333303 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "label": "Eager 292" + }, + { + "children": [], + "enabled": true, + "subrole": null, + "identifier": "eager.293", + "value": null, + "label": "Eager 293", + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "role": "AXButton", + "frame": { + "x": 161.66666666666666, + "y": 6628, + "h": 20.33333333333303, + "w": 79.00000000000003 + } + }, + { + "subrole": null, + "value": null, + "type": "Button", + "role": "AXButton", + "identifier": "eager.294", + "frame": { + "x": 161.66666666666666, + "y": 6650.333333333333, + "w": 79.00000000000003, + "h": 20.33333333333303 + }, + "enabled": true, + "label": "Eager 294", + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button", + "role": "AXButton", + "label": "Eager 295", + "value": null, + "children": [], + "frame": { + "y": 6672.666666666666, + "h": 20.33333333333303, + "w": 78.66666666666666, + "x": 162 + }, + "identifier": "eager.295" + }, + { + "identifier": "eager.296", + "value": null, + "children": [], + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Eager 296", + "frame": { + "h": 20.33333333333303, + "x": 161.66666666666666, + "w": 79.00000000000003, + "y": 6695 + }, + "enabled": true, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "children": [], + "type": "Button", + "label": "Eager 297", + "enabled": true, + "subrole": null, + "frame": { + "w": 77.66666666666666, + "h": 20.33333333333303, + "y": 6717.333333333333, + "x": 162.33333333333334 + }, + "role": "AXButton", + "identifier": "eager.297", + "value": null, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "enabled": true, + "label": "Eager 298", + "subrole": null, + "identifier": "eager.298", + "children": [], + "role": "AXButton", + "type": "Button", + "value": null, + "frame": { + "h": 20.33333333333303, + "y": 6739.666666666666, + "x": 161.66666666666666, + "w": 79.00000000000003 + }, + "traits": [ + "Button", + "Scrollable" + ] + }, + { + "role": "AXButton", + "value": null, + "enabled": true, + "children": [], + "label": "Eager 299", + "type": "Button", + "frame": { + "w": 79.00000000000003, + "h": 20.33333333333303, + "y": 6762, + "x": 161.66666666666666 + }, + "subrole": null, + "traits": [ + "Button", + "Scrollable" + ], + "identifier": "eager.299" + }, + { + "role": "AXGroup", + "value": null, + "frame": { + "y": 791, + "x": 0, + "h": 83, + "w": 402 + }, + "identifier": null, + "traits": [ + "Scrollable" + ], + "children": [], + "subrole": null, + "type": "Group", + "label": "Tab Bar", + "enabled": true + } + ] + } +} \ No newline at end of file diff --git a/skills/subtext-ios-sim/helper/testdata/describe-bench-heavy.json b/skills/subtext-ios-sim/helper/testdata/describe-bench-heavy.json new file mode 100644 index 0000000..32f2751 --- /dev/null +++ b/skills/subtext-ios-sim/helper/testdata/describe-bench-heavy.json @@ -0,0 +1,305 @@ +{ + "screen": { + "h": 874, + "w": 402 + }, + "root": { + "frame": { + "y": 0, + "w": 402, + "h": 874, + "x": 0 + }, + "type": "Application", + "enabled": true, + "value": null, + "traits": [ + "None" + ], + "children": [ + { + "identifier": null, + "frame": { + "w": 100.66666666666666, + "h": 40.66666666666667, + "x": 15.999999999999993, + "y": 119.66666666666667 + }, + "value": null, + "traits": [ + "Header", + "Scrollable" + ], + "enabled": true, + "label": "Heavy", + "type": "Heading", + "children": [], + "subrole": null, + "role": "AXHeading" + }, + { + "frame": { + "w": 402, + "x": 0, + "y": 0, + "h": 874 + }, + "subrole": null, + "role": "AXGroup", + "label": null, + "type": "Group", + "enabled": true, + "children": [ + { + "subrole": null, + "role": "AXStaticText", + "enabled": true, + "label": "last:none n:0", + "type": "StaticText", + "identifier": "lastTapped", + "traits": [ + "StaticText", + "Scrollable" + ], + "children": [], + "value": null, + "frame": { + "x": 16, + "h": 52, + "y": 168, + "w": 370 + } + }, + { + "role": "AXButton", + "value": null, + "children": [], + "type": "Button", + "frame": { + "w": 370, + "h": 66.66666666666669, + "x": 16, + "y": 255 + }, + "traits": [ + "Button", + "Scrollable" + ], + "label": "Row 0, Subtitle 0, $0.00", + "subrole": null, + "enabled": true, + "identifier": "row.0.buy-row.0.fav" + }, + { + "subrole": null, + "label": "Row 1, Subtitle 1, $1.00", + "identifier": "row.1.buy-row.1.fav", + "enabled": true, + "role": "AXButton", + "type": "Button", + "value": null, + "frame": { + "y": 321.66666666666663, + "w": 370, + "h": 66.66666666666669, + "x": 16 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [] + }, + { + "type": "Button", + "enabled": true, + "children": [], + "frame": { + "y": 388.33333333333337, + "w": 370, + "h": 66.66666666666669, + "x": 16 + }, + "role": "AXButton", + "identifier": "row.2.buy-row.2.fav", + "subrole": null, + "label": "Row 2, Subtitle 2, $2.00", + "traits": [ + "Button", + "Scrollable" + ], + "value": null + }, + { + "enabled": true, + "subrole": null, + "identifier": "row.3.buy-row.3.fav", + "traits": [ + "Button", + "Scrollable" + ], + "label": "Row 3, Subtitle 3, $3.00", + "frame": { + "y": 455, + "w": 370, + "h": 66.66666666666663, + "x": 16 + }, + "value": null, + "children": [], + "role": "AXButton", + "type": "Button" + }, + { + "value": null, + "identifier": "row.4.buy-row.4.fav", + "children": [], + "role": "AXButton", + "label": "Row 4, Subtitle 4, $4.00", + "frame": { + "x": 16, + "y": 521.6666666666667, + "w": 370, + "h": 66.66666666666663 + }, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button", + "enabled": true + }, + { + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "value": null, + "frame": { + "h": 66.66666666666663, + "x": 16, + "w": 370, + "y": 588.3333333333333 + }, + "enabled": true, + "label": "Row 5, Subtitle 5, $5.00", + "children": [], + "role": "AXButton", + "identifier": "row.5.buy-row.5.fav" + }, + { + "value": null, + "subrole": null, + "role": "AXButton", + "label": "Row 6, Subtitle 6, $6.00", + "traits": [ + "Button", + "Scrollable" + ], + "type": "Button", + "children": [], + "identifier": "row.6.buy-row.6.fav", + "frame": { + "y": 655, + "x": 16, + "h": 66.66666666666663, + "w": 370 + }, + "enabled": true + }, + { + "enabled": true, + "subrole": null, + "value": null, + "type": "Button", + "children": [], + "identifier": "row.7.buy-row.7.fav", + "label": "Row 7, Subtitle 7, $7.00", + "traits": [ + "Button", + "Scrollable" + ], + "frame": { + "w": 370, + "h": 66.66666666666663, + "y": 721.6666666666666, + "x": 16 + }, + "role": "AXButton" + }, + { + "frame": { + "w": 370, + "y": 788.3333333333334, + "x": 16, + "h": 66.66666666666663 + }, + "traits": [ + "Button", + "Scrollable" + ], + "children": [], + "role": "AXButton", + "value": null, + "enabled": true, + "type": "Button", + "label": "Row 8, Subtitle 8, $8.00", + "identifier": "row.8.buy-row.8.fav", + "subrole": null + }, + { + "children": [], + "role": "AXButton", + "frame": { + "y": 855, + "h": 66.66666666666663, + "w": 370, + "x": 16 + }, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "type": "Button", + "identifier": "row.9.buy-row.9.fav", + "enabled": true, + "label": "Row 9, Subtitle 9, $9.00", + "value": null + } + ], + "value": null, + "identifier": null, + "traits": [ + "Scrollable" + ] + }, + { + "enabled": true, + "value": null, + "frame": { + "x": 0, + "h": 83, + "y": 791, + "w": 402 + }, + "role": "AXGroup", + "subrole": null, + "type": "Group", + "identifier": null, + "traits": [ + "Scrollable" + ], + "label": "Tab Bar", + "children": [] + } + ], + "role": "AXApplication", + "identifier": null, + "subrole": null, + "label": "Bench" + } +} \ No newline at end of file diff --git a/skills/subtext-ios-sim/helper/testdata/describe-bench.json b/skills/subtext-ios-sim/helper/testdata/describe-bench.json new file mode 100644 index 0000000..02b8d4d --- /dev/null +++ b/skills/subtext-ios-sim/helper/testdata/describe-bench.json @@ -0,0 +1,167 @@ +{ + "screen": { + "h": 874, + "w": 402 + }, + "root": { + "children": [ + { + "label": "Simple screen", + "identifier": null, + "frame": { + "h": 33.666666666666686, + "w": 169.66666666666663, + "y": 314.1666666666667, + "x": 116.16666666666667 + }, + "role": "AXStaticText", + "subrole": null, + "type": "StaticText", + "children": [], + "value": null, + "enabled": true, + "traits": [ + "StaticText", + "Scrollable" + ] + }, + { + "frame": { + "y": 363.83333333333337, + "x": 151.83333333333334, + "h": 20.333333333333314, + "w": 98.33333333333334 + }, + "role": "AXStaticText", + "subrole": null, + "identifier": "lastTapped", + "type": "StaticText", + "label": "last:none n:0", + "value": null, + "traits": [ + "StaticText", + "Scrollable" + ], + "enabled": true, + "children": [] + }, + { + "role": "AXButton", + "type": "Button", + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "value": null, + "identifier": "simple.primary", + "frame": { + "w": 58.66666666666666, + "h": 20.333333333333314, + "y": 400.3333333333333, + "x": 171.66666666666666 + }, + "enabled": true, + "label": "Primary", + "children": [] + }, + { + "frame": { + "w": 82, + "x": 160, + "h": 20.333333333333314, + "y": 436.66666666666663 + }, + "enabled": true, + "children": [], + "type": "Button", + "value": null, + "traits": [ + "Button", + "Scrollable" + ], + "subrole": null, + "label": "Secondary", + "identifier": "simple.secondary", + "role": "AXButton" + }, + { + "role": "AXTextField", + "subrole": null, + "identifier": "simple.email", + "type": "TextField", + "value": "Email", + "frame": { + "y": 473, + "h": 22, + "w": 370, + "x": 16 + }, + "traits": [ + "TextEntry", + "Scrollable", + "TextOperationsAvailable" + ], + "children": [], + "enabled": true, + "label": null + }, + { + "value": "1", + "subrole": "AXSwitch", + "role": "AXCheckBox", + "traits": [ + "Button", + "Scrollable", + "Toggle" + ], + "enabled": true, + "children": [], + "frame": { + "x": 16, + "y": 511, + "h": 27.66668701171875, + "w": 370 + }, + "type": "CheckBox", + "label": "Notifications", + "identifier": "simple.toggle" + }, + { + "frame": { + "x": 0, + "h": 83, + "y": 791, + "w": 402 + }, + "value": null, + "children": [], + "label": "Tab Bar", + "identifier": null, + "traits": [ + "Scrollable" + ], + "role": "AXGroup", + "subrole": null, + "type": "Group", + "enabled": true + } + ], + "value": null, + "role": "AXApplication", + "identifier": null, + "enabled": true, + "subrole": null, + "type": "Application", + "label": "Bench", + "frame": { + "x": 0, + "w": 402, + "h": 874, + "y": 0 + }, + "traits": [ + "None" + ] + } +} \ No newline at end of file diff --git a/skills/subtext-ios-sim/lib/BRIDGE.md b/skills/subtext-ios-sim/lib/BRIDGE.md new file mode 100644 index 0000000..c605e8e --- /dev/null +++ b/skills/subtext-ios-sim/lib/BRIDGE.md @@ -0,0 +1,62 @@ +# iOS tree ↔ sightmap bridge + +`normalize.mjs` turns a helper `describe` result into a tree of short-tag +elements. `sightmap-bridge.mjs` layers a `.sightmap/` corpus onto it, renders it +like `sightmap snapshot`, resolves ids and component queries, and writes +`.snap` + `.snap.tree.json` files that sightmap's offline tools read. + +## Decision: our own matcher (b), with the corpus loaded by `sightmap export` + +Checked against sightmap v0.32.0: + +- **No offline command annotates a tree.** `snapshot --json` writes `component` + and `props` per node, but only for a live Chrome page. Offline, `explain --snap + FILE '*'` prints the owning component per node, `coverage` prints only + aggregates, and `sel-check` counts one selector. +- **Offline tools ignore view components.** `explain --snap` and `coverage` + match only globals, even when the `.snap` header names a view or the snap + sits under `snapshots//`. iOS screens have no URL to route on, so most + of a corpus would go unmatched. +- **No properties offline.** `explain` gives no extracted props, so queries + like `ProductRow[title="Row 3"]` can't be resolved from it. +- **`sightmap export` emits the corpus as JSON.** Child selectors come + pre-flattened with their ancestors, `$ref`s are expanded, and `parentChain` and + `memory` are included. That removes YAML parsing and nesting rules from Node. + +So we match in Node (`selector.mjs`: tag, `*`, `#id`, `[a]`, `= ^= $= *= ~= |=` +with an optional `i`, descendant and `>` combinators, selector lists, `:not`, +`:is`, `:where`, `:has`). We extract properties (`text`, `attr=`, `Child.prop`, +`exists:Child`) and apply the live snapshot's rules: view components before +globals, first match wins, and conflicts are reported. Live `sightmap snapshot` +behaves the same way, which we verified with a probe page. + +**We stay compatible with sightmap.** The tests write our tree as +`.snap.tree.json`, run `sightmap explain --snap` over a globals-only parity +corpus (`test/fixtures/sightmap-parity/`), and require the same component on +every node across four recorded trees. Run them with `SIGHTMAP_BIN=… node --test +test/*.test.mjs`. When sightmap's matcher changes, those tests show where we +differ. + +## Element model + +| ComponentNode field | From | +|---|---| +| `element.tag`, `role` | normalized type: `button`, `text`, `textfield`, `switch`, `tab`, `cell`, `image`, `other`, … (subrole wins when present: `AXSwitch` → `switch`, `AXTabButton` → `tab`) | +| `element.attrs.identifier` | `identifier` (AXUniqueId) | +| `element.attrs.text`, `name` | `label` | +| `element.attrs.value`, `value` | `value` | +| `element.id` / `attrs.id` | `identifier`, only when it is a valid CSS identifier (`#lastTapped`) | +| `attrs.disabled` | present when `enabled` is false | +| `text` | `label`, or the children's text joined (innerText analog) | +| `bounds` | `frame`, rounded to integers (sightmap requires ints) | +| `isVisible` / `inViewport` | frame intersects the screen / its center is on screen | +| `isIgnored` | label-, identifier- and value-less `other` groups with 2+ children | + +Unlabeled single-child and empty `other` wrappers are removed during +normalization, so `>` depends on that collapsing. Prefer descendant selectors. + +## Views + +No URL means no routes. `annotate(…, {view})` picks a view explicitly. +Otherwise the view is inferred as the one whose own (non-global) components +match the most nodes, and it's rendered as `[View: X] (inferred)`. diff --git a/skills/subtext-ios-sim/lib/cli.mjs b/skills/subtext-ios-sim/lib/cli.mjs new file mode 100644 index 0000000..d06b32e --- /dev/null +++ b/skills/subtext-ios-sim/lib/cli.mjs @@ -0,0 +1,444 @@ +import { spawn } from 'node:child_process'; +import { closeSync, existsSync, mkdirSync, openSync, readFileSync } from 'node:fs'; +import { join, relative, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { SimError, toWire } from './errors.mjs'; +import { call, findSightmapDir, liveSession, pidAlive, readSession, requireSession } from './client.mjs'; +import { runDaemon, SESSION_FILE } from './daemon.mjs'; +import { CACHE_ROOT } from './sightmap-cli.mjs'; +import { buildHelper } from './helper-client.mjs'; +import * as simctl from './simctl.mjs'; +import { latestRunPath, readRun, RUNS_DIR } from './runlog.mjs'; + +const BIN = fileURLToPath(new URL('../bin/subtext-sim', import.meta.url)); + +const BOOLEAN_FLAGS = new Set(['detach', 'json', 'include-offscreen', 'clear', 'help', 'shutdown', 'fix', 'follow']); +const VALUE_FLAGS = new Set(['device', 'app', 'bundle-id', 'view', 'out', 'tree-out', 'in', 'until', 'max', 'limit', 'grep', 'level', 'duration', 'sightmap-dir', 'label', 'since', 'run']); + +export const USAGE = `subtext-sim: drive an app in the local iOS Simulator (sightmap-browser style) + +Session + doctor [--fix] check prerequisites; builds the helper + start [--device NAME|UDID] [--app PATH] [--bundle-id ID] [--detach] [--label L] + boot, enable accessibility, install, launch, run the daemon + status session health + stop [--shutdown] stop the daemon and the app (--shutdown: the simulator too) + launch | relaunch foreground the app | restart it (re-enables accessibility) + +Observe + snapshot [--view V] [--include-offscreen] [--out F] [--tree-out F] [--json] + screenshot --out F.png + console list [--limit N] [--grep S] [--level L[,L]] levels: debug info default error fault stdout stderr + console get + session-url Fullstory session URL from the app's logs + +Run log (.sightmap/.sim-runs/.jsonl: one line per action, for a consumer agent) + run new [--label L] start a new run (start also begins one) + run current the current run's id and path + run tail [--follow] [--since SEQ] [--run ID] print run entries; --follow keeps streaming + +Act (targets: snapshot id, accessibility identifier, or component query) + click + fill [--clear] + scroll [--in ] [--until ] [--max N] + swipe [--duration S] + tap-xy + type + key return delete tab escape space up down left right … + button home lock siri side apple_pay volume_up volume_down + openurl + +Global: --json (machine-readable output), --sightmap-dir DIR (default: nearest .sightmap/)`; + +export function parseArgs(argv) { + const flags = {}; + const positionals = []; + let i = 0; + for (; i < argv.length; i++) { + const arg = argv[i]; + if (arg === '--') { + positionals.push(...argv.slice(i + 1)); + break; + } + if (arg.startsWith('--') && arg.length > 2) { + const eq = arg.indexOf('='); + const name = eq === -1 ? arg.slice(2) : arg.slice(2, eq); + if (BOOLEAN_FLAGS.has(name)) { + if (eq !== -1) throw new SimError('usage', `--${name} takes no value`); + flags[name] = true; + } else if (VALUE_FLAGS.has(name)) { + const value = eq === -1 ? argv[++i] : arg.slice(eq + 1); + if (value === undefined) throw new SimError('usage', `--${name} needs a value`); + flags[name] = value; + } else { + throw new SimError('usage', `unknown flag --${name}`); + } + } else if (arg === '-h') { + flags.help = true; + } else { + positionals.push(arg); + } + } + const [command = 'help', ...rest] = positionals; + return { command, positionals: rest, flags }; +} + +const num = (value, name) => { + const n = Number(value); + if (value === undefined || value === '' || !Number.isFinite(n)) throw new SimError('usage', `${name} must be a number, got ${JSON.stringify(value)}`); + return n; +}; + +const need = (positionals, count, usage) => { + if (positionals.length < count) throw new SimError('usage', `usage: subtext-sim ${usage}`); +}; + +/** + * Maps a parsed command line to a daemon request `{cmd, args}`, or null for + * commands the CLI handles itself (help, doctor, start, stop, status). + */ +export function toRequest({ command, positionals: p, flags }, cwd = process.cwd()) { + const abs = (f) => (f ? resolve(cwd, f) : undefined); + switch (command) { + case 'snapshot': + return { cmd: 'snapshot', args: { view: flags.view, includeOffscreen: !!flags['include-offscreen'], out: abs(flags.out), treeOut: abs(flags['tree-out']), json: !!flags.json } }; + case 'click': + need(p, 1, 'click '); + return { cmd: 'click', args: { target: p[0] } }; + case 'fill': + need(p, 2, 'fill [--clear]'); + return { cmd: 'fill', args: { target: p[0], text: p[1], clear: !!flags.clear } }; + case 'scroll': + need(p, 1, 'scroll [--in ] [--until ]'); + if (!['up', 'down', 'left', 'right'].includes(p[0])) throw new SimError('usage', `scroll direction must be up, down, left or right, not ${JSON.stringify(p[0])}`); + return { cmd: 'scroll', args: { direction: p[0], within: flags.in, until: flags.until, ...(flags.max ? { max: num(flags.max, '--max') } : {}) } }; + case 'swipe': + need(p, 4, 'swipe [--duration S]'); + return { cmd: 'swipe', args: { x1: num(p[0], 'x1'), y1: num(p[1], 'y1'), x2: num(p[2], 'x2'), y2: num(p[3], 'y2'), ...(flags.duration ? { duration: num(flags.duration, '--duration') } : {}) } }; + case 'tap-xy': + need(p, 2, 'tap-xy '); + return { cmd: 'tapXY', args: { x: num(p[0], 'x'), y: num(p[1], 'y') } }; + case 'type': + need(p, 1, 'type '); + return { cmd: 'type', args: { text: p[0] } }; + case 'key': + need(p, 1, 'key '); + return { cmd: 'key', args: { key: p[0] } }; + case 'button': + need(p, 1, 'button '); + return { cmd: 'button', args: { name: p[0] } }; + case 'openurl': + need(p, 1, 'openurl '); + return { cmd: 'openurl', args: { url: p[0] } }; + case 'screenshot': + if (!flags.out) throw new SimError('usage', 'usage: subtext-sim screenshot --out F.png'); + return { cmd: 'screenshot', args: { out: abs(flags.out) } }; + case 'launch': + return { cmd: 'launch', args: {} }; + case 'relaunch': + return { cmd: 'relaunch', args: {} }; + case 'session-url': + return { cmd: 'sessionUrl', args: {} }; + case 'console': { + const [sub, index] = p; + if (sub === 'list') return { cmd: 'consoleList', args: { limit: flags.limit ? num(flags.limit, '--limit') : 50, grep: flags.grep, level: flags.level } }; + if (sub === 'get') { + need(p, 2, 'console get '); + return { cmd: 'consoleGet', args: { index: num(index, 'index') } }; + } + throw new SimError('usage', 'usage: subtext-sim console list [--limit N] [--grep S] [--level L] | console get '); + } + case 'run': { + const [sub] = p; + if (sub === 'new') return { cmd: 'runNew', args: { label: flags.label } }; + if (sub === 'current') return { cmd: 'runCurrent', args: {} }; + if (sub === 'tail') return null; + throw new SimError('usage', 'usage: subtext-sim run new [--label L] | run current | run tail [--follow] [--since SEQ]'); + } + case 'help': case 'doctor': case 'start': case 'stop': case 'status': + return null; + default: + throw new SimError('usage', `unknown command ${JSON.stringify(command)}`); + } +} + +const consoleLine = (e) => { + const where = e.subsystem ? ` ${e.subsystem}${e.category ? `:${e.category}` : ''}` : ''; + return `#${e.index} [--] ${e.level.padEnd(7)}${where} ${e.message.split('\n')[0]}`; +}; + +/** Human output for each daemon command, in the shape of `sightmap browser`. */ +export function formatResult(cmd, r) { + switch (cmd) { + case 'snapshot': return r.text; + case 'click': return `✓ clicked ${r.target} at (${r.x}, ${r.y})`; + case 'fill': return `✓ filled ${r.target} → value ${JSON.stringify(r.value)}`; + case 'scroll': + if (r.found) return `✓ ${r.found} is on screen after ${r.scrolls} scroll(s) ${r.direction}; snapshot ids refreshed, act on it by id ${r.id} or by the same query`; + return `✓ scrolled ${r.direction}; take a new snapshot`; + case 'swipe': return `✓ swiped (${r.x1}, ${r.y1}) → (${r.x2}, ${r.y2})`; + case 'tapXY': return `✓ tapped (${r.x}, ${r.y})`; + case 'type': return `✓ typed ${JSON.stringify(r.text)}`; + case 'key': return `✓ pressed ${r.key}`; + case 'button': return `✓ pressed ${r.name}`; + case 'openurl': return `✓ opened ${r.url}`; + case 'screenshot': return `✓ wrote ${r.out}`; + case 'launch': return `✓ launched ${r.bundleId} (pid ${r.pid})`; + case 'relaunch': case 'restart': + return `✓ relaunched ${r.bundleId} (pid ${r.pid})${r.ready === false ? '; accessibility tree not readable yet, snapshot again shortly' : ''}`; + case 'sessionUrl': return r.url; + case 'runNew': case 'runCurrent': + return r.run ? `${r.run}\n${r.path}` : '(no run yet)'; + case 'consoleList': + if (!r.entries.length) return '(no console entries match)'; + return [...r.entries.map(consoleLine), ...(r.total > r.entries.length ? [`(${r.total - r.entries.length} earlier; raise --limit)`] : [])].join('\n'); + case 'consoleGet': + return [`#${r.index} [--] ${r.level}`, `time: ${r.time}`, `source: ${r.source}`, + ...(r.subsystem ? [`subsystem: ${r.subsystem}${r.category ? ` (${r.category})` : ''}`] : []), '', r.message].join('\n'); + default: return JSON.stringify(r, null, 2); + } +} + +function formatStatus(s, sessionFile) { + return [ + `✓ session running (daemon pid ${s.pid}, since ${s.startedAt})`, + ` device: ${s.deviceName ?? '?'} (${s.udid})`, + ` app: ${s.bundleId}${s.appPid ? ` (pid ${s.appPid})` : ''}${s.appPath ? ` ${s.appPath}` : ''}`, + ` helper: ${s.helper.alive ? 'running' : 'idle'}${s.helper.restarts ? `, restarted ${s.helper.restarts}x` : ''}`, + ` console: ${s.logs.entries} entries${s.logs.streaming ? '' : ' (log stream not running)'}`, + ...(s.sessionUrl ? [` fullstory: ${s.sessionUrl}`] : []), + ` session: ${sessionFile}`, + ...(s.run ? [` run: ${s.run.run} (${s.run.seq} entries)`] : []), + ].join('\n'); +} + +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +async function prepareDevice(flags, out) { + const device = simctl.pickDevice(await simctl.listDevices(), flags.device); + if (device.state !== 'Booted') { + out(`booting ${device.name} (${device.udid})…`); + await simctl.boot(device.udid); + } + if (!(await simctl.accessibilityEnabled(device.udid))) { + // The accessibility server reads this only at boot, so the first enable needs one reboot. + out(`enabling app accessibility on ${device.name}; rebooting it once…`); + await simctl.enableAccessibility(device.udid); + await simctl.shutdown(device.udid); + await simctl.boot(device.udid); + } + return device; +} + +async function resolveApp(flags, udid) { + const appPath = flags.app ? resolve(flags.app) : null; + if (appPath && !existsSync(appPath)) throw new SimError('bad_app', `${appPath} does not exist`, 'build the app for the simulator and pass the .app path'); + const bundleId = flags['bundle-id'] ?? (appPath ? await simctl.plistValue(appPath, 'CFBundleIdentifier') : null); + if (!bundleId) throw new SimError('usage', 'start needs --app PATH or --bundle-id ID'); + if (appPath) await simctl.install(udid, appPath); + const installed = appPath ?? await simctl.appContainer(udid, bundleId); + const processName = await simctl.plistValue(installed, 'CFBundleExecutable'); + return { appPath, bundleId, processName }; +} + +async function waitForDaemon(sightmapDir, child, logFile) { + let exited = null; + child.on('exit', (code) => { exited = code ?? 1; }); + const deadline = Date.now() + 90_000; + while (Date.now() < deadline) { + if (exited !== null) { + const tail = existsSync(logFile) ? readFileSync(logFile, 'utf8').trim().split('\n').filter((l) => !l.includes('helper:')).at(-1) : ''; + throw new SimError('daemon_failed', `the daemon exited during startup: ${tail}`, `see ${logFile}`); + } + const session = readSession(sightmapDir); + if (session?.pid === child.pid) return session; + await sleep(150); + } + throw new SimError('daemon_timeout', 'the daemon did not start within 90 s', `see ${logFile}`); +} + +async function start(flags, sightmapDir, out) { + const existing = await liveSession(sightmapDir); + if (existing) { + if (flags.device && flags.device !== existing.udid && flags.device !== existing.deviceName) { + throw new SimError('session_running', `a session is already running on ${existing.deviceName} (${existing.udid})`, 'run `subtext-sim stop` first'); + } + const r = await call(existing.socket, 'restart', { appPath: flags.app ? resolve(flags.app) : undefined }); + return { cmd: 'restart', result: r }; + } + + const t0 = Date.now(); + const [helperPath, device] = await Promise.all([buildHelper(), prepareDevice(flags, out)]); + const app = await resolveApp(flags, device.udid); + const runDir = join(CACHE_ROOT, 'sessions', device.udid); + mkdirSync(runDir, { recursive: true }); + const config = { + udid: device.udid, deviceName: `${device.name}, iOS ${device.runtimeVersion}`, ...app, sightmapDir, helperPath, runDir, runLabel: flags.runLabel, + }; + + let session; + let foreground = null; + if (flags.detach) { + const logFile = join(runDir, 'daemon.log'); + const fd = openSync(logFile, 'w'); + const child = spawn(process.execPath, [BIN, '__daemon', JSON.stringify(config)], { detached: true, stdio: ['ignore', fd, fd] }); + closeSync(fd); + session = await waitForDaemon(sightmapDir, child, logFile); + child.unref(); + } else { + foreground = await runDaemon(config); + session = foreground.record; + } + const ready = await call(session.socket, 'ready', {}).catch((err) => ({ ready: false, error: err })); + return { cmd: 'start', result: { ...session, sessionFile: join(sightmapDir, SESSION_FILE), ready, seconds: (Date.now() - t0) / 1000 }, foreground }; +} + +function formatStart(r, cwd) { + const lines = [ + `✓ simulator session started in ${r.seconds.toFixed(1)} s`, + ` device: ${r.deviceName} (${r.udid})`, + ` app: ${r.bundleId}${r.appPath ? ` ${r.appPath}` : ''}`, + ` daemon: pid ${r.pid}; session file ${relative(cwd, r.sessionFile) || r.sessionFile}`, + ` run: ${r.run} (\`subtext-sim run tail --follow\` to follow)`, + ]; + if (r.ready?.error) lines.push(` ⚠ accessibility tree not readable yet: ${r.ready.error.message}${r.ready.error.hint ? `. Next: ${r.ready.error.hint}` : ''}`); + else if (r.ready && !r.ready.ready) lines.push(' ⚠ the app shows no interactive elements yet; snapshot again shortly'); + lines.push(' next: subtext-sim snapshot'); + return lines.join('\n'); +} + +export function formatRunEntry(e) { + if (e.action === 'session') return `${e.seq} ${e.ts} session ${e.url}`; + const t = e.target; + const target = t ? ` → ${t.id} ${t.component ? `[${t.component}]` : t.role}${t.identifier ? ` ${t.identifier}` : ''}${t.text ? ` ${JSON.stringify(t.text)}` : ''}` : ''; + const args = Object.entries(e.args ?? {}).filter(([k]) => k !== 'target').map(([k, v]) => `${k}=${JSON.stringify(v)}`).join(' '); + return `${e.seq} ${e.ts} ${e.ok ? '✓' : '✗'} ${e.action}${args ? ` ${args}` : ''}${target}${e.view ? ` (view ${e.view})` : ''}${e.error ? ` error: ${e.error.message.split('\n')[0]}` : ''}`; +} + +async function runTail(flags, sightmapDir, json) { + let path; + if (flags.run) path = join(sightmapDir, RUNS_DIR, flags.run.endsWith('.jsonl') ? flags.run : `${flags.run}.jsonl`); + else { + const session = await liveSession(sightmapDir); + path = session ? (await call(session.socket, 'runCurrent')).path : latestRunPath(sightmapDir); + } + if (!path || !existsSync(path)) throw new SimError('not_found', 'no run log found', 'run `subtext-sim start --detach` (or `subtext-sim run new`) first'); + let since = flags.since ? num(flags.since, '--since') : 0; + const emit = () => { + for (const e of readRun(path, { since })) { + console.log(json ? JSON.stringify(e) : formatRunEntry(e)); + since = e.seq; + } + }; + emit(); + if (!flags.follow) return 0; + await new Promise((resolveFollow) => { + process.on('SIGINT', resolveFollow); + process.on('SIGTERM', resolveFollow); + setInterval(emit, 500); + }); + return 0; +} + +function printError(err, json) { + const wire = toWire(err); + if (json) { + console.log(JSON.stringify({ ok: false, error: wire })); + return; + } + const [first, ...rest] = wire.message.split('\n'); + console.error(`error: ${first}${wire.hint ? `. Next: ${wire.hint}` : ''}`); + if (rest.length) console.error(rest.join('\n')); +} + +export async function main(argv = process.argv.slice(2), { cwd = process.cwd() } = {}) { + if (argv[0] === '__daemon') { + const config = JSON.parse(argv[1]); + let stopped; + const done = new Promise((r) => { stopped = r; }); + const { stop } = await runDaemon({ ...config, onStopped: () => stopped() }); + const onSignal = () => stop().then(() => stopped()); + process.on('SIGTERM', onSignal); + process.on('SIGINT', onSignal); + await done; + process.exit(0); + } + + let parsed; + try { + parsed = parseArgs(argv); + } catch (err) { + printError(err, argv.includes('--json')); + return 2; + } + const { command, flags } = parsed; + const json = !!flags.json; + const print = (cmd, result) => console.log(json ? JSON.stringify({ ok: true, result }) : formatResult(cmd, result)); + + try { + if (command === 'help' || flags.help) { + console.log(USAGE); + return 0; + } + if (command === 'doctor') { + const { doctor } = await import('./doctor.mjs'); + const checks = await doctor({ fix: !!flags.fix }); + return checks.some((c) => c.ok === false) ? 1 : 0; + } + const sightmapDir = findSightmapDir(cwd, flags['sightmap-dir']); + if (command === 'start') { + const { cmd, result, foreground } = await start({ ...flags, runLabel: flags.label }, sightmapDir, (m) => { if (!json) console.error(m); }); + if (json) console.log(JSON.stringify({ ok: true, result: { ...result, ready: result.ready?.error ? { ready: false, error: toWire(result.ready.error) } : result.ready } })); + else console.log(cmd === 'restart' ? formatResult('restart', result) : formatStart(result, cwd)); + if (foreground) { + if (!json) console.error('daemon running in the foreground; Ctrl-C or `subtext-sim stop` ends it (use --detach in scripts)'); + await new Promise((resolveStop) => { + const end = () => foreground.stop().then(resolveStop); + process.on('SIGINT', end); + process.on('SIGTERM', end); + const timer = setInterval(() => { if (!existsSync(join(sightmapDir, SESSION_FILE))) { clearInterval(timer); resolveStop(); } }, 500); + }); + } + return 0; + } + if (command === 'status') { + const session = await liveSession(sightmapDir); + if (!session) { + if (json) console.log(JSON.stringify({ ok: true, result: { running: false } })); + else console.log('✗ no running session. Next: run `subtext-sim start --detach`'); + return 1; + } + const s = await call(session.socket, 'status'); + if (json) console.log(JSON.stringify({ ok: true, result: { running: true, ...s } })); + else console.log(formatStatus(s, relative(cwd, session.file) || session.file)); + return 0; + } + if (command === 'stop') { + const session = readSession(sightmapDir); + if (!session || !pidAlive(session.pid)) { + print('stop', { stopped: false }); + if (!json) console.log('(no running session)'); + return 0; + } + await call(session.socket, 'stop', { shutdown: !!flags.shutdown }); + for (let i = 0; i < 50 && pidAlive(session.pid); i++) await sleep(100); + if (json) console.log(JSON.stringify({ ok: true, result: { stopped: true, shutdown: !!flags.shutdown } })); + else console.log(`✓ stopped${flags.shutdown ? '; simulator shut down' : ''}`); + return 0; + } + + const request = toRequest(parsed, cwd); + if (!request && command === 'run') return await runTail(flags, sightmapDir, json); + const session = await requireSession(sightmapDir); + const result = await call(session.socket, request.cmd, request.args); + print(request.cmd, result); + if (request.cmd === 'snapshot' && result.coverage?.interactive === 0) { + console.error('error: 0 interactive elements; the app may still be loading. Next: wait a moment and snapshot again'); + return 1; + } + return 0; + } catch (err) { + printError(err, json); + return err.code === 'usage' ? 2 : 1; + } +} diff --git a/skills/subtext-ios-sim/lib/client.mjs b/skills/subtext-ios-sim/lib/client.mjs new file mode 100644 index 0000000..169bd8c --- /dev/null +++ b/skills/subtext-ios-sim/lib/client.mjs @@ -0,0 +1,84 @@ +import { existsSync, readFileSync, rmSync } from 'node:fs'; +import { createConnection } from 'node:net'; +import { dirname, join, resolve } from 'node:path'; +import { SimError, fromWire } from './errors.mjs'; +import { SESSION_FILE } from './daemon.mjs'; + +/** The nearest `.sightmap/` at or above `cwd`, else `/.sightmap` (created by `start`). */ +export function findSightmapDir(cwd = process.cwd(), explicit) { + if (explicit) return resolve(cwd, explicit); + for (let dir = resolve(cwd); ; dir = dirname(dir)) { + if (existsSync(join(dir, '.sightmap'))) return join(dir, '.sightmap'); + if (dirname(dir) === dir) return join(resolve(cwd), '.sightmap'); + } +} + +export function readSession(sightmapDir) { + const file = join(sightmapDir, SESSION_FILE); + if (!existsSync(file)) return null; + try { + return { ...JSON.parse(readFileSync(file, 'utf8')), file }; + } catch { + return null; + } +} + +export function pidAlive(pid) { + try { + process.kill(pid, 0); + return true; + } catch (err) { + return err.code === 'EPERM'; + } +} + +export function call(socketPath, cmd, args = {}, { timeoutMs = 120_000 } = {}) { + return new Promise((resolvePromise, reject) => { + const socket = createConnection(socketPath); + let data = ''; + const timer = setTimeout(() => { + socket.destroy(); + reject(new SimError('daemon_timeout', `the daemon did not answer ${cmd} within ${Math.round(timeoutMs / 1000)} s`, 'check `subtext-sim status`; if it is stuck, `subtext-sim stop` then `subtext-sim start --detach`')); + }, timeoutMs); + socket.on('connect', () => socket.write(`${JSON.stringify({ id: 1, cmd, args })}\n`)); + socket.on('data', (chunk) => { data += chunk; }); + socket.on('error', (err) => { + clearTimeout(timer); + reject(new SimError('daemon_unreachable', `cannot reach the subtext-sim daemon at ${socketPath} (${err.code ?? err.message})`)); + }); + socket.on('end', () => { + clearTimeout(timer); + let msg; + try { + msg = JSON.parse(data.trim().split('\n')[0]); + } catch { + reject(new SimError('daemon_unreachable', 'the daemon closed the connection without answering')); + return; + } + if (msg.ok) resolvePromise(msg.result); + else reject(fromWire(msg.error)); + }); + }); +} + +/** The live session for `sightmapDir`; removes a stale session file whose daemon is gone. */ +export async function liveSession(sightmapDir) { + const session = readSession(sightmapDir); + if (!session) return null; + if (pidAlive(session.pid)) { + try { + await call(session.socket, 'ping', {}, { timeoutMs: 5000 }); + return session; + } catch { + // A live pid that does not answer is treated as stale; fall through. + } + } + rmSync(session.file, { force: true }); + return null; +} + +export async function requireSession(sightmapDir) { + const session = await liveSession(sightmapDir); + if (!session) throw new SimError('no_session', `no running subtext-sim session (no live ${join(sightmapDir, SESSION_FILE)})`); + return session; +} diff --git a/skills/subtext-ios-sim/lib/corpus.mjs b/skills/subtext-ios-sim/lib/corpus.mjs new file mode 100644 index 0000000..93477f1 --- /dev/null +++ b/skills/subtext-ios-sim/lib/corpus.mjs @@ -0,0 +1,53 @@ +import { execFileSync } from 'node:child_process'; +import { existsSync, readFileSync } from 'node:fs'; + +export function sightmapBin(explicit) { + return explicit || process.env.SIGHTMAP_BIN || 'sightmap'; +} + +// `sightmap export` emits the canonical Corpus wire: children are flattened into +// components whose selectors already carry their ancestors' selectors, `$ref`s +// are expanded, and view components are listed separately from globals. +export function loadCorpus(sightmapDir, { bin } = {}) { + if (!existsSync(sightmapDir)) return { globals: [], views: [], memory: [] }; + let out; + try { + out = execFileSync(sightmapBin(bin), ['export', '--sightmap-dir', sightmapDir], { + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'pipe'], + maxBuffer: 64 * 1024 * 1024, + }); + } catch (err) { + const detail = err.code === 'ENOENT' + ? `sightmap CLI not found (${sightmapBin(bin)}); install it with: npm install -g @sightmap/sightmap` + : (err.stderr || err.message).toString().trim(); + const wrapped = new Error(`could not load .sightmap corpus from ${sightmapDir}: ${detail}`); + wrapped.code = 'corpus_unavailable'; + throw wrapped; + } + return parseCorpus(JSON.parse(out)); +} + +export function loadCorpusFile(path) { + return parseCorpus(JSON.parse(readFileSync(path, 'utf8'))); +} + +export function parseCorpus(wire) { + const component = (c) => ({ + name: c.name, + selectors: c.selectors ?? [], + properties: c.properties ?? [], + parentChain: c.parentChain ?? [], + memory: c.memory ?? [], + }); + return { + globals: (wire.globals ?? []).map(component), + views: (wire.views ?? []).map((v) => ({ + name: v.name, + route: v.route ?? null, + memory: v.memory ?? [], + components: (v.components ?? []).map(component), + })), + memory: wire.memory ?? [], + }; +} diff --git a/skills/subtext-ios-sim/lib/daemon.mjs b/skills/subtext-ios-sim/lib/daemon.mjs new file mode 100644 index 0000000..7afd16e --- /dev/null +++ b/skills/subtext-ios-sim/lib/daemon.mjs @@ -0,0 +1,567 @@ +import { createHash } from 'node:crypto'; +import { existsSync, mkdirSync, rmSync, unlinkSync, writeFileSync } from 'node:fs'; +import { createServer } from 'node:net'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve as resolvePath } from 'node:path'; +import { createInterface } from 'node:readline'; +import { BridgeError, annotate, centerOf, render, resolve as resolveQuery, toComponentTree, writeSnap } from './sightmap-bridge.mjs'; +import { normalize } from './normalize.mjs'; +import { SimError, toWire } from './errors.mjs'; +import { ConsoleBuffer, LogCollector } from './logs.mjs'; +import { HelperClient } from './helper-client.mjs'; +import { CorpusCache } from './sightmap-cli.mjs'; +import * as realSimctl from './simctl.mjs'; +import { RunLog, targetOf } from './runlog.mjs'; + +export const SESSION_FILE = '.sim-session'; + +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); +const quote = (s) => JSON.stringify(String(s)); + +export function socketPathFor(sightmapDir, udid) { + const hash = createHash('sha256').update(`${resolvePath(sightmapDir)}\0${udid}`).digest('hex').slice(0, 12); + return join(tmpdir(), `subtext-sim-${hash}.sock`); +} + +/** One line per node, in the snapshot's own notation. */ +export function describeNode(node) { + let body; + if (node.component) { + const entries = Object.entries(node.props ?? {}).sort(([a], [b]) => a.localeCompare(b)); + const props = entries.map(([k, v]) => `${k}=${quote(v)}`); + const showLabel = node.label && !entries.some(([, v]) => v === node.label); + body = `[${[node.component, ...props, ...(showLabel ? [quote(node.label)] : [])].join(' ')}]`; + } else { + body = node.tag; + if (node.identifier) body += `[identifier=${quote(node.identifier)}]`; + if (node.label) body += ` ${quote(node.label)}`; + } + return `${node.id} ${body}`; +} + +function bridgeToSim(err) { + if (err instanceof BridgeError) return new SimError(err.code, err.message); + return err; +} + +/** + * Resolves a target against a snapshot: a snapshot id, an exact accessibility + * identifier, or a component query (sightmap-bridge `resolve`). + */ +export function resolveTarget(annotated, target) { + const text = String(target).trim(); + if (!/^\d+$/.test(text) && /^[^\s[\]"'#]+$/.test(text)) { + const byIdentifier = annotated.nodes.filter((n) => n.identifier === text); + if (byIdentifier.length === 1) return byIdentifier[0]; + if (byIdentifier.length > 1) { + const lines = byIdentifier.map(describeNode); + throw new SimError('ambiguous', `identifier ${quote(text)} is on ${byIdentifier.length} elements; use a snapshot id or \`${byIdentifier[0].tag}[identifier=${quote(text)}]#N\`:\n ${lines.join('\n ')}`, 'pick one by snapshot id'); + } + } + try { + return resolveQuery(annotated, text); + } catch (err) { + throw bridgeToSim(err); + } +} + +const isAncestor = (a, node) => { + for (let p = node.parent; p; p = p.parent) if (p === a) return true; + return false; +}; + +const OBSTRUCTING = new Set(['tabbar', 'keyboard', 'navbar', 'toolbar']); +const OBSTRUCTING_LABELS = new Set(['Tab Bar', 'Toolbar']); + +const obstructs = (o) => OBSTRUCTING.has(o.tag) || (o.tag === 'other' && OBSTRUCTING_LABELS.has(o.label)); +const barName = (o) => (OBSTRUCTING.has(o.tag) ? o.tag : o.label.toLowerCase()); + +/** Throws unless the node's center is on screen, uncovered by bars or the keyboard, and enabled. */ +export function assertActionable(annotated, node, target) { + const { screen } = annotated; + const desc = describeNode(node); + if (!node.visible || !node.onScreen) { + const { x, y, w, h } = node.frame; + const dir = y + h / 2 > screen.h ? 'down' : y + h / 2 < 0 ? 'up' : x + w / 2 > screen.w ? 'right' : 'left'; + throw new SimError('offscreen', `${desc} is off-screen (center ${Math.round(x + w / 2)},${Math.round(y + h / 2)} on a ${screen.w}x${screen.h} screen); refusing to tap`, + `run \`subtext-sim scroll ${dir} --until ${quote(target)}\`, then retry`); + } + const c = centerOf(node); + const cover = annotated.nodes.find((o) => obstructs(o) && o.visible && o !== node && !isAncestor(o, node) && !isAncestor(node, o) + && c.x >= o.frame.x && c.x <= o.frame.x + o.frame.w && c.y >= o.frame.y && c.y <= o.frame.y + o.frame.h); + if (cover) { + const hint = cover.tag === 'keyboard' + ? 'dismiss the keyboard (`subtext-sim key return`) or scroll, then retry' + : `scroll it clear of the ${barName(cover)} (\`subtext-sim scroll down\` or \`up\`), then retry`; + throw new SimError('covered', `${desc} is covered by the ${barName(cover)} (${describeNode(cover)})`, hint); + } + if (!node.enabled) throw new SimError('disabled', `${desc} is disabled`); +} + +/** Finds the element a previous snapshot's node refers to in a fresh snapshot. */ +export function relocate(fresh, node, target) { + if (!/^\d+$/.test(String(target).trim())) return resolveTarget(fresh, target); + const same = fresh.nodes.filter((n) => n.tag === node.tag && n.identifier === node.identifier && n.label === node.label); + if (!same.length) throw new SimError('stale_target', `element ${describeNode(node)} is no longer on screen`); + const c = centerOf(node); + const dist = (n) => Math.hypot(centerOf(n).x - c.x, centerOf(n).y - c.y); + return same.sort((a, b) => dist(a) - dist(b))[0]; +} + +function scrollVector(direction, region) { + const { x, y, w, h } = region; + const cx = x + w / 2; + const cy = y + h / 2; + switch (direction) { + case 'down': return { x1: cx, y1: y + h * 0.8, x2: cx, y2: y + h * 0.2 }; + case 'up': return { x1: cx, y1: y + h * 0.2, x2: cx, y2: y + h * 0.8 }; + case 'right': return { x1: x + w * 0.8, y1: cy, x2: x + w * 0.2, y2: cy }; + case 'left': return { x1: x + w * 0.2, y1: cy, x2: x + w * 0.8, y2: cy }; + default: throw new SimError('usage', `scroll direction must be up, down, left or right, not ${quote(direction)}`); + } +} + +function clip(frame, screen) { + const x = Math.max(0, frame.x); + const y = Math.max(0, frame.y); + return { x, y, w: Math.min(screen.w, frame.x + frame.w) - x, h: Math.min(screen.h, frame.y + frame.h) - y }; +} + +const SECURE = new Set(['securetextfield']); + +// Points per second. Faster swipes fling the list, which can carry a target +// past the screen between two reads; this speed stops where the finger stops. +const SCROLL_SPEED = 300; + +const ACTION_BY_CMD = { + snapshot: 'snapshot', click: 'click', fill: 'fill', scroll: 'scroll', swipe: 'swipe', tapXY: 'tap-xy', + button: 'button', key: 'key', type: 'type', openurl: 'openurl', launch: 'launch', relaunch: 'launch', restart: 'launch', +}; + +/** The command engine behind the socket. Dependencies are injectable so tests can fake the helper and simctl. */ +export class SimSession { + constructor({ udid, deviceName = null, bundleId, appPath = null, processName, sightmapDir, runDir, helper, simctl = realSimctl, logs = null, buffer = new ConsoleBuffer(), corpus = new CorpusCache(sightmapDir), log = () => {}, timing = {} }) { + Object.assign(this, { udid, deviceName, bundleId, appPath, processName, sightmapDir, runDir, helper, simctl, logs, buffer, corpus, log }); + this.timing = { settleMs: 350, scrollSettleMs: 150, fillReadbackMs: 2000, readyTimeoutMs: 20_000, ...timing }; + this.last = null; + this.lastView = null; + this.lastTarget = null; + this.appPid = null; + this.runLog = new RunLog(sightmapDir); + this.buffer.onSessionUrl = (url) => this.runLog.run && this.runLog.append({ action: 'session', url }); + this.startedAt = new Date().toISOString(); + } + + get stdoutPath() { + return this.runDir ? join(this.runDir, 'app-stdout.log') : undefined; + } + + get stderrPath() { + return this.runDir ? join(this.runDir, 'app-stderr.log') : undefined; + } + + async describe() { + return this.helper.request('describe', { udid: this.udid }); + } + + async fresh({ view } = {}) { + try { + const snap = annotate(normalize(await this.describe()), this.corpus.get(), { view }); + this.lastView = snap.view; + return snap; + } catch (err) { + throw bridgeToSim(err); + } + } + + /** Re-describes until two reads agree on every element's frame, so a list's momentum has stopped. */ + async settled({ timeoutMs = 4000 } = {}) { + const layout = (snap) => snap.nodes.map((n) => `${n.tag}|${n.identifier}|${n.label}|${Math.round(n.frame.x)},${Math.round(n.frame.y)}`).join('\n'); + let snap = await this.fresh(); + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + await sleep(150); + const next = await this.fresh(); + if (layout(next) === layout(snap)) return next; + snap = next; + } + return snap; + } + + input(cmd, args) { + return this.helper.request(cmd, { udid: this.udid, ...args }); + } + + async launchApp({ terminate = true } = {}) { + if (terminate) await this.simctl.enableAccessibility(this.udid); + this.appPid = await this.simctl.launch(this.udid, this.bundleId, { stdout: this.stdoutPath, stderr: this.stderrPath, terminate }); + this.last = null; + return { bundleId: this.bundleId, pid: this.appPid }; + } + + /** Waits until the accessibility tree is readable, which lags the launch by a moment. */ + async waitReady() { + const deadline = Date.now() + this.timing.readyTimeoutMs; + let lastErr; + while (Date.now() < deadline) { + try { + const snap = await this.fresh(); + if (snap.coverage.interactive > 0) return { ready: true, interactive: snap.coverage.interactive }; + } catch (err) { + lastErr = err; + if (!['ax_error', 'ax_unavailable', 'ax_disabled', 'helper_crashed'].includes(err.code)) throw err; + } + await sleep(500); + } + if (lastErr) throw lastErr; + return { ready: false, interactive: 0 }; + } + + async handle(cmd, args = {}) { + const fn = this.commands[cmd]; + if (!fn) throw new SimError('unknown_cmd', `unknown daemon command ${quote(cmd)}`); + const action = ACTION_BY_CMD[cmd]; + if (!action) return fn.call(this, args); + this.lastTarget = null; + const logArgs = Object.fromEntries(Object.entries(args).filter(([k, v]) => v !== undefined && k !== 'json')); + const redact = () => { + if (cmd === 'fill' && SECURE.has(this.lastTarget?.tag)) logArgs.text = ''; + }; + try { + const result = await fn.call(this, args); + redact(); + this.logStep(action, logArgs, true); + return result; + } catch (err) { + redact(); + this.logStep(action, logArgs, false, err); + throw err; + } + } + + logStep(action, args, ok, err) { + try { + this.runLog.append({ + action, args, target: targetOf(this.lastTarget), view: this.lastView, ok, + ...(err ? { error: { code: err.code ?? 'internal', message: err.message } } : {}), + }); + } catch (e) { + this.log(`run log write failed: ${e.message}`); + } + } + + commands = { + ping() { + return { pong: true }; + }, + + ready() { + return this.waitReady(); + }, + + runNew({ label }) { + const run = this.runLog.begin(label); + if (this.buffer.sessionUrl) this.runLog.append({ action: 'session', url: this.buffer.sessionUrl }); + return run; + }, + + runCurrent() { + return { run: this.runLog.run, path: this.runLog.path, label: this.runLog.label, seq: this.runLog.seq }; + }, + + status() { + return { + udid: this.udid, + deviceName: this.deviceName, + bundleId: this.bundleId, + appPath: this.appPath, + appPid: this.appPid, + pid: process.pid, + startedAt: this.startedAt, + helper: { alive: this.helper.alive, restarts: this.helper.restarts }, + logs: { streaming: this.logs?.streaming ?? false, entries: this.buffer.nextIndex }, + sessionUrl: this.buffer.sessionUrl, + run: { run: this.runLog.run, path: this.runLog.path, seq: this.runLog.seq }, + }; + }, + + async snapshot({ view, includeOffscreen = false, out, treeOut, json = false }) { + const snap = await this.fresh({ view }); + this.last = snap; + const text = includeOffscreen ? render(snap, { includeOffscreen: true }) : snap.text; + if (out) writeSnap({ ...snap, text }, out); + if (treeOut) { + mkdirSync(dirname(treeOut), { recursive: true }); + writeFileSync(treeOut, JSON.stringify({ tree: toComponentTree(snap, { annotations: true }) })); + } + const result = { text, view: snap.view, viewInferred: snap.viewInferred, coverage: snap.coverage, conflicts: snap.conflicts }; + if (json) result.tree = toComponentTree(snap, { annotations: true }); + return result; + }, + + async click({ target }) { + const { node, ...tapped } = await this.tap(target); + return tapped; + }, + + async fill({ target, text, clear = false }) { + if (typeof text !== 'string') throw new SimError('usage', 'fill needs the text to type'); + const tapped = await this.tap(target); + await sleep(this.timing.settleMs); + if (clear) { + const field = relocate(await this.fresh(), tapped.node, target); + const count = (field.value ?? '').length; + for (let i = 0; i < count; i++) await this.input('key', { key: 'forward_delete' }); + for (let i = 0; i < count; i++) await this.input('key', { key: 'delete' }); + } + if (text) await this.input('type', { text }); + const stuck = (n) => SECURE.has(n.tag) || !text || (n.value ?? '').toLowerCase().includes(text.toLowerCase()); + let after; + const deadline = Date.now() + this.timing.fillReadbackMs; + do { + await sleep(this.timing.settleMs); + after = relocate(await this.fresh(), tapped.node, target); + } while (!stuck(after) && Date.now() < deadline); + const value = after.value ?? ''; + if (!stuck(after)) { + throw new SimError('fill_mismatch', `typed ${quote(text)} but ${describeNode(after)} shows ${quote(value)}`, + 'retry with --clear, or check with `subtext-sim snapshot` that the field took focus'); + } + return { target: tapped.target, x: tapped.x, y: tapped.y, value: SECURE.has(after.tag) ? `(${value.length} secure characters)` : value }; + }, + + async scroll({ direction, within, until, max = 20 }) { + let snap = await this.fresh(); + let region = { x: 0, y: 0, w: snap.screen.w, h: snap.screen.h }; + if (within) { + const container = resolveTarget(snap, within); + this.lastTarget = container; + region = clip(container.frame, snap.screen); + } + const vector = scrollVector(direction, region); + const distance = Math.hypot(vector.x2 - vector.x1, vector.y2 - vector.y1); + const duration = Math.max(0.3, Math.round((distance / SCROLL_SPEED) * 100) / 100); + const swipe = () => this.input('swipe', { ...vector, duration }); + if (!until) { + await swipe(); + await sleep(this.timing.scrollSettleMs); + await this.settled(); + this.last = null; + return { direction, scrolls: 1 }; + } + let previous = null; + for (let i = 0; i <= max; i++) { + if (i > 0) snap = await this.settled(); + let found = null; + try { + found = resolveTarget(snap, until); + assertActionable(snap, found, until); + this.lastTarget = found; + this.last = snap; + return { direction, scrolls: i, found: describeNode(found), id: found.id }; + } catch (err) { + if (!['not_found', 'offscreen', 'covered'].includes(err.code)) throw err; + } + if (previous !== null && previous === snap.text) { + this.last = snap; + throw new SimError('not_found', `${quote(until)} not found; reached the end after scrolling ${direction} ${i} time(s)`, + `try \`subtext-sim scroll ${direction === 'down' ? 'up' : direction === 'up' ? 'down' : direction === 'left' ? 'right' : 'left'} --until ${quote(until)}\``); + } + previous = snap.text; + if (i === max) break; + await swipe(); + await sleep(this.timing.scrollSettleMs); + } + this.last = snap; + throw new SimError('not_found', `${quote(until)} not found after ${max} scroll(s) ${direction}`, 'raise --max, or check the query with `subtext-sim snapshot`'); + }, + + async swipe({ x1, y1, x2, y2, duration }) { + await this.input('swipe', { x1, y1, x2, y2, ...(duration ? { duration } : {}) }); + this.last = null; + return { x1, y1, x2, y2 }; + }, + + async tapXY({ x, y }) { + await this.input('tap', { x, y }); + return { x, y }; + }, + + async button({ name }) { + await this.input('button', { name }); + return { name }; + }, + + async key({ key }) { + await this.input('key', { key }); + return { key }; + }, + + async type({ text }) { + await this.input('type', { text }); + return { text }; + }, + + async openurl({ url }) { + await this.simctl.openUrl(this.udid, url); + this.last = null; + return { url }; + }, + + async screenshot({ out }) { + await this.simctl.screenshot(this.udid, out); + return { out }; + }, + + launch() { + return this.launchApp({ terminate: false }); + }, + + async relaunch() { + const r = await this.launchApp({ terminate: true }); + return { ...r, ...(await this.waitReady().catch(() => ({ ready: false }))) }; + }, + + async restart({ appPath }) { + if (appPath) { + await this.simctl.install(this.udid, appPath); + this.appPath = appPath; + } + return this.commands.relaunch.call(this); + }, + + consoleList({ limit = 50, grep, level }) { + return this.buffer.list({ limit, grep, level }); + }, + + consoleGet({ index }) { + const entry = this.buffer.get(Number(index)); + if (!entry) throw new SimError('not_found', `no console entry #${index} (the buffer holds the most recent ${this.buffer.capacity})`, 'run `subtext-sim console list` for current indexes'); + return entry; + }, + + sessionUrl() { + if (!this.buffer.sessionUrl) { + throw new SimError('no_session_url', 'the app has not logged a Fullstory session URL', + 'the app needs the Fullstory iOS SDK (for the sample app: FS_ORG_ID= fixtures/SampleApp/build.sh), then `subtext-sim relaunch`'); + } + return { url: this.buffer.sessionUrl }; + }, + }; + + /** Re-describes, re-resolves the target, checks it is tappable, and taps its center. */ + async tap(target) { + const numeric = /^\d+$/.test(String(target).trim()); + let base = this.last; + if (numeric && !base) throw new SimError('not_found', `no snapshot to resolve id ${target} against`, 'run `subtext-sim snapshot` first'); + const fresh = await this.fresh({ view: base?.viewInferred === false ? base.view ?? undefined : undefined }); + base ??= fresh; + let node; + try { + node = numeric ? relocate(fresh, resolveTarget(base, target), target) : resolveTarget(fresh, target); + } catch (err) { + if (err.code !== 'not_found' || numeric) throw err; + throw new SimError('not_found', `${err.message}; long lists expose only the rows near the screen`, + `if it is further down a list, run \`subtext-sim scroll down --until ${quote(target)}\`; otherwise \`subtext-sim snapshot\``); + } + this.lastTarget = node; + assertActionable(fresh, node, target); + const { x, y } = centerOf(node); + await this.input('tap', { x, y }); + return { target: describeNode(node), x: Math.round(x), y: Math.round(y), node }; + } + + close() { + this.helper.close(); + this.logs?.close(); + } +} + +/** + * Runs the daemon: owns the helper, the console collector and the last + * snapshot, serves the CLI over a Unix socket, and records the session in + * `.sightmap/.sim-session`. Resolves with `{ session, stop }` once serving. + */ +export async function runDaemon(config, { simctl = realSimctl, helperArgs = [], withLogs = true, log = (m) => console.error(`[${new Date().toISOString()}] ${m}`) } = {}) { + const { udid, deviceName, bundleId, appPath, processName, sightmapDir, helperPath, runDir, launch = true } = config; + mkdirSync(sightmapDir, { recursive: true }); + if (runDir) mkdirSync(runDir, { recursive: true }); + const helper = new HelperClient(helperPath, helperArgs, { log }); + const buffer = new ConsoleBuffer(); + const session = new SimSession({ udid, deviceName, bundleId, appPath, processName, sightmapDir, runDir, helper, simctl, buffer, log, timing: config.timing }); + if (runDir) for (const p of [session.stdoutPath, session.stderrPath]) writeFileSync(p, ''); + if (withLogs) { + session.logs = new LogCollector({ udid, processName, stdoutPath: session.stdoutPath, stderrPath: session.stderrPath, buffer, log }); + session.logs.start(); + await sleep(500); + } + helper.start(); + const hello = await helper.request('hello'); + log(`helper ${hello.version} ready`); + session.runLog.begin(config.runLabel); + if (launch) { + await session.launchApp({ terminate: true }); + log(`launched ${bundleId} (pid ${session.appPid})`); + } + session.runLog.append({ action: 'start', args: { udid, deviceName, bundleId, appPath: appPath ?? null }, target: null, view: null, ok: true }); + + const socketPath = config.socketPath ?? socketPathFor(sightmapDir, udid); + if (existsSync(socketPath)) unlinkSync(socketPath); + const sessionFile = join(sightmapDir, SESSION_FILE); + let queue = Promise.resolve(); + let stopping = null; + + const stop = async ({ shutdown = false, terminateApp = true } = {}) => { + stopping ??= (async () => { + log('stopping'); + if (terminateApp) await simctl.terminate(udid, bundleId); + if (shutdown) await simctl.shutdown(udid); + session.close(); + server.close(); + rmSync(socketPath, { force: true }); + rmSync(sessionFile, { force: true }); + })(); + return stopping; + }; + + const server = createServer((socket) => { + socket.on('error', () => {}); + const rl = createInterface({ input: socket }); + rl.once('line', async (line) => { + let req; + try { + req = JSON.parse(line); + } catch { + socket.end(`${JSON.stringify({ ok: false, error: { code: 'bad_request', message: 'request is not JSON' } })}\n`); + return; + } + const reply = (msg) => socket.end(`${JSON.stringify({ id: req.id, ...msg })}\n`); + if (req.cmd === 'stop') { + reply({ ok: true, result: { stopped: true, shutdown: !!req.args?.shutdown } }); + await stop({ shutdown: !!req.args?.shutdown }); + config.onStopped?.(); + return; + } + const run = () => session.handle(req.cmd, req.args ?? {}); + const task = req.cmd === 'status' || req.cmd === 'ping' ? run() : (queue = queue.then(run, run)); + try { + reply({ ok: true, result: await task }); + } catch (err) { + if (!(err instanceof SimError)) log(`internal error in ${req.cmd}: ${err.stack ?? err}`); + reply({ ok: false, error: toWire(err) }); + } + }); + }); + + await new Promise((resolve, reject) => { + server.once('error', reject); + server.listen(socketPath, resolve); + }); + const record = { + socket: socketPath, udid, deviceName, bundleId, appPath: appPath ?? null, pid: process.pid, + startedAt: session.startedAt, run: session.runLog.run, runDir: runDir ?? null, logFile: runDir ? join(runDir, 'daemon.log') : null, + }; + writeFileSync(sessionFile, `${JSON.stringify(record, null, 2)}\n`); + log(`serving on ${socketPath}`); + return { session, stop, record, socketPath }; +} diff --git a/skills/subtext-ios-sim/lib/doctor.mjs b/skills/subtext-ios-sim/lib/doctor.mjs new file mode 100644 index 0000000..8058ec9 --- /dev/null +++ b/skills/subtext-ios-sim/lib/doctor.mjs @@ -0,0 +1,154 @@ +import { execFile } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; + +const SKILL_DIR = join(dirname(fileURLToPath(import.meta.url)), '..'); +const HELPER_BUILD = join(SKILL_DIR, 'helper', 'build.sh'); +const SIGHTMAP_PACKAGE = '@sightmap/sightmap@0.32.0'; +const MIN_XCODE = 26; +const MIN_MACOS = 15; +const MIN_NODE = 18; +const XCODE_DOWNLOAD = 'Install Xcode 26 or later from the Mac App Store or https://developer.apple.com/download/applications/, open it once, then re-run doctor'; + +function run(cmd, args, { timeout = 60_000 } = {}) { + return new Promise((resolve) => { + execFile(cmd, args, { timeout, maxBuffer: 64 * 1024 * 1024 }, (error, stdout, stderr) => { + resolve({ ok: !error, code: error?.code ?? 0, stdout: String(stdout).trim(), stderr: String(stderr).trim() }); + }); + }); +} + +const majorOf = (version) => Number.parseInt(String(version).split('.')[0], 10); +const pass = (name, detail) => ({ name, ok: true, detail, fix: null }); +const fail = (name, detail, fix, autoFix) => ({ name, ok: false, detail, fix, autoFix }); + +async function checkXcodeSelect() { + const r = await run('xcode-select', ['-p']); + if (!r.ok) return fail('xcode-select', 'xcode-select -p failed: no developer directory', 'xcode-select --install, then ' + XCODE_DOWNLOAD); + if (!/\.app\/Contents\/Developer$/.test(r.stdout)) { + const xcodes = (await run('/bin/sh', ['-c', 'ls -d /Applications/Xcode*.app 2>/dev/null'])).stdout.split('\n').filter(Boolean); + const fix = xcodes.length + ? `sudo xcode-select -s ${xcodes.at(-1)}/Contents/Developer` + : XCODE_DOWNLOAD + ', then sudo xcode-select -s /Applications/Xcode.app/Contents/Developer'; + return fail('xcode-select', `points at ${r.stdout} (Command Line Tools), not an Xcode.app`, fix); + } + return pass('xcode-select', r.stdout); +} + +async function checkXcodeVersion() { + const r = await run('xcodebuild', ['-version']); + const version = r.stdout.match(/^Xcode (\S+)/m)?.[1]; + if (!r.ok || !version) return fail('xcode-version', `xcodebuild -version failed: ${r.stderr || r.stdout}`, XCODE_DOWNLOAD); + if (majorOf(version) < MIN_XCODE) return fail('xcode-version', `Xcode ${version}; need ${MIN_XCODE}+`, XCODE_DOWNLOAD); + const firstLaunch = await run('xcodebuild', ['-checkFirstLaunchStatus']); + if (!firstLaunch.ok) return fail('xcode-version', `Xcode ${version}, but first-launch setup is incomplete`, 'sudo xcodebuild -runFirstLaunch'); + return pass('xcode-version', `Xcode ${version}`); +} + +async function checkSimulators() { + const runtimes = await run('xcrun', ['simctl', 'list', 'runtimes', '-j']); + if (!runtimes.ok) return [fail('simctl', `xcrun simctl failed: ${runtimes.stderr}`, 'Fix xcode-select first, then: sudo xcodebuild -runFirstLaunch')]; + const ios = JSON.parse(runtimes.stdout).runtimes.filter((rt) => rt.platform === 'iOS' && rt.isAvailable); + if (!ios.length) { + return [fail('ios-runtime', 'no available iOS Simulator runtime', 'xcodebuild -downloadPlatform iOS', ['xcodebuild', ['-downloadPlatform', 'iOS']])]; + } + const newest = ios.at(-1); + const checks = [pass('ios-runtime', ios.map((rt) => `iOS ${rt.version}`).join(', '))]; + + const devices = JSON.parse((await run('xcrun', ['simctl', 'list', 'devices', 'available', '-j'])).stdout).devices; + const iosDevices = ios.flatMap((rt) => devices[rt.identifier] ?? []); + if (iosDevices.length) { + const booted = iosDevices.filter((d) => d.state === 'Booted').map((d) => d.name); + checks.push(pass('ios-device', `${iosDevices.length} iOS simulators${booted.length ? `; booted: ${booted.join(', ')}` : ''}`)); + } else { + const type = newest.supportedDeviceTypes.filter((t) => t.productFamily === 'iPhone').at(0); + const args = ['simctl', 'create', type?.name ?? 'iPhone', type?.identifier ?? '', newest.identifier]; + checks.push(fail('ios-device', `no simulator devices for iOS ${newest.version}`, `xcrun ${args.map((a) => (a.includes(' ') ? `"${a}"` : a)).join(' ')}`, type && ['xcrun', args])); + } + return checks; +} + +async function checkSwift() { + const r = await run('swift', ['--version']); + const line = (r.stdout || r.stderr).split('\n').find((l) => /Swift version/.test(l)); + if (!r.ok || !line) return fail('swift', 'swift --version failed', 'Fix xcode-select first; swift ships with Xcode'); + return pass('swift', line.trim()); +} + +async function checkArch() { + const r = await run('sysctl', ['-n', 'hw.optional.arm64']); + if (r.stdout !== '1') return fail('apple-silicon', `this Mac is ${process.arch}`, 'An Apple Silicon (arm64) Mac is required; the simulator helper ships arm64-only frameworks'); + return pass('apple-silicon', 'arm64'); +} + +async function checkMacOS() { + const r = await run('sw_vers', ['-productVersion']); + if (majorOf(r.stdout) < MIN_MACOS) return fail('macos', `macOS ${r.stdout}; need ${MIN_MACOS}+`, `Update to macOS ${MIN_MACOS} or later (System Settings > General > Software Update)`); + return pass('macos', `macOS ${r.stdout}`); +} + +async function checkNode() { + const version = process.versions.node; + if (majorOf(version) < MIN_NODE) return fail('node', `Node ${version}; need ${MIN_NODE}+`, 'brew install node (or nvm install --lts)'); + return pass('node', `Node ${version}`); +} + +async function checkSightmap() { + const onPath = await run('sightmap', ['version']); + if (onPath.ok) return pass('sightmap', `${onPath.stdout} (on PATH)`); + const viaNpx = await run('npx', ['-y', SIGHTMAP_PACKAGE, 'version'], { timeout: 180_000 }); + if (viaNpx.ok) return pass('sightmap', `${viaNpx.stdout} (via npx ${SIGHTMAP_PACKAGE})`); + return fail('sightmap', `sightmap not on PATH and npx ${SIGHTMAP_PACKAGE} failed: ${viaNpx.stderr.split('\n').at(-1)}`, `npm install -g ${SIGHTMAP_PACKAGE}`, ['npm', ['install', '-g', SIGHTMAP_PACKAGE]]); +} + +async function checkHelper() { + if (!existsSync(HELPER_BUILD)) return { name: 'helper', ok: null, detail: `pending: ${HELPER_BUILD} not present yet`, fix: null }; + const r = await run('bash', [HELPER_BUILD], { timeout: 15 * 60_000 }); + if (!r.ok) { + const tail = (r.stderr || r.stdout).split('\n').slice(-5).join('\n'); + return fail('helper', `helper build failed:\n${tail}`, `bash ${HELPER_BUILD} (fix the errors above; a newer Xcode may need a helper update)`); + } + return pass('helper', r.stdout.split('\n').at(-1) || 'built'); +} + +const ICON = { true: 'ok ', false: 'FAIL', null: 'PEND' }; + +function report(checks) { + const width = Math.max(...checks.map((c) => c.name.length)); + const lines = checks.map((c) => { + const head = `[${ICON[c.ok]}] ${c.name.padEnd(width)} ${c.detail}`; + return c.fix ? `${head}\n ${' '.repeat(width)} fix: ${c.fix}` : head; + }); + const failed = checks.filter((c) => c.ok === false).length; + const pending = checks.filter((c) => c.ok === null).length; + lines.push('', failed ? `${failed} check(s) failed.` : `All checks passed${pending ? ` (${pending} pending)` : ''}.`); + console.log(lines.join('\n')); +} + +async function runChecks() { + const prerequisites = await Promise.all([checkXcodeSelect(), checkXcodeVersion(), checkSimulators(), checkSwift(), checkArch(), checkMacOS(), checkNode(), checkSightmap()]); + return [...prerequisites.flat(), await checkHelper()]; +} + +/** Runs every check, prints a report, and returns `[{name, ok, detail, fix}]` (`ok: null` means pending). With `fix`, runs the unprivileged fixes once and re-checks. */ +export async function doctor({ fix = false } = {}) { + let checks = await runChecks(); + const fixable = checks.filter((c) => c.ok === false && c.autoFix); + if (fix && fixable.length) { + for (const c of fixable) { + const [cmd, args] = c.autoFix; + console.log(`fixing ${c.name}: ${cmd} ${args.join(' ')}`); + await run(cmd, args, { timeout: 60 * 60_000 }); + } + checks = await runChecks(); + } + checks = checks.map(({ autoFix, ...c }) => c); + report(checks); + return checks; +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + const checks = await doctor({ fix: process.argv.includes('--fix') }); + process.exitCode = checks.some((c) => c.ok === false) ? 1 : 0; +} diff --git a/skills/subtext-ios-sim/lib/errors.mjs b/skills/subtext-ios-sim/lib/errors.mjs new file mode 100644 index 0000000..5973c0e --- /dev/null +++ b/skills/subtext-ios-sim/lib/errors.mjs @@ -0,0 +1,36 @@ +export class SimError extends Error { + constructor(code, message, hint) { + super(message); + this.code = code; + this.hint = hint ?? HINTS[code] ?? null; + } +} + +const HINTS = { + no_session: 'run `subtext-sim start --detach` first', + daemon_unreachable: 'run `subtext-sim start --detach` to start a new session', + device_not_found: 'list devices with `xcrun simctl list devices available`, then pass --device', + not_booted: 'run `subtext-sim start` to boot the simulator', + ax_disabled: 'run `subtext-sim relaunch` (it re-enables app accessibility before launching)', + ax_unavailable: 'the app may still be launching; retry, or run `subtext-sim relaunch`', + ax_error: 'the app may not be in the foreground; run `subtext-sim launch`, or `subtext-sim relaunch`', + hid_error: 'retry; if it keeps failing, run `subtext-sim stop` then `subtext-sim start`', + helper_crashed: 'retry; the daemon restarts the helper automatically', + not_found: 'run `subtext-sim snapshot` to see what is on screen', + stale_target: 'run `subtext-sim snapshot` and act on the new ids', + offscreen: 'scroll it into view first, e.g. `subtext-sim scroll down --until `', + covered: 'dismiss what covers it (e.g. `subtext-sim key return` for the keyboard) or scroll, then retry', + disabled: 'the control is disabled; complete the prerequisite steps, then snapshot again', + bad_query: 'queries look like `SignInButton`, `ProductRow[title^="Product 1"]`, `button[identifier="login-submit"]`', + unknown_view: 'check the view names under .sightmap/', + corpus_unavailable: 'install the sightmap CLI (`npm install -g @sightmap/sightmap`) or set SIGHTMAP_BIN', + usage: 'run `subtext-sim help`', +}; + +export function toWire(err) { + return { code: err.code ?? 'internal', message: err.message, hint: err.hint ?? HINTS[err.code] ?? null }; +} + +export function fromWire({ code, message, hint }) { + return new SimError(code, message, hint ?? undefined); +} diff --git a/skills/subtext-ios-sim/lib/helper-client.mjs b/skills/subtext-ios-sim/lib/helper-client.mjs new file mode 100644 index 0000000..a526d59 --- /dev/null +++ b/skills/subtext-ios-sim/lib/helper-client.mjs @@ -0,0 +1,109 @@ +import { execFile, spawn } from 'node:child_process'; +import { dirname, join } from 'node:path'; +import { createInterface } from 'node:readline'; +import { fileURLToPath } from 'node:url'; +import { SimError } from './errors.mjs'; + +const HELPER_BUILD = join(dirname(fileURLToPath(import.meta.url)), '..', 'helper', 'build.sh'); + +/** Runs helper/build.sh (a no-op when already built) and returns the binary path. */ +export function buildHelper() { + return new Promise((resolve, reject) => { + execFile('bash', [HELPER_BUILD], { timeout: 15 * 60_000, maxBuffer: 64 * 1024 * 1024 }, (error, stdout, stderr) => { + if (error) { + const tail = String(stderr).trim().split('\n').at(-1); + reject(new SimError('helper_build_failed', `simulator helper build failed: ${tail}`, 'run `subtext-sim doctor` for details')); + } else resolve(String(stdout).trim().split('\n').at(-1)); + }); + }); +} + +/** + * A long-lived helper process speaking JSON lines. If the process dies, pending + * requests fail with `helper_crashed` and the next request starts a fresh one. + */ +export class HelperClient { + constructor(command, args = [], { log = () => {}, timeoutMs = 30_000 } = {}) { + this.command = command; + this.args = args; + this.log = log; + this.timeoutMs = timeoutMs; + this.nextId = 1; + this.pending = new Map(); + this.proc = null; + this.restarts = 0; + this.closed = false; + } + + get alive() { + return !!this.proc; + } + + start() { + if (this.proc || this.closed) return; + const proc = spawn(this.command, this.args, { stdio: ['pipe', 'pipe', 'pipe'] }); + this.proc = proc; + createInterface({ input: proc.stdout }).on('line', (line) => this.onLine(line)); + const verbose = !!process.env.SUBTEXT_SIM_DEBUG; + createInterface({ input: proc.stderr }).on('line', (line) => { + if (verbose || /fail|error/i.test(line)) this.log(`helper: ${line}`); + }); + proc.on('error', (err) => this.onExit(`failed to start: ${err.message}`)); + proc.on('exit', (code, signal) => this.onExit(`exited (${signal ?? code})`)); + proc.stdin.on('error', () => {}); + } + + onExit(reason) { + if (!this.proc) return; + this.proc = null; + if (!this.closed) { + this.restarts++; + this.log(`helper ${reason}; restarting on next request`); + } + for (const { reject, timer } of this.pending.values()) { + clearTimeout(timer); + reject(new SimError('helper_crashed', `simulator helper ${reason}`)); + } + this.pending.clear(); + } + + onLine(line) { + let msg; + try { + msg = JSON.parse(line); + } catch { + this.log(`helper: unparseable line ${line.slice(0, 200)}`); + return; + } + const entry = this.pending.get(msg.id); + if (!entry) return; + this.pending.delete(msg.id); + clearTimeout(entry.timer); + if (msg.ok) entry.resolve(msg.result ?? {}); + else entry.reject(new SimError(msg.error?.code ?? 'internal', msg.error?.message ?? 'helper error')); + } + + request(cmd, args = {}, { timeoutMs = this.timeoutMs } = {}) { + if (this.closed) return Promise.reject(new SimError('helper_closed', 'the helper is shut down')); + this.start(); + const id = this.nextId++; + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + this.pending.delete(id); + reject(new SimError('helper_timeout', `helper ${cmd} timed out after ${timeoutMs} ms`, 'retry; if it keeps timing out, run `subtext-sim stop` then `subtext-sim start`')); + }, timeoutMs); + this.pending.set(id, { resolve, reject, timer }); + this.proc.stdin.write(`${JSON.stringify({ id, cmd, args })}\n`); + }); + } + + close() { + this.closed = true; + const proc = this.proc; + this.onExit('closed'); + if (proc) { + proc.stdin.end(); + proc.kill(); + } + } +} diff --git a/skills/subtext-ios-sim/lib/logs.mjs b/skills/subtext-ios-sim/lib/logs.mjs new file mode 100644 index 0000000..c2f0372 --- /dev/null +++ b/skills/subtext-ios-sim/lib/logs.mjs @@ -0,0 +1,165 @@ +import { spawn } from 'node:child_process'; +import { closeSync, openSync, readSync, statSync } from 'node:fs'; +import { createInterface } from 'node:readline'; + +const LEVEL_BY_TYPE = { Default: 'default', Info: 'info', Debug: 'debug', Error: 'error', Fault: 'fault' }; + +export const LEVELS = ['debug', 'info', 'default', 'error', 'fault', 'stdout', 'stderr']; + +/** Parses one `log stream --style ndjson` line into a console entry, or null for non-log events. */ +export function parseLogLine(line) { + if (!line.startsWith('{')) return null; + let j; + try { + j = JSON.parse(line); + } catch { + return null; + } + if (j.eventType !== 'logEvent' || typeof j.eventMessage !== 'string') return null; + return { + time: j.timestamp ?? null, + level: LEVEL_BY_TYPE[j.messageType] ?? String(j.messageType ?? 'default').toLowerCase(), + source: 'os_log', + subsystem: j.subsystem || null, + category: j.category || null, + message: j.eventMessage, + }; +} + +const SESSION_URL = [/FULLSTORY_SESSION_URL\s+(\S+)/, /Beginning session:?\s*(https?:\/\/\S+)/i]; + +export function sessionUrlIn(message) { + for (const re of SESSION_URL) { + const m = re.exec(message); + if (m) return m[1]; + } + return null; +} + +/** A bounded console buffer with stable, ever-increasing indexes. */ +export class ConsoleBuffer { + constructor(capacity = 5000) { + this.capacity = capacity; + this.entries = []; + this.nextIndex = 0; + this.sessionUrl = null; + } + + push(entry) { + const full = { index: this.nextIndex++, ...entry, time: entry.time ?? new Date().toISOString() }; + this.entries.push(full); + if (this.entries.length > this.capacity) this.entries.shift(); + const url = sessionUrlIn(full.message); + if (url && url !== this.sessionUrl) { + this.sessionUrl = url; + this.onSessionUrl?.(url); + } + return full; + } + + list({ limit = 50, grep, level } = {}) { + const levels = level ? new Set(String(level).split(',').map((l) => l.trim().toLowerCase())) : null; + const needle = grep ? String(grep).toLowerCase() : null; + const hits = this.entries.filter((e) => + (!levels || levels.has(e.level)) && (!needle || e.message.toLowerCase().includes(needle) + || (e.subsystem ?? '').toLowerCase().includes(needle))); + return { total: hits.length, entries: limit > 0 ? hits.slice(-limit) : hits }; + } + + get(index) { + return this.entries.find((e) => e.index === index) ?? null; + } +} + +/** Follows a file that another process appends to (the app's redirected stdout/stderr). */ +class FileTail { + constructor(path, onLine) { + this.path = path; + this.onLine = onLine; + this.offset = 0; + this.partial = ''; + this.timer = setInterval(() => this.poll(), 200); + this.timer.unref?.(); + } + + poll() { + let size; + try { + size = statSync(this.path).size; + } catch { + return; + } + if (size < this.offset) this.offset = 0; + if (size === this.offset) return; + const fd = openSync(this.path, 'r'); + try { + const buf = Buffer.alloc(size - this.offset); + readSync(fd, buf, 0, buf.length, this.offset); + this.offset = size; + const lines = (this.partial + buf.toString('utf8')).split('\n'); + this.partial = lines.pop(); + for (const line of lines) if (line.trim()) this.onLine(line); + } finally { + closeSync(fd); + } + } + + close() { + this.poll(); + clearInterval(this.timer); + } +} + +/** + * Collects the app's console: os_log / Logger output from `log stream` (the + * app's process, minus Apple frameworks' info and default chatter) plus its + * redirected stdout and stderr. + */ +export class LogCollector { + constructor({ udid, processName, stdoutPath, stderrPath, buffer = new ConsoleBuffer(), log = () => {} }) { + Object.assign(this, { udid, processName, stdoutPath, stderrPath, buffer, log }); + this.proc = null; + this.tails = []; + this.closed = false; + } + + start() { + this.startStream(); + this.tails = [ + new FileTail(this.stdoutPath, (message) => this.buffer.push({ level: 'stdout', source: 'stdout', subsystem: null, category: null, message })), + new FileTail(this.stderrPath, (message) => this.buffer.push({ level: 'stderr', source: 'stderr', subsystem: null, category: null, message })), + ]; + } + + startStream() { + if (this.closed) return; + const predicate = `process == ${JSON.stringify(this.processName)} AND (NOT (subsystem BEGINSWITH "com.apple.") OR messageType == error OR messageType == fault)`; + const proc = spawn('xcrun', ['simctl', 'spawn', this.udid, 'log', 'stream', '--style', 'ndjson', '--level', 'info', '--predicate', predicate], { + stdio: ['ignore', 'pipe', 'pipe'], + }); + this.proc = proc; + createInterface({ input: proc.stdout }).on('line', (line) => { + const entry = parseLogLine(line); + if (entry) this.buffer.push(entry); + }); + proc.on('exit', (code) => { + if (this.proc !== proc) return; + this.proc = null; + if (!this.closed) { + this.log(`log stream exited (${code}); restarting in 1 s`); + setTimeout(() => this.startStream(), 1000).unref?.(); + } + }); + } + + get streaming() { + return !!this.proc; + } + + close() { + this.closed = true; + this.proc?.kill(); + this.proc = null; + for (const t of this.tails) t.close(); + } +} diff --git a/skills/subtext-ios-sim/lib/normalize.mjs b/skills/subtext-ios-sim/lib/normalize.mjs new file mode 100644 index 0000000..6253fa5 --- /dev/null +++ b/skills/subtext-ios-sim/lib/normalize.mjs @@ -0,0 +1,153 @@ +const TAG_BY_TYPE = { + application: 'application', + window: 'window', + group: 'other', + other: 'other', + button: 'button', + statictext: 'text', + text: 'text', + textfield: 'textfield', + securetextfield: 'securetextfield', + searchfield: 'searchfield', + textview: 'textview', + textarea: 'textview', + image: 'image', + icon: 'image', + cell: 'cell', + table: 'table', + collectionview: 'collection', + scrollarea: 'scrollview', + scrollview: 'scrollview', + heading: 'heading', + link: 'link', + checkbox: 'checkbox', + switch: 'switch', + toggle: 'switch', + radiobutton: 'radio', + slider: 'slider', + stepper: 'stepper', + incrementor: 'stepper', + segmentedcontrol: 'segmentedcontrol', + picker: 'picker', + pickerwheel: 'pickerwheel', + popupbutton: 'popup', + menu: 'menu', + menuitem: 'menuitem', + navigationbar: 'navbar', + tabbar: 'tabbar', + toolbar: 'toolbar', + alert: 'alert', + sheet: 'sheet', + keyboard: 'keyboard', + key: 'key', + progressindicator: 'progress', + activityindicator: 'progress', + webview: 'webview', +}; + +const TAG_BY_SUBROLE = { + AXSwitch: 'switch', + AXTabButton: 'tab', + AXSearchField: 'searchfield', + AXSecureTextField: 'securetextfield', +}; + +const INTERACTIVE_TAGS = new Set([ + 'button', 'link', 'cell', 'textfield', 'securetextfield', 'searchfield', 'textview', + 'checkbox', 'switch', 'radio', 'tab', 'slider', 'stepper', 'segmentedcontrol', + 'picker', 'pickerwheel', 'popup', 'menuitem', 'key', +]); + +const MAX_TEXT = 300; + +export function tagFor(type, role, subrole) { + if (subrole && TAG_BY_SUBROLE[subrole]) return TAG_BY_SUBROLE[subrole]; + const raw = String(type || role || 'other').replace(/^XCUIElementType/, '').replace(/^AX/, ''); + const key = raw.toLowerCase(); + return TAG_BY_TYPE[key] ?? key; +} + +const present = (s) => (typeof s === 'string' && s !== '' ? s : null); + +function isWrapper(raw, tag) { + return tag === 'other' && !present(raw.label) && !present(raw.identifier) && !present(raw.value); +} + +function collapse(raw, isRoot) { + const tag = tagFor(raw.type, raw.role, raw.subrole); + let children = (raw.children ?? []).flatMap((c) => collapse(c, false)); + if (!isRoot && isWrapper(raw, tag)) { + if (children.length <= 1) return children; + } + return [{ raw, tag, children }]; +} + +function intersects(f, screen) { + return f.w > 0 && f.h > 0 && f.x < screen.w && f.y < screen.h && f.x + f.w > 0 && f.y + f.h > 0; +} + +function centerOnScreen(f, screen) { + const cx = f.x + f.w / 2; + const cy = f.y + f.h / 2; + return f.w > 0 && f.h > 0 && cx >= 0 && cy >= 0 && cx <= screen.w && cy <= screen.h; +} + +function frameOf(raw) { + const f = raw.frame ?? {}; + return { x: f.x ?? 0, y: f.y ?? 0, w: f.w ?? f.width ?? 0, h: f.h ?? f.height ?? 0 }; +} + +export function normalize(describeResult) { + const rawRoot = describeResult.root; + const rootFrame = frameOf(rawRoot); + const screen = { + w: describeResult.screen?.w || rootFrame.w, + h: describeResult.screen?.h || rootFrame.h, + }; + const nodes = []; + const build = ({ raw, tag, children }, parent, nthChild) => { + const frame = frameOf(raw); + const node = { + id: nodes.length + 1, + tag, + type: raw.type ?? null, + role: raw.role ?? null, + label: present(raw.label), + identifier: present(raw.identifier), + value: present(raw.value), + enabled: raw.enabled !== false, + frame, + visible: intersects(frame, screen), + onScreen: centerOnScreen(frame, screen), + interactive: INTERACTIVE_TAGS.has(tag), + ignored: isWrapper(raw, tag), + nthChild, + text: '', + children: [], + }; + Object.defineProperty(node, 'parent', { value: parent, enumerable: false, writable: true }); + nodes.push(node); + node.children = children.map((c, i) => build(c, node, i + 1)); + node.text = node.label ?? node.children.map((c) => c.text).filter(Boolean).join(' ').slice(0, MAX_TEXT); + return node; + }; + const [top] = collapse(rawRoot, true); + const root = build(top, null, 1); + return { screen, root, nodes, byId: new Map(nodes.map((n) => [n.id, n])) }; +} + +const CSS_IDENT = /^-?[A-Za-z_][A-Za-z0-9_-]*$/; + +export function cssId(node) { + return node.identifier && CSS_IDENT.test(node.identifier) ? node.identifier : null; +} + +export function attrsOf(node) { + const attrs = {}; + if (node.identifier) attrs.identifier = node.identifier; + if (cssId(node)) attrs.id = node.identifier; + if (node.label) attrs.text = node.label; + if (node.value) attrs.value = node.value; + if (!node.enabled) attrs.disabled = ''; + return attrs; +} diff --git a/skills/subtext-ios-sim/lib/runlog.mjs b/skills/subtext-ios-sim/lib/runlog.mjs new file mode 100644 index 0000000..29c5c19 --- /dev/null +++ b/skills/subtext-ios-sim/lib/runlog.mjs @@ -0,0 +1,66 @@ +import { appendFileSync, existsSync, mkdirSync, readdirSync, readFileSync, statSync } from 'node:fs'; +import { join } from 'node:path'; + +export const RUNS_DIR = '.sim-runs'; + +export const LOGGED_ACTIONS = new Set(['start', 'snapshot', 'click', 'fill', 'scroll', 'swipe', 'tap-xy', 'button', 'key', 'type', 'openurl', 'launch']); + +const stamp = (d = new Date()) => d.toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z'); + +export function newRunId(label) { + const slug = label ? `-${String(label).toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '').slice(0, 40)}` : ''; + return `${stamp()}-${Math.random().toString(36).slice(2, 6)}${slug}`; +} + +export function targetOf(node) { + if (!node) return null; + return { id: node.id, component: node.component ?? null, identifier: node.identifier ?? null, text: node.label ?? null, role: node.tag }; +} + +/** Appends one JSON line per producer step to `.sightmap/.sim-runs/.jsonl`. */ +export class RunLog { + constructor(sightmapDir) { + this.dir = join(sightmapDir, RUNS_DIR); + this.run = null; + this.seq = 0; + this.label = null; + } + + get path() { + return this.run ? join(this.dir, `${this.run}.jsonl`) : null; + } + + begin(label) { + mkdirSync(this.dir, { recursive: true }); + this.run = newRunId(label); + this.label = label ?? null; + this.seq = 0; + return { run: this.run, path: this.path, label: this.label }; + } + + append(entry) { + if (!this.run) this.begin(); + const line = { ts: new Date().toISOString(), run: this.run, seq: ++this.seq, ...entry }; + appendFileSync(this.path, `${JSON.stringify(line)}\n`); + return line; + } +} + +export function latestRunPath(sightmapDir) { + const dir = join(sightmapDir, RUNS_DIR); + if (!existsSync(dir)) return null; + const files = readdirSync(dir).filter((f) => f.endsWith('.jsonl')).map((f) => join(dir, f)); + if (!files.length) return null; + return files.sort((a, b) => statSync(b).mtimeMs - statSync(a).mtimeMs)[0]; +} + +export function readRun(path, { since = 0 } = {}) { + if (!existsSync(path)) return []; + return readFileSync(path, 'utf8').split('\n').filter(Boolean).flatMap((line) => { + try { + return [JSON.parse(line)]; + } catch { + return []; + } + }).filter((e) => e.seq > since); +} diff --git a/skills/subtext-ios-sim/lib/selector.mjs b/skills/subtext-ios-sim/lib/selector.mjs new file mode 100644 index 0000000..5c92386 --- /dev/null +++ b/skills/subtext-ios-sim/lib/selector.mjs @@ -0,0 +1,264 @@ +import { attrsOf } from './normalize.mjs'; + +export class SelectorError extends Error { + constructor(message, selector) { + super(`${message} in selector ${JSON.stringify(selector)}`); + this.code = 'bad_selector'; + } +} + +const IDENT_CHAR = /[A-Za-z0-9_\-\u0080-￿]/; +const LOGICAL_PSEUDOS = new Set(['not', 'is', 'where', 'has']); + +class Parser { + constructor(source) { + this.s = source; + this.i = 0; + } + + fail(message) { + throw new SelectorError(`${message} at offset ${this.i}`, this.s); + } + + peek() { + return this.s[this.i]; + } + + eof() { + return this.i >= this.s.length; + } + + skipWs() { + const start = this.i; + while (!this.eof() && /\s/.test(this.peek())) this.i++; + return this.i > start; + } + + expect(ch) { + if (this.peek() !== ch) this.fail(`expected "${ch}"`); + this.i++; + } + + ident() { + let out = ''; + while (!this.eof()) { + const ch = this.peek(); + if (ch === '\\') { + out += this.s[this.i + 1] ?? ''; + this.i += 2; + } else if (IDENT_CHAR.test(ch)) { + out += ch; + this.i++; + } else break; + } + if (!out) this.fail('expected an identifier'); + return out; + } + + quoted() { + const quote = this.peek(); + this.i++; + let out = ''; + while (!this.eof() && this.peek() !== quote) { + if (this.peek() === '\\') this.i++; + out += this.s[this.i++]; + } + this.expect(quote); + return out; + } + + list(relative) { + const items = []; + for (;;) { + this.skipWs(); + items.push(this.complex(relative)); + this.skipWs(); + if (this.peek() !== ',') return items; + this.i++; + } + } + + complex(relative) { + const parts = []; + let combinator = null; + if (relative) { + combinator = ' '; + if (this.peek() === '>') { + combinator = '>'; + this.i++; + this.skipWs(); + } + } + for (;;) { + parts.push({ combinator, compound: this.compound() }); + const sawWs = this.skipWs(); + const ch = this.peek(); + if (this.eof() || ch === ',' || ch === ')') return { parts }; + if (ch === '+' || ch === '~') this.fail(`sibling combinator "${ch}" is not supported`); + if (ch === '>') { + combinator = '>'; + this.i++; + this.skipWs(); + } else if (sawWs) { + combinator = ' '; + } else this.fail(`unexpected "${ch}"`); + } + } + + compound() { + const c = { tag: null, ids: [], classes: [], attrs: [], pseudos: [] }; + if (this.peek() === '*') { + this.i++; + c.tag = '*'; + } else if (!this.eof() && IDENT_CHAR.test(this.peek())) { + c.tag = this.ident().toLowerCase(); + } + for (;;) { + const ch = this.peek(); + if (ch === '#') { + this.i++; + c.ids.push(this.ident()); + } else if (ch === '.') { + this.i++; + c.classes.push(this.ident()); + } else if (ch === '[') { + c.attrs.push(this.attr()); + } else if (ch === ':') { + c.pseudos.push(this.pseudo()); + } else break; + } + if (!c.tag && !c.ids.length && !c.classes.length && !c.attrs.length && !c.pseudos.length) { + this.fail('empty compound selector'); + } + return c; + } + + attr() { + this.expect('['); + this.skipWs(); + const name = this.ident().toLowerCase(); + this.skipWs(); + if (this.peek() === ']') { + this.i++; + return { name, op: null }; + } + const op = ['^=', '$=', '*=', '~=', '|='].find((o) => this.s.startsWith(o, this.i)) ?? (this.peek() === '=' ? '=' : null); + if (!op) this.fail('expected an attribute operator'); + this.i += op.length; + this.skipWs(); + const value = this.peek() === '"' || this.peek() === "'" ? this.quoted() : this.ident(); + this.skipWs(); + let ci = false; + if (/[iIsS]/.test(this.peek() ?? '') && !IDENT_CHAR.test(this.s[this.i + 1] ?? '')) { + ci = this.peek().toLowerCase() === 'i'; + this.i++; + this.skipWs(); + } + this.expect(']'); + return { name, op, value, ci }; + } + + pseudo() { + this.expect(':'); + const name = this.ident().toLowerCase(); + if (!LOGICAL_PSEUDOS.has(name)) this.fail(`pseudo-class ":${name}" is not supported`); + this.expect('('); + const list = this.list(name === 'has'); + this.skipWs(); + this.expect(')'); + return { name, list }; + } +} + +const cache = new Map(); + +export function parseSelector(source) { + let parsed = cache.get(source); + if (!parsed) { + const p = new Parser(source); + parsed = p.list(false); + if (!p.eof()) p.fail('trailing input'); + cache.set(source, parsed); + } + return parsed; +} + +export function compareValue(actual, op, expected, ci) { + if (actual == null) return false; + let a = String(actual); + let e = expected; + if (ci) { + a = a.toLowerCase(); + e = e.toLowerCase(); + } + switch (op) { + case '=': return a === e; + case '^=': return e !== '' && a.startsWith(e); + case '$=': return e !== '' && a.endsWith(e); + case '*=': return e !== '' && a.includes(e); + case '~=': return a.split(/\s+/).includes(e); + case '|=': return a === e || a.startsWith(`${e}-`); + default: return false; + } +} + +const attrCache = new WeakMap(); +function cachedAttrs(node) { + let attrs = attrCache.get(node); + if (!attrs) attrCache.set(node, (attrs = attrsOf(node))); + return attrs; +} + +function* descendants(node) { + for (const child of node.children) { + yield child; + yield* descendants(child); + } +} + +function matchCompound(node, c) { + if (c.tag && c.tag !== '*' && c.tag !== node.tag) return false; + if (c.classes.length) return false; + for (const id of c.ids) if (node.identifier !== id) return false; + const attrs = cachedAttrs(node); + for (const a of c.attrs) { + const actual = attrs[a.name]; + if (a.op === null ? actual === undefined : !compareValue(actual, a.op, a.value, a.ci)) return false; + } + for (const p of c.pseudos) { + if (p.name === 'not' && matchesList(node, p.list)) return false; + if ((p.name === 'is' || p.name === 'where') && !matchesList(node, p.list)) return false; + if (p.name === 'has' && !hasMatch(node, p.list)) return false; + } + return true; +} + +function hasMatch(scope, list) { + for (const d of descendants(scope)) if (matchesList(d, list, scope)) return true; + return false; +} + +function matchFrom(node, parts, k, scope) { + if (!matchCompound(node, parts[k].compound)) return false; + const { combinator } = parts[k]; + if (k === 0) { + if (!scope || combinator === null) return true; + if (combinator === '>') return node.parent === scope; + for (let a = node.parent; a; a = a.parent) if (a === scope) return true; + return false; + } + if (combinator === '>') return !!node.parent && matchFrom(node.parent, parts, k - 1, scope); + for (let a = node.parent; a; a = a.parent) { + if (scope && a === scope) return false; + if (matchFrom(a, parts, k - 1, scope)) return true; + } + return false; +} + +function matchesList(node, list, scope = null) { + return list.some((complex) => matchFrom(node, complex.parts, complex.parts.length - 1, scope)); +} + +export function matches(node, selector) { + return matchesList(node, parseSelector(selector)); +} diff --git a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs new file mode 100644 index 0000000..d015c9b --- /dev/null +++ b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs @@ -0,0 +1,389 @@ +import { mkdirSync, writeFileSync } from 'node:fs'; +import { dirname } from 'node:path'; +import { attrsOf, cssId, normalize } from './normalize.mjs'; +import { compareValue, matches } from './selector.mjs'; +import { loadCorpus } from './corpus.mjs'; + +export class BridgeError extends Error { + constructor(code, message, candidates = []) { + super(message); + this.code = code; + this.candidates = candidates; + } +} + +const quote = (s) => JSON.stringify(String(s)); + +function* descendants(node) { + for (const child of node.children) { + yield child; + yield* descendants(child); + } +} + +function firstDescendant(node, componentName) { + for (const d of descendants(node)) if (d.component === componentName) return d; + return null; +} + +function matchedNodes(nodes, def) { + return nodes.filter((n) => def.selectors.some((sel) => matches(n, sel))); +} + +function inferView(nodes, corpus) { + const globalNames = new Set(corpus.globals.map((c) => c.name)); + let best = null; + let bestScore = 0; + for (const view of corpus.views) { + const own = view.components.filter((c) => !globalNames.has(c.name)); + const hit = new Set(own.flatMap((def) => matchedNodes(nodes, def))); + if (hit.size > bestScore) { + best = view; + bestScore = hit.size; + } + } + return best; +} + +function extractProps(node, memo) { + if (memo.has(node)) return memo.get(node); + const props = {}; + memo.set(node, props); + for (const { name, extract } of node.componentDef.properties) { + const value = extractOne(node, extract ?? 'text', memo); + if (value !== undefined && value !== null && value !== '') props[name] = String(value); + } + return props; +} + +function extractOne(node, extract, memo) { + if (extract === 'text') return node.label ?? node.text; + if (extract.startsWith('attr=')) return attrsOf(node)[extract.slice(5).toLowerCase()]; + if (extract.startsWith('exists:')) return firstDescendant(node, extract.slice(7)) ? 'true' : undefined; + const path = extract.split('.'); + const prop = path.pop(); + let cur = node; + for (const name of path) { + cur = firstDescendant(cur, name); + if (!cur) return undefined; + } + return extractProps(cur, memo)[prop]; +} + +function computeCoverage(nodes) { + const counts = { interactive: 0, t1: 0, t2: 0, t3: 0 }; + for (const n of nodes) { + if (!n.interactive || !n.visible) continue; + counts.interactive++; + if (n.component) counts.t1++; + else { + let scoped = false; + for (let a = n.parent; a && !scoped; a = a.parent) scoped = !!a.component; + counts[scoped ? 't2' : 't3']++; + } + } + return counts; +} + +/** + * Layers a sightmap corpus onto a normalized tree. `sightmap` is a `.sightmap/` + * directory (exported through the sightmap CLI) or an already loaded corpus. + * `view` names the view whose components apply; without it the view is inferred + * as the one whose own components match the most nodes. + */ +export function annotate(normalized, sightmap, { view, sightmapBin, url } = {}) { + const corpus = typeof sightmap === 'string' ? loadCorpus(sightmap, { bin: sightmapBin }) : sightmap; + const { nodes } = normalized; + let activeView = null; + if (view) { + activeView = corpus.views.find((v) => v.name === view); + if (!activeView) throw new BridgeError('unknown_view', `no view named ${quote(view)} in the corpus`); + } else { + activeView = inferView(nodes, corpus); + } + + const defs = [...(activeView?.components ?? []), ...corpus.globals]; + const candidates = new Map(); + for (const def of defs) { + for (const node of matchedNodes(nodes, def)) { + const list = candidates.get(node) ?? []; + if (!list.some((d) => d.name === def.name)) list.push(def); + candidates.set(node, list); + } + } + + const conflicts = []; + for (const node of nodes) { + const list = candidates.get(node); + node.component = list ? list[0].name : null; + Object.defineProperty(node, 'componentDef', { value: list ? list[0] : null, enumerable: false, writable: true }); + if (list && list.length > 1) conflicts.push({ id: node.id, names: list.map((d) => d.name) }); + } + + const memo = new Map(); + const guide = new Map(); + for (const node of nodes) { + node.props = node.component ? extractProps(node, memo) : null; + if (node.component) guide.set(node.component, (guide.get(node.component) ?? 0) + 1); + } + + const memory = [ + ...corpus.memory.map((m) => ({ scope: null, text: m })), + ...(activeView?.memory ?? []).map((m) => ({ scope: activeView.name, text: m })), + ...defs.filter((d) => guide.has(d.name) && d.memory.length) + .filter((d, i, all) => all.findIndex((o) => o.name === d.name) === i) + .flatMap((d) => d.memory.map((m) => ({ scope: d.name, text: m }))), + ]; + + const annotated = { + ...normalized, + view: activeView?.name ?? null, + viewInferred: !view && !!activeView, + url: url ?? null, + conflicts, + guide, + memory, + coverage: computeCoverage(nodes), + }; + annotated.text = render(annotated); + return annotated; +} + +export function snapshot(describeResult, sightmap, options) { + return annotate(normalize(describeResult), sightmap, options); +} + +function nodeLabel(node, { showIdentifiers = true } = {}) { + let body; + if (node.component) { + const props = Object.entries(node.props).sort(([a], [b]) => a.localeCompare(b)); + const parts = [node.component, ...props.map(([k, v]) => `${k}=${quote(v)}`)]; + if (node.label && !props.some(([, v]) => v === node.label)) parts.push(quote(node.label)); + body = `[${parts.join(' ')}]`; + } else { + body = node.tag; + if (showIdentifiers && node.identifier) body += `[identifier=${quote(node.identifier)}]`; + if (node.label) body += ` ${quote(node.label)}`; + if (node.value) body += ` value=${quote(node.value)}`; + } + if (!node.enabled) body += ' (disabled)'; + if (!node.visible) body += ' (off-screen)'; + else if (!node.onScreen) body += ' (partly off-screen)'; + return `${node.id} ${body}`; +} + +function coverageLine({ interactive, t1, t2, t3 }) { + if (!interactive) return '0 interactive ∅'; + const pct = (n) => Math.round((100 * n) / interactive); + return `${interactive} interactive · ${t1} direct T1 (${pct(t1)}%) · ${t2} scoped T2 (${pct(t2)}%) · ${t3} orphaned T3 ${t3 ? '✗' : '✓'}`; +} + +/** Text rendering in the shape of `sightmap snapshot` output. */ +export function render(annotated, { includeOffscreen = false, showIdentifiers = true } = {}) { + const lines = []; + const where = annotated.url ? ` ${annotated.url}` : ''; + if (annotated.view) lines.push(`[View: ${annotated.view}]${annotated.viewInferred ? ' (inferred)' : ''}${where}`); + else lines.push(`[No view matched]${where}`); + lines.push(''); + + if (annotated.guide.size) { + lines.push('[Guide]'); + const entries = [...annotated.guide].sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])); + const width = Math.max(...entries.map(([name]) => name.length)); + for (const [name, count] of entries) lines.push(`${name.padEnd(width)} ${count}`); + lines.push(''); + } + if (annotated.memory.length) { + lines.push('[Memory]'); + for (const m of annotated.memory) lines.push(`- ${m.scope ? `${m.scope}: ` : ''}${m.text}`); + lines.push(''); + } + + lines.push('--- component tree ---', ''); + const hidden = { nodes: 0, interactive: 0 }; + const walk = (node, depth) => { + let childDepth = depth; + const skipped = node.ignored && !node.component; + if (!node.visible && !includeOffscreen) { + hidden.nodes++; + if (node.interactive) hidden.interactive++; + } else if (!skipped) { + lines.push(`${' '.repeat(depth)}${nodeLabel(node, { showIdentifiers })}`); + childDepth = depth + 1; + } + for (const child of node.children) walk(child, childDepth); + }; + walk(annotated.root, 0); + + if (annotated.conflicts.length) { + lines.push('', '[Conflicts]'); + for (const { id, names } of annotated.conflicts) { + const node = annotated.byId.get(id); + const desc = `${node.tag}${node.label ? ` ${quote(node.label)}` : ''}`; + lines.push(`⚠ #${id} ${desc} matched by ${names.length} components (${names.join(', ')}) — only ${names[0]} applied`); + } + } + if (hidden.nodes) { + lines.push('', '[Off-screen]', `${hidden.nodes} off-screen node(s) not shown (${hidden.interactive} interactive); scroll to reveal them. Queries still resolve them.`); + } + lines.push('', '[Coverage] (visible only)', coverageLine(annotated.coverage)); + return lines.join('\n'); +} + +/** + * The tree in sightmap's ComponentNode shape, so `sightmap coverage`, + * `explain --snap` and `sel-check` work on iOS snapshots. With `annotations`, + * nodes also carry `component` and `props`, like `sightmap snapshot --json`. + */ +export function toComponentTree(annotated, { annotations = false } = {}) { + const convert = (node) => { + const id = cssId(node); + const element = { tag: node.tag }; + if (id) element.id = id; + const attrs = attrsOf(node); + if (Object.keys(attrs).length) element.attrs = attrs; + const out = { + id: String(node.id), + role: node.tag, + name: node.label ?? '', + text: node.text, + value: node.value ?? '', + element, + bounds: { + x: Math.round(node.frame.x), + y: Math.round(node.frame.y), + width: Math.round(node.frame.w), + height: Math.round(node.frame.h), + }, + isVisible: node.visible, + isInteractive: node.interactive, + inViewport: node.onScreen, + isIgnored: node.ignored, + nthChild: node.nthChild, + }; + if (annotations && node.component) { + out.component = node.component; + if (Object.keys(node.props).length) out.props = node.props; + } + if (node.children.length) out.children = node.children.map(convert); + return out; + }; + return convert(annotated.root); +} + +/** Writes `` (the text rendering) and its `.tree.json` companion. */ +export function writeSnap(annotated, snapPath, { annotationsPath } = {}) { + mkdirSync(dirname(snapPath), { recursive: true }); + writeFileSync(snapPath, `${annotated.text}\n`); + writeFileSync(`${snapPath}.tree.json`, JSON.stringify(toComponentTree(annotated))); + if (annotationsPath) { + writeFileSync(annotationsPath, JSON.stringify({ tree: toComponentTree(annotated, { annotations: true }) })); + } +} + +const QUERY_STEP = /^([A-Za-z_][A-Za-z0-9_-]*)((?:\[[^\]]*\])*)(?:#(\d+))?$/; +const QUERY_PRED = /\[\s*([A-Za-z_][A-Za-z0-9_-]*)\s*(\^=|\*=|\$=|=)\s*(?:"((?:[^"\\]|\\.)*)"|'((?:[^'\\]|\\.)*)'|([^\s\]]+))\s*(i)?\s*\]/g; + +function splitQuery(query) { + const tokens = []; + let cur = ''; + let depth = 0; + let inQuote = null; + for (const ch of query.trim()) { + if (inQuote) { + if (ch === inQuote) inQuote = null; + } else if (ch === '"' || ch === "'") inQuote = ch; + else if (ch === '[') depth++; + else if (ch === ']') depth--; + else if (/\s/.test(ch) && depth === 0) { + if (cur) tokens.push(cur); + cur = ''; + continue; + } + cur += ch; + } + if (cur) tokens.push(cur); + return tokens; +} + +/** + * Parses a component query: whitespace-separated steps, each + * `Name[prop op "value" i]…#N`. Capitalized names are sightmap components; + * lowercase names match normalized roles (`button[identifier="x"]`), which keeps + * unmapped apps addressable. + */ +export function parseQuery(query) { + const tokens = splitQuery(query); + if (!tokens.length) throw new BridgeError('bad_query', 'empty component query'); + return tokens.map((token, i) => { + if (token === '>') throw new BridgeError('bad_query', 'there is no ">" child combinator in component queries; use whitespace (descendant)'); + const m = QUERY_STEP.exec(token); + if (!m) throw new BridgeError('bad_query', `cannot parse ${quote(token)} in component query ${quote(query)}`); + const preds = []; + let consumed = ''; + for (const p of m[2].matchAll(QUERY_PRED)) { + consumed += p[0]; + const raw = p[3] ?? p[4] ?? p[5]; + preds.push({ prop: p[1], op: p[2], value: raw.replace(/\\(.)/g, '$1'), ci: !!p[6] }); + } + if (consumed.replace(/\s/g, '') !== m[2].replace(/\s/g, '')) { + throw new BridgeError('bad_query', `cannot parse predicates in ${quote(token)}; supported operators are =, ^=, *=, $=`); + } + if (m[3] !== undefined && i !== tokens.length - 1) { + throw new BridgeError('bad_query', '#N is only allowed on the last step of a component query'); + } + return { name: m[1], isComponent: /^[A-Z]/.test(m[1]), preds, nth: m[3] === undefined ? null : Number(m[3]) }; + }); +} + +function stepMatches(node, step) { + if (step.isComponent ? node.component !== step.name : node.tag !== step.name.toLowerCase()) return false; + const attrs = attrsOf(node); + return step.preds.every(({ prop, op, value, ci }) => { + const actual = node.props?.[prop] ?? attrs[prop]; + return compareValue(actual, op, value, ci); + }); +} + +function ancestorsMatch(node, steps) { + let j = steps.length - 1; + for (let a = node.parent; a && j >= 0; a = a.parent) if (stepMatches(a, steps[j])) j--; + return j < 0; +} + +/** + * Resolves a snapshot id (`42`) or a component query + * (`ProductRow[title^="Row 1"] BuyButton`) to one node. Throws a BridgeError + * with code `not_found`, `ambiguous` (with `candidates`) or `bad_query`. + */ +export function resolve(annotated, target) { + const asText = String(target).trim(); + if (/^\d+$/.test(asText)) { + const node = annotated.byId.get(Number(asText)); + if (!node) throw new BridgeError('not_found', `no element with id ${asText} in this snapshot; take a new snapshot`); + return node; + } + const steps = parseQuery(asText); + const last = steps[steps.length - 1]; + const found = annotated.nodes.filter((n) => stepMatches(n, last) && ancestorsMatch(n, steps.slice(0, -1))); + if (last.nth !== null) { + if (found[last.nth]) return found[last.nth]; + throw new BridgeError('not_found', `${quote(asText)} has ${found.length} match(es); occurrence #${last.nth} does not exist`); + } + if (found.length === 1) return found[0]; + if (!found.length) { + const known = !last.isComponent || annotated.nodes.some((n) => n.component === last.name); + const hint = known ? '' : ` (no ${last.name} component in this snapshot)`; + throw new BridgeError('not_found', `no element matches ${quote(asText)}${hint}`); + } + const candidates = found.map((n) => nodeLabel(n)); + throw new BridgeError( + 'ambiguous', + `${quote(asText)} matches ${found.length} elements; add a predicate or append #N:\n ${candidates.join('\n ')}`, + candidates, + ); +} + +export function centerOf(node) { + return { x: node.frame.x + node.frame.w / 2, y: node.frame.y + node.frame.h / 2 }; +} diff --git a/skills/subtext-ios-sim/lib/sightmap-cli.mjs b/skills/subtext-ios-sim/lib/sightmap-cli.mjs new file mode 100644 index 0000000..c39ecd6 --- /dev/null +++ b/skills/subtext-ios-sim/lib/sightmap-cli.mjs @@ -0,0 +1,79 @@ +import { execFileSync, spawnSync } from 'node:child_process'; +import { existsSync, mkdirSync, readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs'; +import { homedir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { parseCorpus } from './corpus.mjs'; +import { SimError } from './errors.mjs'; + +export const SIGHTMAP_PACKAGE = '@sightmap/sightmap@0.32.0'; +export const CACHE_ROOT = process.env.SUBTEXT_SIM_CACHE || join(homedir(), 'Library', 'Caches', 'subtext-sim'); +const NPX_PATH_CACHE = join(CACHE_ROOT, `sightmap-${SIGHTMAP_PACKAGE.split('@').at(-1)}.path`); + +const works = (bin) => spawnSync(bin, ['version'], { encoding: 'utf8', timeout: 30_000 }).status === 0; + +/** SIGHTMAP_BIN, then `sightmap` on PATH, then the pinned package through npx (its resolved path is cached). */ +export function resolveSightmapBin() { + if (process.env.SIGHTMAP_BIN) return process.env.SIGHTMAP_BIN; + if (works('sightmap')) return 'sightmap'; + if (existsSync(NPX_PATH_CACHE)) { + const cached = readFileSync(NPX_PATH_CACHE, 'utf8').trim(); + if (cached && existsSync(cached)) return cached; + } + const r = spawnSync('npx', ['-y', '-p', SIGHTMAP_PACKAGE, 'sh', '-c', 'command -v sightmap'], { encoding: 'utf8', timeout: 180_000 }); + const path = r.status === 0 ? r.stdout.trim().split('\n').at(-1) : ''; + if (!path || !works(path)) { + throw new SimError('corpus_unavailable', `sightmap CLI not found (no SIGHTMAP_BIN, not on PATH, and npx ${SIGHTMAP_PACKAGE} failed)`); + } + mkdirSync(dirname(NPX_PATH_CACHE), { recursive: true }); + writeFileSync(NPX_PATH_CACHE, `${path}\n`); + return path; +} + +function yamlFiles(dir) { + const out = []; + const walk = (d) => { + for (const entry of readdirSync(d, { withFileTypes: true })) { + if (entry.name.startsWith('.')) continue; + const p = join(d, entry.name); + if (entry.isDirectory()) walk(p); + else if (/\.ya?ml$/.test(entry.name)) out.push(p); + } + }; + if (existsSync(dir)) walk(dir); + return out.sort(); +} + +const EMPTY = { globals: [], views: [], memory: [] }; + +/** Loads `.sightmap/` through `sightmap export`, re-exporting only when a YAML file changes. */ +export class CorpusCache { + constructor(sightmapDir) { + this.dir = sightmapDir; + this.signature = null; + this.corpus = EMPTY; + this.bin = null; + } + + get() { + const files = yamlFiles(this.dir); + const signature = files.map((f) => `${f}:${statSync(f).mtimeMs}`).join('|'); + if (signature === this.signature) return this.corpus; + if (!files.length) { + this.corpus = EMPTY; + } else { + this.bin ??= resolveSightmapBin(); + let out; + try { + out = execFileSync(this.bin, ['export', '--sightmap-dir', this.dir], { + encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], maxBuffer: 64 * 1024 * 1024, + }); + } catch (err) { + const detail = String(err.stderr || err.message).trim().split('\n').at(-1); + throw new SimError('corpus_invalid', `could not load ${this.dir}: ${detail}`, 'fix the YAML under .sightmap/, then retry'); + } + this.corpus = parseCorpus(JSON.parse(out)); + } + this.signature = signature; + return this.corpus; + } +} diff --git a/skills/subtext-ios-sim/lib/simctl.mjs b/skills/subtext-ios-sim/lib/simctl.mjs new file mode 100644 index 0000000..3607234 --- /dev/null +++ b/skills/subtext-ios-sim/lib/simctl.mjs @@ -0,0 +1,95 @@ +import { execFile } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { join } from 'node:path'; +import { SimError } from './errors.mjs'; + +export function xcrun(args, { timeout = 120_000, env } = {}) { + return new Promise((resolve, reject) => { + execFile('xcrun', args, { timeout, maxBuffer: 64 * 1024 * 1024, env: env ? { ...process.env, ...env } : process.env }, (error, stdout, stderr) => { + if (error) { + const detail = String(stderr || error.message).trim().split('\n').filter(Boolean).at(-1) ?? error.message; + reject(new SimError('simctl_failed', `xcrun ${args.slice(0, 2).join(' ')} failed: ${detail}`)); + } else resolve(String(stdout)); + }); + }); +} + +export async function listDevices() { + const { devices } = JSON.parse(await xcrun(['simctl', 'list', 'devices', 'available', '-j'])); + return Object.entries(devices) + .filter(([runtime]) => /SimRuntime\.iOS-/.test(runtime)) + .flatMap(([runtime, list]) => list.map((d) => ({ ...d, runtime, runtimeVersion: runtime.split('iOS-')[1].replace(/-/g, '.') }))); +} + +const versionKey = (v) => v.split('.').map((n) => n.padStart(4, '0')).join('.'); + +/** Picks a device by UDID or name; without one, a booted iOS device, else the newest-runtime iPhone. */ +export function pickDevice(devices, wanted) { + if (wanted) { + const byUdid = devices.find((d) => d.udid.toLowerCase() === wanted.toLowerCase()); + if (byUdid) return byUdid; + const byName = devices.filter((d) => d.name === wanted) + .sort((a, b) => (b.state === 'Booted') - (a.state === 'Booted') || versionKey(b.runtimeVersion).localeCompare(versionKey(a.runtimeVersion))); + if (byName.length) return byName[0]; + throw new SimError('device_not_found', `no available iOS simulator named or with UDID "${wanted}"`, 'list them with `xcrun simctl list devices available`'); + } + const booted = devices.find((d) => d.state === 'Booted'); + if (booted) return booted; + const iphones = devices.filter((d) => d.name.startsWith('iPhone')) + .sort((a, b) => versionKey(b.runtimeVersion).localeCompare(versionKey(a.runtimeVersion))); + if (!iphones.length) throw new SimError('device_not_found', 'no available iOS simulator', 'run `subtext-sim doctor` for the fix'); + return iphones[0]; +} + +export async function boot(udid) { + await xcrun(['simctl', 'boot', udid]).catch((err) => { + if (!/current state: Booted/i.test(err.message)) throw err; + }); + await xcrun(['simctl', 'bootstatus', udid], { timeout: 300_000 }); +} + +export const shutdown = (udid) => xcrun(['simctl', 'shutdown', udid]).catch(() => {}); + +export async function accessibilityEnabled(udid) { + const out = await xcrun(['simctl', 'spawn', udid, 'defaults', 'read', 'com.apple.Accessibility', 'ApplicationAccessibilityEnabled']).catch(() => '0'); + return out.trim() === '1'; +} + +export const enableAccessibility = (udid) => + xcrun(['simctl', 'spawn', udid, 'defaults', 'write', 'com.apple.Accessibility', 'ApplicationAccessibilityEnabled', '-bool', 'true']); + +export const install = (udid, appPath) => xcrun(['simctl', 'install', udid, appPath], { timeout: 300_000 }); + +/** Launches with stdout/stderr redirected to files; NSUnbufferedIO keeps `print` output from sitting in a buffer. */ +export async function launch(udid, bundleId, { stdout, stderr, terminate = true } = {}) { + const args = ['simctl', 'launch']; + if (terminate) args.push('--terminate-running-process'); + if (stdout) args.push(`--stdout=${stdout}`); + if (stderr) args.push(`--stderr=${stderr}`); + const out = await xcrun([...args, udid, bundleId], { env: { SIMCTL_CHILD_NSUnbufferedIO: 'YES' } }); + const pid = Number(out.trim().split(':').at(-1)); + return Number.isFinite(pid) ? pid : null; +} + +export const terminate = (udid, bundleId) => xcrun(['simctl', 'terminate', udid, bundleId]).catch(() => {}); +export const openUrl = (udid, url) => xcrun(['simctl', 'openurl', udid, url]); +export const screenshot = (udid, path) => xcrun(['simctl', 'io', udid, 'screenshot', path]); + +export async function appContainer(udid, bundleId) { + try { + return (await xcrun(['simctl', 'get_app_container', udid, bundleId, 'app'])).trim(); + } catch { + throw new SimError('app_not_installed', `${bundleId} is not installed on this simulator`, 'pass --app path/to/MyApp.app to install it'); + } +} + +export function plistValue(appPath, key) { + const plist = join(appPath, 'Info.plist'); + if (!existsSync(plist)) throw new SimError('bad_app', `${appPath} has no Info.plist; pass the .app bundle built for the simulator`); + return new Promise((resolve, reject) => { + execFile('plutil', ['-extract', key, 'raw', '-o', '-', plist], (error, stdout) => { + if (error) reject(new SimError('bad_app', `${plist} has no ${key}`)); + else resolve(String(stdout).trim()); + }); + }); +} diff --git a/skills/subtext-ios-sim/test/axe-to-helper.mjs b/skills/subtext-ios-sim/test/axe-to-helper.mjs new file mode 100644 index 0000000..7056d75 --- /dev/null +++ b/skills/subtext-ios-sim/test/axe-to-helper.mjs @@ -0,0 +1,39 @@ +#!/usr/bin/env node +// Converts AXe `describe-ui` JSON into the helper's `describe` result shape, to +// build fixtures from recordings made before the helper existed. `subrole` is +// kept because AXe reports it; the helper may omit it. +import { readFileSync, writeFileSync } from 'node:fs'; +import { pathToFileURL } from 'node:url'; + +function node(n) { + const f = n.frame ?? {}; + const out = { + role: n.role ?? null, + type: n.type ?? null, + label: n.AXLabel ?? null, + identifier: n.AXUniqueId ?? null, + value: n.AXValue ?? null, + frame: { x: f.x ?? 0, y: f.y ?? 0, w: f.width ?? 0, h: f.height ?? 0 }, + enabled: n.enabled !== false, + children: (n.children ?? []).map(node), + }; + if (n.subrole) out.subrole = n.subrole; + return out; +} + +export function axeToHelper(axe) { + const roots = Array.isArray(axe) ? axe : [axe]; + const root = roots.length === 1 + ? node(roots[0]) + : { role: 'AXGroup', type: 'Group', label: null, identifier: null, value: null, frame: { x: 0, y: 0, w: 0, h: 0 }, enabled: true, children: roots.map(node) }; + return { screen: { w: root.frame.w, h: root.frame.h }, root }; +} + +if (import.meta.url === pathToFileURL(process.argv[1]).href) { + const [input, output] = process.argv.slice(2); + if (!input || !output) { + console.error('usage: axe-to-helper.mjs AXE.json OUT.json'); + process.exit(2); + } + writeFileSync(output, `${JSON.stringify(axeToHelper(JSON.parse(readFileSync(input, 'utf8'))))}\n`); +} diff --git a/skills/subtext-ios-sim/test/bridge.test.mjs b/skills/subtext-ios-sim/test/bridge.test.mjs new file mode 100644 index 0000000..8ad6ad5 --- /dev/null +++ b/skills/subtext-ios-sim/test/bridge.test.mjs @@ -0,0 +1,261 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { existsSync, mkdtempSync, readFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { normalize } from '../lib/normalize.mjs'; +import { annotate, BridgeError, resolve, snapshot, toComponentTree, writeSnap } from '../lib/sightmap-bridge.mjs'; +import { loadCorpus, loadCorpusFile, parseCorpus } from '../lib/corpus.mjs'; +import { matches, parseSelector } from '../lib/selector.mjs'; + +const here = dirname(fileURLToPath(import.meta.url)); +const fixture = (name) => JSON.parse(readFileSync(join(here, 'fixtures', `describe-${name}.json`), 'utf8')); +const corpus = loadCorpusFile(join(here, 'fixtures', 'corpus.json')); +const parityCorpus = loadCorpusFile(join(here, 'fixtures', 'parity-corpus.json')); +const benchSample = join(here, '..', 'helper', 'testdata', 'describe-bench.json'); + +const sightmapBin = (() => { + const bin = process.env.SIGHTMAP_BIN || 'sightmap'; + return spawnSync(bin, ['version'], { encoding: 'utf8' }).status === 0 ? bin : null; +})(); +const needsSightmap = sightmapBin ? {} : { skip: 'sightmap CLI not found (set SIGHTMAP_BIN)' }; + +const byIdentifier = (snap, identifier) => snap.nodes.find((n) => n.identifier === identifier); + +test('normalize collapses wrappers and maps roles to short tags', () => { + const { root, nodes, screen } = normalize(fixture('simple')); + assert.deepEqual(screen, { w: 402, h: 874 }); + assert.equal(root.tag, 'application'); + assert.equal(root.label, 'Bench'); + assert.ok(nodes.length < 25, `expected wrappers collapsed, got ${nodes.length} nodes`); + for (const n of nodes) { + if (n.tag === 'other' && n.ignored) assert.ok(n.children.length > 1, `wrapper ${n.id} should have been collapsed`); + } + const tags = Object.fromEntries(nodes.filter((n) => n.identifier).map((n) => [n.identifier, n.tag])); + assert.equal(tags['simple.primary'], 'button'); + assert.equal(tags['simple.email'], 'textfield'); + assert.equal(tags['simple.toggle'], 'switch'); + assert.equal(tags['1.circle'], 'tab'); + assert.equal(tags.lastTapped, 'text'); + assert.deepEqual(nodes.map((n) => n.id), nodes.map((_, i) => i + 1)); +}); + +test('normalize computes visibility against the screen', () => { + const { nodes } = normalize(fixture('eager')); + const buttons = nodes.filter((n) => n.identifier?.startsWith('eager.')); + assert.equal(buttons.length, 300); + assert.ok(byIdentifier({ nodes }, 'eager.0').visible); + assert.ok(byIdentifier({ nodes }, 'eager.0').onScreen); + assert.equal(byIdentifier({ nodes }, 'eager.299').visible, false); + assert.ok(buttons.filter((b) => !b.visible).length > 200); +}); + +test('normalize keeps identified groups in deep trees', () => { + const { nodes } = normalize(fixture('deep')); + const groups = nodes.filter((n) => n.identifier?.startsWith('deepc.')); + assert.equal(groups.length, 40); + assert.ok(groups.every((g) => g.tag === 'other' && !g.ignored)); + assert.ok(byIdentifier({ nodes }, 'deep.leaf')); +}); + +test('annotate infers the view and names components by identifier and text', () => { + const snap = snapshot(fixture('simple'), corpus); + assert.equal(snap.view, 'Simple'); + assert.equal(snap.viewInferred, true); + assert.equal(byIdentifier(snap, 'simple.primary').component, 'PrimaryButton'); + assert.equal(byIdentifier(snap, 'simple.secondary').component, 'SecondaryButton'); + assert.deepEqual(byIdentifier(snap, 'simple.toggle').props, { on: '1' }); + assert.equal(byIdentifier(snap, 'lastTapped').component, 'LastTapped'); + const tabs = snap.nodes.filter((n) => n.component === 'Tab').map((n) => n.props.title); + assert.deepEqual(tabs, ['Simple', 'Heavy', 'Deep', 'Eager']); + assert.match(snap.text, /^\[View: Simple\] \(inferred\)/); + assert.match(snap.text, /\n {2}\d+ \[PrimaryButton "Primary"\]\n/); + assert.match(snap.text, /\[NotificationsToggle on="1" "Notifications"\]/); + assert.match(snap.text, /- The tab bar is present on every screen\./); + assert.match(snap.text, /9 interactive · 8 direct T1 \(89%\) · 1 scoped T2 \(11%\) · 0 orphaned T3 ✓/); +}); + +test('annotate extracts child-component and exists: properties', () => { + const snap = snapshot(fixture('heavy'), corpus); + assert.equal(snap.view, 'Heavy'); + const rows = snap.nodes.filter((n) => n.component === 'ProductRow'); + assert.equal(rows.length, 10); + assert.deepEqual(rows[3].props, { title: 'Row 3', price: '$3.00', favoritable: 'true' }); + assert.match(snap.text, /\[ProductRow favoritable="true" price="\$0\.00" title="Row 0" "Row 0, Subtitle 0, \$0\.00"\]/); + assert.match(snap.text, /- Heavy: Rows lazy-load/); +}); + +test('annotate honors an explicit view and rejects unknown views', () => { + const snap = snapshot(fixture('heavy'), corpus, { view: 'Simple' }); + assert.equal(snap.view, 'Simple'); + assert.equal(snap.viewInferred, false); + assert.ok(!snap.nodes.some((n) => n.component === 'ProductRow')); + assert.ok(snap.nodes.some((n) => n.component === 'Tab')); + assert.throws(() => snapshot(fixture('heavy'), corpus, { view: 'Nope' }), { code: 'unknown_view' }); +}); + +test('first match wins and conflicts are reported, view components before globals', () => { + const conflicting = parseCorpus({ + globals: [ + { name: 'AnyButton', selectors: ['button'] }, + { name: 'Primary', selectors: ['[identifier="simple.primary"]'] }, + ], + views: [{ name: 'Simple', route: '/simple', components: [{ name: 'ViewPrimary', selectors: ['[text="Primary"]'] }] }], + }); + const snap = snapshot(fixture('simple'), conflicting); + const primary = byIdentifier(snap, 'simple.primary'); + assert.equal(primary.component, 'ViewPrimary'); + assert.deepEqual(snap.conflicts.find((c) => c.id === primary.id).names, ['ViewPrimary', 'AnyButton', 'Primary']); + assert.match(snap.text, /matched by 3 components \(ViewPrimary, AnyButton, Primary\) — only ViewPrimary applied/); +}); + +test('render hides off-screen nodes and says so', () => { + const snap = snapshot(fixture('eager'), corpus); + assert.equal(snap.view, 'Eager'); + assert.ok(!snap.text.includes('key="eager.299"')); + assert.match(snap.text, /\[Off-screen\]\n264 off-screen node\(s\) not shown \(264 interactive\)/); +}); + +test('resolve accepts snapshot ids', () => { + const snap = snapshot(fixture('simple'), corpus); + const primary = byIdentifier(snap, 'simple.primary'); + assert.equal(resolve(snap, String(primary.id)), primary); + assert.equal(resolve(snap, primary.id), primary); + assert.throws(() => resolve(snap, '9999'), { code: 'not_found' }); +}); + +test('resolve accepts component queries with predicates, descendants and #N', () => { + const snap = snapshot(fixture('heavy'), corpus); + assert.equal(resolve(snap, 'ProductRow[title="Row 3"]').identifier, 'row.3.buy-row.3.fav'); + assert.equal(resolve(snap, 'ProductRow[title^="row 7" i] BuyButton').identifier, 'row.7.buy'); + assert.equal(resolve(snap, "ProductRow[price*='5.'] FavoriteButton").identifier, 'row.5.fav'); + assert.equal(resolve(snap, 'BuyButton#2').identifier, 'row.2.buy'); + assert.equal(resolve(snap, 'Tab[title="Eager"]').label, 'Eager'); + assert.equal(resolve(snap, 'button[identifier="row.4.buy"]').label, 'Buy'); + assert.equal(resolve(snap, 'ProductRow[identifier$="row.1.fav"]').props.title, 'Row 1'); +}); + +test('resolve reports ambiguity, absence and bad queries', () => { + const snap = snapshot(fixture('heavy'), corpus); + assert.throws(() => resolve(snap, 'BuyButton'), (err) => { + assert.ok(err instanceof BridgeError); + assert.equal(err.code, 'ambiguous'); + assert.equal(err.candidates.length, 10); + return true; + }); + assert.throws(() => resolve(snap, 'ProductRow[title="Row 42"]'), { code: 'not_found' }); + assert.throws(() => resolve(snap, 'Checkout'), /no Checkout component in this snapshot/); + assert.throws(() => resolve(snap, 'BuyButton#99'), { code: 'not_found' }); + assert.throws(() => resolve(snap, 'ProductRow > BuyButton'), { code: 'bad_query' }); + assert.throws(() => resolve(snap, 'ProductRow[title~="x"]'), { code: 'bad_query' }); +}); + +test('resolve finds off-screen nodes so callers can refuse or scroll', () => { + const snap = snapshot(fixture('eager'), corpus); + const node = resolve(snap, 'EagerButton[key="eager.250"]'); + assert.equal(node.label, 'Eager 250'); + assert.equal(node.visible, false); + assert.equal(node.onScreen, false); +}); + +test('selector parser covers the sightmap subset and rejects the rest', () => { + const { nodes } = normalize(fixture('heavy')); + const count = (sel) => nodes.filter((n) => matches(n, sel)).length; + assert.equal(count('button:has(> button[identifier$=".buy"])'), 10); + assert.equal(count('other[text="Tab Bar"] tab'), 4); + assert.equal(count('other[text="Tab Bar"] > tab'), 0); + assert.equal(count('text:not([text^="Row"]):not([text^="$"])'), count('text') - 20); + assert.equal(count(':is(heading, slider)'), 3); + assert.equal(count('[identifier^="ROW.1." i]'), 3); + assert.equal(count('#lastTapped'), 1); + assert.equal(count('.anything'), 0); + assert.throws(() => parseSelector('button + text'), { code: 'bad_selector' }); + assert.throws(() => parseSelector('button:first-child'), { code: 'bad_selector' }); + assert.throws(() => parseSelector('button['), { code: 'bad_selector' }); +}); + +test('toComponentTree emits sightmap ComponentNode shape', () => { + const snap = snapshot(fixture('simple'), corpus); + const tree = toComponentTree(snap); + assert.equal(tree.id, '1'); + assert.equal(tree.element.tag, 'application'); + const walk = (n, out = []) => { + out.push(n); + (n.children ?? []).forEach((c) => walk(c, out)); + return out; + }; + const all = walk(tree); + assert.equal(all.length, snap.nodes.length); + for (const n of all) { + for (const k of ['x', 'y', 'width', 'height']) assert.ok(Number.isInteger(n.bounds[k])); + for (const k of ['isVisible', 'isInteractive', 'inViewport', 'isIgnored']) assert.equal(typeof n[k], 'boolean'); + assert.equal(n.component, undefined); + } + const email = all.find((n) => n.element.attrs?.identifier === 'simple.email'); + assert.deepEqual(email.element, { tag: 'textfield', attrs: { identifier: 'simple.email', value: 'Email' } }); + const last = all.find((n) => n.element.attrs?.identifier === 'lastTapped'); + assert.equal(last.element.id, 'lastTapped'); + const annotated = toComponentTree(snap, { annotations: true }); + assert.equal(walk(annotated).find((n) => n.element.attrs?.identifier === 'simple.toggle').props.on, '1'); +}); + +test('writeSnap writes the text snap and tree companion', () => { + const dir = mkdtempSync(join(tmpdir(), 'subtext-sim-')); + const snap = snapshot(fixture('simple'), corpus, { url: 'ios-sim://bench' }); + const path = join(dir, 'simple.snap'); + writeSnap(snap, path, { annotationsPath: join(dir, 'simple.annotated.json') }); + assert.match(readFileSync(path, 'utf8'), /^\[View: Simple\] \(inferred\) ios-sim:\/\/bench\n/); + assert.equal(JSON.parse(readFileSync(`${path}.tree.json`, 'utf8')).element.tag, 'application'); + assert.ok(JSON.parse(readFileSync(join(dir, 'simple.annotated.json'), 'utf8')).tree); +}); + +test('helper describe sample annotates when present', { skip: !existsSync(benchSample) && 'no helper sample yet' }, () => { + const snap = snapshot(JSON.parse(readFileSync(benchSample, 'utf8')), corpus); + assert.ok(snap.nodes.length > 1); + assert.match(snap.text, /--- component tree ---/); +}); + +test('sightmap export of the fixture corpus matches the committed wire', needsSightmap, () => { + const live = loadCorpus(join(here, 'fixtures', 'sightmap'), { bin: sightmapBin }); + assert.deepEqual(live, corpus); +}); + +const parseExplain = (out) => { + const owners = new Map(); + let id = null; + for (const line of out.split('\n')) { + const head = /\s#(\d+)$/.exec(line); + if (head && !line.startsWith(' ')) id = Number(head[1]); + const cov = /coverage: T1 \(direct\)\s+← \[([^\]]+)\]/.exec(line); + if (cov && id !== null) owners.set(id, cov[1]); + } + return owners; +}; + +for (const name of ['simple', 'heavy', 'eager', 'deep']) { + test(`matcher agrees with sightmap explain on ${name}`, needsSightmap, () => { + const snap = snapshot(fixture(name), parityCorpus); + const dir = mkdtempSync(join(tmpdir(), 'subtext-sim-')); + const path = join(dir, `${name}.snap`); + writeSnap(snap, path); + const run = spawnSync(sightmapBin, [ + 'explain', '--sightmap-dir', join(here, 'fixtures', 'sightmap-parity'), '--snap', `${path}.tree.json`, '--max', '100000', '*', + ], { encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 }); + assert.equal(run.status, 0, run.stderr); + const theirs = parseExplain(run.stdout); + const ours = new Map(snap.nodes.filter((n) => n.component).map((n) => [n.id, n.component])); + assert.ok(ours.size > 0); + assert.deepEqual(ours, theirs); + }); +} + +test('sightmap coverage reads an iOS snap', needsSightmap, () => { + const dir = mkdtempSync(join(tmpdir(), 'subtext-sim-')); + const path = join(dir, 'simple.snap'); + writeSnap(snapshot(fixture('simple'), parityCorpus), path); + const run = spawnSync(sightmapBin, ['coverage', '--sightmap-dir', join(here, 'fixtures', 'sightmap-parity'), path], { encoding: 'utf8' }); + assert.match(run.stdout, /9 interactive · \d+ direct T1/); + assert.doesNotMatch(run.stdout + run.stderr, /parse tree JSON/); +}); diff --git a/skills/subtext-ios-sim/test/cli.test.mjs b/skills/subtext-ios-sim/test/cli.test.mjs new file mode 100644 index 0000000..0a55157 --- /dev/null +++ b/skills/subtext-ios-sim/test/cli.test.mjs @@ -0,0 +1,185 @@ +import { test, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { formatResult, parseArgs, toRequest } from '../lib/cli.mjs'; +import { call, readSession } from '../lib/client.mjs'; +import { runDaemon } from '../lib/daemon.mjs'; +import { HelperClient } from '../lib/helper-client.mjs'; +import { ConsoleBuffer, parseLogLine } from '../lib/logs.mjs'; +import { pickDevice } from '../lib/simctl.mjs'; +import { readRun } from '../lib/runlog.mjs'; + +const here = dirname(fileURLToPath(import.meta.url)); +const FAKE_HELPER = join(here, 'fakes', 'fake-helper.mjs'); + +test('parseArgs splits command, positionals and flags', () => { + assert.deepEqual(parseArgs(['fill', 'login-email', 'a@b.co', '--clear']), { + command: 'fill', positionals: ['login-email', 'a@b.co'], flags: { clear: true }, + }); + assert.deepEqual(parseArgs(['scroll', 'down', '--until=ProductRow[title="A b"]', '--in', 'list']).flags, { until: 'ProductRow[title="A b"]', in: 'list' }); + assert.deepEqual(parseArgs(['type', '--', '--not-a-flag']).positionals, ['--not-a-flag']); + assert.equal(parseArgs([]).command, 'help'); + assert.throws(() => parseArgs(['snapshot', '--bogus']), { code: 'usage' }); + assert.throws(() => parseArgs(['start', '--device']), { code: 'usage' }); + assert.throws(() => parseArgs(['snapshot', '--json=1']), { code: 'usage' }); +}); + +test('toRequest maps commands onto daemon requests', () => { + const req = (argv) => toRequest(parseArgs(argv), '/proj'); + assert.deepEqual(req(['click', 'SignInButton']), { cmd: 'click', args: { target: 'SignInButton' } }); + assert.deepEqual(req(['tap-xy', '10', '20.5']), { cmd: 'tapXY', args: { x: 10, y: 20.5 } }); + assert.deepEqual(req(['screenshot', '--out', 'shots/a.png']), { cmd: 'screenshot', args: { out: '/proj/shots/a.png' } }); + assert.deepEqual(req(['console', 'list', '--limit', '5', '--level', 'error']), { cmd: 'consoleList', args: { limit: 5, grep: undefined, level: 'error' } }); + assert.deepEqual(req(['console', 'get', '7']), { cmd: 'consoleGet', args: { index: 7 } }); + assert.deepEqual(req(['scroll', 'down', '--until', 'far', '--max', '3']), { cmd: 'scroll', args: { direction: 'down', within: undefined, until: 'far', max: 3 } }); + assert.deepEqual(req(['run', 'new', '--label', 'checkout']), { cmd: 'runNew', args: { label: 'checkout' } }); + assert.equal(req(['run', 'tail']), null); + assert.equal(req(['start']), null); + assert.throws(() => req(['scroll', 'sideways']), { code: 'usage' }); + assert.throws(() => req(['tap-xy', '1']), { code: 'usage' }); + assert.throws(() => req(['tap-xy', 'a', '1']), { code: 'usage' }); + assert.throws(() => req(['screenshot']), { code: 'usage' }); + assert.throws(() => req(['frobnicate']), { code: 'usage' }); +}); + +test('pickDevice prefers an exact UDID, then name, then a booted device', () => { + const devices = [ + { udid: 'A', name: 'iPhone 16', state: 'Shutdown', runtimeVersion: '18.5' }, + { udid: 'B', name: 'iPhone 17 Pro', state: 'Shutdown', runtimeVersion: '26.2' }, + { udid: 'C', name: 'iPad Air', state: 'Booted', runtimeVersion: '26.2' }, + ]; + assert.equal(pickDevice(devices, 'a').udid, 'A'); + assert.equal(pickDevice(devices, 'iPhone 17 Pro').udid, 'B'); + assert.equal(pickDevice(devices).udid, 'C'); + assert.equal(pickDevice(devices.slice(0, 2)).udid, 'B'); + assert.throws(() => pickDevice(devices, 'Pixel'), { code: 'device_not_found' }); +}); + +test('log lines parse into console entries and surface the session URL', () => { + const line = JSON.stringify({ eventType: 'logEvent', messageType: 'Error', subsystem: 'dev.x', category: 'app', eventMessage: 'boom', timestamp: 't' }); + assert.deepEqual(parseLogLine(line), { time: 't', level: 'error', source: 'os_log', subsystem: 'dev.x', category: 'app', message: 'boom' }); + assert.equal(parseLogLine('Filtering the log data'), null); + assert.equal(parseLogLine(JSON.stringify({ eventType: 'activityCreateEvent', eventMessage: 'x' })), null); + + const buf = new ConsoleBuffer(3); + const seen = []; + buf.onSessionUrl = (url) => seen.push(url); + for (const message of ['a', 'FULLSTORY_SESSION_URL https://app.example/s/1', 'b', 'c']) buf.push({ level: 'info', message }); + assert.deepEqual(buf.entries.map((e) => e.index), [1, 2, 3]); + assert.equal(buf.sessionUrl, 'https://app.example/s/1'); + assert.deepEqual(seen, ['https://app.example/s/1']); + assert.deepEqual(buf.list({ grep: 'B' }).entries.map((e) => e.message), ['b']); + assert.equal(buf.get(0), null); +}); + +test('helper client restarts the helper after it dies', async () => { + const helper = new HelperClient(process.execPath, [FAKE_HELPER]); + assert.equal((await helper.request('hello')).version, 'fake'); + await assert.rejects(helper.request('crash'), { code: 'helper_crashed' }); + assert.equal((await helper.request('hello')).version, 'fake'); + assert.equal(helper.restarts, 1); + await assert.rejects(helper.request('nope'), { code: 'unknown_cmd' }); + helper.close(); +}); + +const dir = mkdtempSync(join(tmpdir(), 'subtext-sim-test-')); +let stopDaemon = async () => {}; +after(async () => { + await stopDaemon(); + rmSync(dir, { recursive: true, force: true }); +}); + +function fakeSimctl() { + const calls = []; + const rec = (name) => async (...args) => { calls.push([name, ...args]); return name === 'launch' ? 4242 : undefined; }; + return { + calls, + enableAccessibility: rec('enableAccessibility'), launch: rec('launch'), terminate: rec('terminate'), shutdown: rec('shutdown'), + openUrl: rec('openUrl'), screenshot: rec('screenshot'), install: rec('install'), + }; +} + +test('daemon serves the protocol over its socket', async () => { + const sightmapDir = join(dir, '.sightmap'); + const inputLog = join(dir, 'inputs.jsonl'); + writeFileSync(inputLog, ''); + process.env.FAKE_HELPER_LOG = inputLog; + const simctl = fakeSimctl(); + let stopped = false; + const { record, stop } = await runDaemon({ + udid: 'FAKE-UDID', deviceName: 'Fake Phone', bundleId: 'dev.example.app', processName: 'App', sightmapDir, + helperPath: process.execPath, onStopped: () => { stopped = true; }, timing: { settleMs: 0, scrollSettleMs: 0, readyTimeoutMs: 1000 }, + }, { simctl, helperArgs: [FAKE_HELPER], withLogs: false, log: () => {} }); + stopDaemon = stop; + const inputs = () => readFileSync(inputLog, 'utf8').split('\n').filter(Boolean).map((l) => JSON.parse(l)); + const sock = record.socket; + + const session = readSession(sightmapDir); + assert.equal(session.udid, 'FAKE-UDID'); + assert.equal(session.socket, sock); + assert.ok(session.run); + assert.deepEqual(simctl.calls.map((c) => c[0]), ['enableAccessibility', 'launch']); + + assert.deepEqual(await call(sock, 'ready'), { ready: true, interactive: 4 }); + const snap = await call(sock, 'snapshot', {}); + assert.match(snap.text, /button\[identifier="go"\] "Go"/); + assert.match(snap.text, /\[Off-screen\]/); + assert.equal(snap.tree, undefined); + assert.ok((await call(sock, 'snapshot', { json: true })).tree.children.length); + + assert.deepEqual(await call(sock, 'click', { target: 'go' }), { target: '2 button[identifier="go"] "Go"', x: 60, y: 30 }); + assert.deepEqual((await call(sock, 'click', { target: '2' })).y, 30); + assert.deepEqual((await call(sock, 'click', { target: 'button[text="Go"]' })).x, 60); + await assert.rejects(call(sock, 'click', { target: 'far' }), (err) => err.code === 'offscreen' && /scroll down --until/.test(err.hint)); + await assert.rejects(call(sock, 'click', { target: 'under' }), { code: 'covered' }); + await assert.rejects(call(sock, 'click', { target: 'nope' }), { code: 'disabled' }); + await assert.rejects(call(sock, 'click', { target: 'Missing' }), { code: 'not_found' }); + assert.equal(inputs().filter((i) => i.cmd === 'tap').length, 3); + + const filled = await call(sock, 'fill', { target: 'email', text: 'a@b.co' }); + assert.equal(filled.value, 'a@b.co'); + assert.equal((await call(sock, 'fill', { target: 'email', text: 'x@y.z', clear: true })).value, 'x@y.z'); + + await call(sock, 'scroll', { direction: 'down' }); + assert.deepEqual(inputs().at(-1), { cmd: "swipe", args: { udid: "FAKE-UDID", x1: 200, y1: 640, x2: 200, y2: 160, duration: 1.6 } }); + await assert.rejects(call(sock, 'scroll', { direction: 'down', until: 'far', max: 2 }), { code: 'not_found' }); + + await call(sock, 'tapXY', { x: 1, y: 2 }); + await call(sock, 'key', { key: 'return' }); + await call(sock, 'openurl', { url: 'app://x' }); + assert.deepEqual(simctl.calls.at(-1), ['openUrl', 'FAKE-UDID', 'app://x']); + await assert.rejects(call(sock, 'sessionUrl'), { code: 'no_session_url' }); + await assert.rejects(call(sock, 'bogus'), { code: 'unknown_cmd' }); + + const status = await call(sock, 'status'); + assert.equal(status.appPid, 4242); + assert.equal(status.helper.alive, true); + + const current = await call(sock, 'runCurrent'); + const entries = readRun(current.path); + assert.deepEqual(entries.map((e) => e.seq), entries.map((_, i) => i + 1)); + assert.equal(entries[0].action, 'start'); + const firstClick = entries.find((e) => e.action === 'click'); + assert.deepEqual(firstClick.target, { id: 2, component: null, identifier: 'go', text: 'Go', role: 'button' }); + assert.equal(firstClick.ok, true); + const refused = entries.find((e) => e.action === 'click' && !e.ok); + assert.equal(refused.error.code, 'offscreen'); + assert.ok(entries.some((e) => e.action === 'tap-xy')); + assert.ok(!entries.some((e) => e.action === 'status')); + assert.equal(readRun(current.path, { since: entries.length - 1 }).length, 1); + + const fresh = await call(sock, 'runNew', { label: 'Checkout flow' }); + assert.match(fresh.run, /-checkout-flow$/); + assert.notEqual(fresh.path, current.path); + + assert.equal(formatResult('click', { target: '2 button "Go"', x: 1, y: 2 }), '✓ clicked 2 button "Go" at (1, 2)'); + + await call(sock, 'stop', { shutdown: true }); + for (let i = 0; i < 50 && !stopped; i++) await new Promise((r) => setTimeout(r, 20)); + assert.ok(stopped); + assert.ok(!existsSync(join(sightmapDir, '.sim-session'))); + assert.deepEqual(simctl.calls.slice(-2).map((c) => c[0]), ['terminate', 'shutdown']); +}); diff --git a/skills/subtext-ios-sim/test/e2e.sh b/skills/subtext-ios-sim/test/e2e.sh new file mode 100755 index 0000000..757d7ac --- /dev/null +++ b/skills/subtext-ios-sim/test/e2e.sh @@ -0,0 +1,129 @@ +#!/usr/bin/env bash +# End-to-end run of subtext-sim against the sample app on a real simulator. +# +# test/e2e.sh --device # required; the device is shut down at the end +# +# Env: E2E_KEEP_BOOTED=1 leaves the simulator running; OUT (default: a temp dir) collects artifacts. +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +skill="$(cd "$here/.." && pwd)" +sim=(node "$skill/bin/subtext-sim") +device="" +while [[ $# -gt 0 ]]; do + case "$1" in + --device) device="$2"; shift 2 ;; + *) echo "usage: $0 --device " >&2; exit 2 ;; + esac +done +[[ -n "$device" ]] || { echo "usage: $0 --device " >&2; exit 2; } +out="${OUT:-$(mktemp -d "${TMPDIR:-/tmp}/subtext-sim-e2e.XXXXXX")}" +mkdir -p "$out" + +now() { node -e 'process.stdout.write(String(Date.now()))'; } +step_start=0 +step() { step_start=$(now); printf '\n== %s\n' "$*"; } +took() { printf ' (%s ms)\n' "$(( $(now) - step_start ))"; } +fail() { echo "E2E FAILED: $*" >&2; exit 1; } +expect() { grep -qF -- "$1" <<<"$2" || fail "expected output to contain: $1"; } + +cleanup() { + local code=$? + if [[ "${E2E_KEEP_BOOTED:-}" == 1 ]]; then "${sim[@]}" stop >/dev/null 2>&1 || true + else "${sim[@]}" stop --shutdown >/dev/null 2>&1 || true; fi + exit $code +} + +step "build helper" +bash "$skill/helper/build.sh" >/dev/null +took + +step "build sample app" +app="$(bash "$skill/fixtures/SampleApp/build.sh" 2>/dev/null | tail -1)" +echo " $app" +took + +# The sample app's .sightmap/ lives next to it, so run from there. +cd "$skill/fixtures/SampleApp" +trap cleanup EXIT + +step "start --detach" +"${sim[@]}" start --device "$device" --app "$app" --detach --label e2e +took + +step "snapshot (login)" +snap="$("${sim[@]}" snapshot)" +took +echo "$snap" +expect '[View: Login]' "$snap" +expect '[SignInButton "Sign In"]' "$snap" + +step "click SignInButton by component name (empty form → validation error)" +"${sim[@]}" click SignInButton +took +sleep 0.5 +expect '[LoginError' "$("${sim[@]}" snapshot)" + +step "fill email and password by identifier" +"${sim[@]}" fill login-email "agent@example.com" +"${sim[@]}" fill login-password "hunter2" +took + +step "click login-submit by identifier" +"${sim[@]}" click login-submit +took +sleep 1 + +step "snapshot (products)" +snap="$("${sim[@]}" snapshot)" +took +echo "$snap" | head -12 +expect '[View: Products]' "$snap" + +step "click an off-screen product is refused" +if "${sim[@]}" click product-row-150 2>"$out/refused.txt"; then fail "off-screen click was not refused"; fi +cat "$out/refused.txt" +expect 'scroll' "$(cat "$out/refused.txt")" + +step "scroll down until Product 150 is on screen" +"${sim[@]}" scroll down --until 'ProductRow[title^="Product 150,"]' +took + +step "click product-row-150" +"${sim[@]}" click product-row-150 +took +sleep 1 +snap="$("${sim[@]}" snapshot)" +expect '[ProductName "Product 150"]' "$snap" + +step "click AddToCartButton" +"${sim[@]}" click AddToCartButton +took +sleep 1 + +step "console list" +logs="$("${sim[@]}" console list --grep 'Product 150')" +echo "$logs" +expect 'added Product 150 to cart' "$logs" +expect 'signed in as agent@example.com' "$("${sim[@]}" console list --grep 'signed in')" + +step "screenshot" +"${sim[@]}" screenshot --out "$out/product-150.png" +file "$out/product-150.png" | grep -q 'PNG image' || fail "screenshot is not a PNG" +took + +step "run log" +runlog="$("${sim[@]}" run tail)" +echo "$runlog" +expect 'click → ' "$runlog" +grep -q '"text":""' "$(ls -t .sightmap/.sim-runs/*.jsonl | head -1)" || fail "password was not redacted in the run log" + +step "status" +"${sim[@]}" status + +step "stop" +trap - EXIT +if [[ "${E2E_KEEP_BOOTED:-}" == 1 ]]; then "${sim[@]}" stop; else "${sim[@]}" stop --shutdown; fi +took +echo +echo "E2E PASSED (artifacts in $out)" diff --git a/skills/subtext-ios-sim/test/fakes/fake-helper.mjs b/skills/subtext-ios-sim/test/fakes/fake-helper.mjs new file mode 100644 index 0000000..b66a564 --- /dev/null +++ b/skills/subtext-ios-sim/test/fakes/fake-helper.mjs @@ -0,0 +1,36 @@ +// Stands in for subtext-sim-helper: same JSON-lines protocol, a fixed screen, +// and every input request appended to $FAKE_HELPER_LOG for assertions. +import { appendFileSync } from 'node:fs'; +import { createInterface } from 'node:readline'; + +let email = ''; +const node = (type, frame, extra = {}) => ({ role: `AX${type}`, type, subrole: null, traits: [], label: null, identifier: null, value: null, frame, enabled: true, children: [], ...extra }); +const tree = () => ({ + screen: { w: 400, h: 800 }, + root: node('Application', { x: 0, y: 0, w: 400, h: 800 }, { + label: 'Fake', + children: [ + node('Button', { x: 10, y: 10, w: 100, h: 40 }, { label: 'Go', identifier: 'go' }), + node('TextField', { x: 10, y: 60, w: 200, h: 40 }, { identifier: 'email', value: email || 'Email' }), + node('Button', { x: 10, y: 120, w: 100, h: 40 }, { label: 'Nope', identifier: 'nope', enabled: false }), + node('Button', { x: 10, y: 1200, w: 100, h: 40 }, { label: 'Far', identifier: 'far' }), + node('Group', { x: 0, y: 740, w: 400, h: 60 }, { label: 'Tab Bar' }), + node('Button', { x: 10, y: 750, w: 100, h: 40 }, { label: 'Under', identifier: 'under' }), + ], + }), +}); + +createInterface({ input: process.stdin }).on('line', (line) => { + const { id, cmd, args } = JSON.parse(line); + const reply = (msg) => process.stdout.write(`${JSON.stringify({ id, ...msg })}\n`); + if (cmd === 'crash') process.exit(3); + if (process.env.FAKE_HELPER_LOG && cmd !== 'hello' && cmd !== 'describe') appendFileSync(process.env.FAKE_HELPER_LOG, `${JSON.stringify({ cmd, args })}\n`); + switch (cmd) { + case 'hello': return reply({ ok: true, result: { version: 'fake' } }); + case 'describe': return reply({ ok: true, result: tree() }); + case 'type': email += args.text; return reply({ ok: true, result: {} }); + case 'key': if (args.key === 'delete' || args.key === 'forward_delete') email = email.slice(0, -1); return reply({ ok: true, result: {} }); + case 'tap': case 'swipe': case 'button': return reply({ ok: true, result: {} }); + default: return reply({ ok: false, error: { code: 'unknown_cmd', message: `unknown cmd '${cmd}'` } }); + } +}); diff --git a/skills/subtext-ios-sim/test/fixtures/corpus.json b/skills/subtext-ios-sim/test/fixtures/corpus.json new file mode 100644 index 0000000..7c06ca4 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/corpus.json @@ -0,0 +1,214 @@ +{ + "memory": [ + "The tab bar is present on every screen." + ], + "globals": [ + { + "name": "TabBar", + "selectors": [ + "other[text=\"Tab Bar\"]" + ] + }, + { + "name": "Tab", + "selectors": [ + "other[text=\"Tab Bar\"] tab" + ], + "properties": [ + { + "name": "title", + "extract": "text" + }, + { + "name": "selected", + "extract": "attr=value" + } + ], + "parentChain": [ + "TabBar" + ] + }, + { + "name": "LastTapped", + "selectors": [ + "[identifier=\"lastTapped\"]" + ], + "properties": [ + { + "name": "text", + "extract": "text" + } + ] + } + ], + "views": [ + { + "name": "Eager", + "route": "/eager", + "components": [ + { + "name": "EagerButton", + "selectors": [ + "button[identifier^=\"eager.\"]" + ], + "properties": [ + { + "name": "key", + "extract": "attr=identifier" + } + ] + } + ] + }, + { + "name": "Heavy", + "route": "/heavy", + "memory": [ + "Rows lazy-load; scroll to reach rows past the first screen." + ], + "components": [ + { + "name": "ScreenTitle", + "selectors": [ + "heading" + ] + }, + { + "name": "ProductRow", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"])" + ], + "properties": [ + { + "name": "title", + "extract": "Title.text" + }, + { + "name": "price", + "extract": "Price.text" + }, + { + "name": "favoritable", + "extract": "exists:FavoriteButton" + } + ] + }, + { + "name": "Title", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"]) text[text^=\"Row\"]" + ], + "properties": [ + { + "name": "text", + "extract": "text" + } + ], + "parentChain": [ + "ProductRow" + ] + }, + { + "name": "Price", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"]) text[text^=\"$\"]" + ], + "properties": [ + { + "name": "text", + "extract": "text" + } + ], + "parentChain": [ + "ProductRow" + ] + }, + { + "name": "BuyButton", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"]) button[identifier$=\".buy\"]" + ], + "parentChain": [ + "ProductRow" + ] + }, + { + "name": "FavoriteButton", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"]) button[identifier$=\".fav\"]" + ], + "parentChain": [ + "ProductRow" + ] + } + ] + }, + { + "name": "Simple", + "route": "/simple", + "components": [ + { + "name": "TabBar", + "selectors": [ + "other[text=\"Tab Bar\"]" + ] + }, + { + "name": "Tab", + "selectors": [ + "other[text=\"Tab Bar\"] tab" + ], + "properties": [ + { + "name": "title", + "extract": "text" + }, + { + "name": "selected", + "extract": "attr=value" + } + ], + "parentChain": [ + "TabBar" + ] + }, + { + "name": "PrimaryButton", + "selectors": [ + "button[identifier=\"simple.primary\"]" + ] + }, + { + "name": "SecondaryButton", + "selectors": [ + "[text=\"Secondary\"]" + ] + }, + { + "name": "EmailField", + "selectors": [ + "textfield[identifier=\"simple.email\"]" + ], + "properties": [ + { + "name": "value", + "extract": "attr=value" + } + ] + }, + { + "name": "NotificationsToggle", + "selectors": [ + "switch[identifier=\"simple.toggle\"]" + ], + "properties": [ + { + "name": "on", + "extract": "attr=value" + } + ] + } + ] + } + ] +} diff --git a/skills/subtext-ios-sim/test/fixtures/describe-deep.json b/skills/subtext-ios-sim/test/fixtures/describe-deep.json new file mode 100644 index 0000000..f072eee --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-deep.json @@ -0,0 +1 @@ +{"screen":{"w":402,"h":874},"root":{"role":"AXApplication","type":"Application","label":"Bench","identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXScrollArea","type":"ScrollArea","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":0,"w":64,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":0,"w":64,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":0,"w":64,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":null,"identifier":"AdditionalDimmingOverlay","value":null,"frame":{"x":49,"y":731,"w":304,"h":286},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":726.2,"w":64,"h":147.79999999999995},"enabled":true,"children":[]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":169,"y":62,"w":64,"h":913.6666666666666},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.40","value":null,"frame":{"x":169,"y":62,"w":64,"h":913.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L40","identifier":"deep.40","value":null,"frame":{"x":186,"y":62,"w":30,"h":20.33333333333333},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.39","value":null,"frame":{"x":169,"y":84.33333587646484,"w":64,"h":891.3333129882812},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L39","identifier":"deep.39","value":null,"frame":{"x":186,"y":84.33333333333333,"w":30,"h":20.33333333333333},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.38","value":null,"frame":{"x":169,"y":106.66666412353516,"w":64,"h":868.9999389648438},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L38","identifier":"deep.38","value":null,"frame":{"x":186,"y":106.66666666666666,"w":30,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.37","value":null,"frame":{"x":169,"y":129,"w":64,"h":846.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L37","identifier":"deep.37","value":null,"frame":{"x":186.5,"y":129,"w":29,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.36","value":null,"frame":{"x":169,"y":151.3333282470703,"w":64,"h":824.3333129882812},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L36","identifier":"deep.36","value":null,"frame":{"x":186,"y":151.33333333333331,"w":30,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.35","value":null,"frame":{"x":169,"y":173.6666717529297,"w":64,"h":801.9999389648438},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L35","identifier":"deep.35","value":null,"frame":{"x":186.16666666666666,"y":173.66666666666666,"w":29.666666666666657,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.34","value":null,"frame":{"x":169,"y":196,"w":64,"h":779.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L34","identifier":"deep.34","value":null,"frame":{"x":186,"y":196,"w":30,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.33","value":null,"frame":{"x":169,"y":218.3333282470703,"w":64,"h":757.3333129882812},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L33","identifier":"deep.33","value":null,"frame":{"x":186,"y":218.33333333333334,"w":30,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.32","value":null,"frame":{"x":169,"y":240.6666717529297,"w":64,"h":734.9999389648438},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L32","identifier":"deep.32","value":null,"frame":{"x":186.33333333333334,"y":240.66666666666669,"w":29.333333333333343,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.31","value":null,"frame":{"x":169,"y":263,"w":64,"h":712.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L31","identifier":"deep.31","value":null,"frame":{"x":187.5,"y":263,"w":27,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.30","value":null,"frame":{"x":169,"y":285.3333435058594,"w":64,"h":690.333251953125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L30","identifier":"deep.30","value":null,"frame":{"x":186,"y":285.33333333333337,"w":30,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.29","value":null,"frame":{"x":169,"y":307.6666564941406,"w":64,"h":668},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L29","identifier":"deep.29","value":null,"frame":{"x":186.16666666666666,"y":307.66666666666674,"w":29.666666666666657,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.28","value":null,"frame":{"x":169,"y":330,"w":64,"h":645.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L28","identifier":"deep.28","value":null,"frame":{"x":186.16666666666666,"y":330.00000000000006,"w":29.666666666666657,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.27","value":null,"frame":{"x":169,"y":352.3333435058594,"w":64,"h":623.333251953125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L27","identifier":"deep.27","value":null,"frame":{"x":186.83333333333334,"y":352.33333333333337,"w":28.333333333333343,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.26","value":null,"frame":{"x":169,"y":374.6666564941406,"w":64,"h":601},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L26","identifier":"deep.26","value":null,"frame":{"x":186.16666666666666,"y":374.6666666666667,"w":29.666666666666657,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.25","value":null,"frame":{"x":169,"y":397,"w":64,"h":578.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L25","identifier":"deep.25","value":null,"frame":{"x":186.33333333333334,"y":397,"w":29.333333333333343,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.24","value":null,"frame":{"x":169,"y":419.3333435058594,"w":64,"h":556.333251953125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L24","identifier":"deep.24","value":null,"frame":{"x":186.33333333333334,"y":419.3333333333333,"w":29.333333333333343,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.23","value":null,"frame":{"x":169,"y":441.6666564941406,"w":64,"h":534},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L23","identifier":"deep.23","value":null,"frame":{"x":186.33333333333334,"y":441.66666666666663,"w":29.333333333333343,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.22","value":null,"frame":{"x":169,"y":464,"w":64,"h":511.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L22","identifier":"deep.22","value":null,"frame":{"x":186.5,"y":463.99999999999994,"w":29,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.21","value":null,"frame":{"x":169,"y":486.3333435058594,"w":64,"h":489.3332824707031},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L21","identifier":"deep.21","value":null,"frame":{"x":187.66666666666666,"y":486.33333333333326,"w":26.666666666666657,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.20","value":null,"frame":{"x":169,"y":508.6666564941406,"w":64,"h":466.9999694824219},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L20","identifier":"deep.20","value":null,"frame":{"x":186.33333333333334,"y":508.6666666666666,"w":29.333333333333343,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.19","value":null,"frame":{"x":169,"y":531,"w":64,"h":444.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L19","identifier":"deep.19","value":null,"frame":{"x":187.83333333333334,"y":530.9999999999999,"w":26.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.18","value":null,"frame":{"x":169,"y":553.3333129882812,"w":64,"h":422.33331298828125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L18","identifier":"deep.18","value":null,"frame":{"x":187.66666666666666,"y":553.3333333333333,"w":26.666666666666657,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.17","value":null,"frame":{"x":169,"y":575.6666870117188,"w":64,"h":399.99993896484375},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L17","identifier":"deep.17","value":null,"frame":{"x":188.33333333333334,"y":575.6666666666665,"w":25.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.16","value":null,"frame":{"x":169,"y":598,"w":64,"h":377.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L16","identifier":"deep.16","value":null,"frame":{"x":187.83333333333334,"y":597.9999999999999,"w":26.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.15","value":null,"frame":{"x":169,"y":620.3333129882812,"w":64,"h":355.33331298828125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L15","identifier":"deep.15","value":null,"frame":{"x":187.83333333333334,"y":620.3333333333333,"w":26.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.14","value":null,"frame":{"x":169,"y":642.6666870117188,"w":64,"h":332.99993896484375},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L14","identifier":"deep.14","value":null,"frame":{"x":187.66666666666666,"y":642.6666666666666,"w":26.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.13","value":null,"frame":{"x":169,"y":665,"w":64,"h":310.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L13","identifier":"deep.13","value":null,"frame":{"x":187.83333333333334,"y":665,"w":26.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.12","value":null,"frame":{"x":169,"y":687.3333129882812,"w":64,"h":288.33331298828125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L12","identifier":"deep.12","value":null,"frame":{"x":188,"y":687.3333333333334,"w":26,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.11","value":null,"frame":{"x":169,"y":709.6666870117188,"w":64,"h":265.99993896484375},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L11","identifier":"deep.11","value":null,"frame":{"x":189.16666666666666,"y":709.6666666666667,"w":23.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.10","value":null,"frame":{"x":169,"y":732,"w":64,"h":243.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L10","identifier":"deep.10","value":null,"frame":{"x":187.83333333333334,"y":732.0000000000001,"w":26.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.9","value":null,"frame":{"x":169,"y":754.3333129882812,"w":64,"h":221.33331298828125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L9","identifier":"deep.9","value":null,"frame":{"x":191.16666666666666,"y":754.3333333333335,"w":19.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.8","value":null,"frame":{"x":169,"y":776.6666870117188,"w":64,"h":198.99993896484375},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L8","identifier":"deep.8","value":null,"frame":{"x":191.16666666666666,"y":776.6666666666669,"w":19.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.7","value":null,"frame":{"x":169,"y":799,"w":64,"h":176.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L7","identifier":"deep.7","value":null,"frame":{"x":191.83333333333334,"y":799.0000000000002,"w":18.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.6","value":null,"frame":{"x":169,"y":821.3333129882812,"w":64,"h":154.33331298828125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L6","identifier":"deep.6","value":null,"frame":{"x":191.16666666666666,"y":821.3333333333336,"w":19.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.5","value":null,"frame":{"x":169,"y":843.6666870117188,"w":64,"h":131.99993896484375},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L5","identifier":"deep.5","value":null,"frame":{"x":191.33333333333334,"y":843.666666666667,"w":19.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.4","value":null,"frame":{"x":169,"y":866,"w":64,"h":109.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L4","identifier":"deep.4","value":null,"frame":{"x":191.16666666666666,"y":866.0000000000003,"w":19.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.3","value":null,"frame":{"x":169,"y":888.3333129882812,"w":64,"h":87.33331298828125},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L3","identifier":"deep.3","value":null,"frame":{"x":191.16666666666666,"y":888.3333333333337,"w":19.666666666666657,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.2","value":null,"frame":{"x":169,"y":910.6666870117188,"w":64,"h":64.99993896484375},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L2","identifier":"deep.2","value":null,"frame":{"x":191.33333333333334,"y":910.6666666666671,"w":19.333333333333343,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":"deepc.1","value":null,"frame":{"x":169,"y":933,"w":64,"h":42.6666259765625},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"L1","identifier":"deep.1","value":null,"frame":{"x":193,"y":933.0000000000005,"w":16,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Deepest","identifier":"deep.leaf","value":null,"frame":{"x":169,"y":955.3333333333333,"w":64,"h":20.33333333333337},"enabled":true,"children":[]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]}]},{"role":"AXSlider","type":"Slider","label":"Vertical scroll bar, 2 pages","identifier":null,"value":"0%","frame":{"x":200,"y":50.666666666666664,"w":30,"h":740.3333333333334},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":227,"y":53.666666666666686,"w":3,"h":606.3333333333333},"enabled":true,"children":[]}]},{"role":"AXSlider","type":"Slider","label":"Vertical scroll bar, 2 pages","identifier":null,"value":"0%","frame":{"x":200,"y":50.666666666666664,"w":30,"h":740.3333333333334},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":227,"y":53.666666666666686,"w":3,"h":606.3333333333333},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":"Tab Bar","identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":98.00000000000003,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXRadioButton","type":"RadioButton","label":"Simple","identifier":null,"value":0,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"1.circle.fill","identifier":"1.circle.fill","value":null,"frame":{"x":60,"y":801.3333333333334,"w":27.33333333333333,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Heavy","identifier":null,"value":0,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"2.circle.fill","identifier":"2.circle.fill","value":null,"frame":{"x":144.66666666666669,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Deep","identifier":null,"value":1,"frame":{"x":194.33333333333334,"y":795,"w":98.00000000000003,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"3.circle.fill","identifier":"3.circle.fill","value":null,"frame":{"x":229.33333333333334,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Eager","identifier":null,"value":0,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"4.circle.fill","identifier":"4.circle.fill","value":null,"frame":{"x":314,"y":801.3333333333334,"w":27.333333333333314,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":194.33333333333337,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]}]}]}]}]}]}]}]}]}]}]}} diff --git a/skills/subtext-ios-sim/test/fixtures/describe-eager.json b/skills/subtext-ios-sim/test/fixtures/describe-eager.json new file mode 100644 index 0000000..8c8225e --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-eager.json @@ -0,0 +1 @@ +{"screen":{"w":402,"h":874},"root":{"role":"AXApplication","type":"Application","label":"Bench","identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXScrollArea","type":"ScrollArea","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":0,"w":98.33333333333331,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":0,"w":98.33333333333331,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":0,"w":98.33333333333331,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":null,"identifier":"AdditionalDimmingOverlay","value":null,"frame":{"x":32,"y":731,"w":338.3333333333333,"h":286},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":726.2,"w":98.33333333333331,"h":147.79999999999995},"enabled":true,"children":[]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":152,"y":62,"w":98.33333333333331,"h":6720.333333333333},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"last:none n:0","identifier":"lastTapped","value":null,"frame":{"x":152,"y":62,"w":98.33333333333331,"h":20.33333333333333},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 0","identifier":"eager.0","value":null,"frame":{"x":172,"y":84.33333333333333,"w":58.66666666666666,"h":20.33333333333333},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 1","identifier":"eager.1","value":null,"frame":{"x":173.33333333333334,"y":106.66666666666666,"w":55.99999999999997,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 2","identifier":"eager.2","value":null,"frame":{"x":172,"y":129,"w":58.333333333333314,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 3","identifier":"eager.3","value":null,"frame":{"x":172,"y":151.33333333333331,"w":58.66666666666666,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 4","identifier":"eager.4","value":null,"frame":{"x":171.66666666666666,"y":173.66666666666666,"w":59,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 5","identifier":"eager.5","value":null,"frame":{"x":172,"y":196,"w":58.333333333333314,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 6","identifier":"eager.6","value":null,"frame":{"x":172,"y":218.33333333333331,"w":58.66666666666666,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 7","identifier":"eager.7","value":null,"frame":{"x":172.33333333333334,"y":240.66666666666666,"w":57.66666666666666,"h":20.333333333333343},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 8","identifier":"eager.8","value":null,"frame":{"x":171.66666666666666,"y":263,"w":59,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 9","identifier":"eager.9","value":null,"frame":{"x":172,"y":285.3333333333333,"w":58.66666666666666,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 10","identifier":"eager.10","value":null,"frame":{"x":168.33333333333334,"y":307.66666666666663,"w":65.99999999999997,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 11","identifier":"eager.11","value":null,"frame":{"x":169.66666666666666,"y":330,"w":63.33333333333334,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 12","identifier":"eager.12","value":null,"frame":{"x":168.33333333333334,"y":352.3333333333333,"w":65.66666666666666,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 13","identifier":"eager.13","value":null,"frame":{"x":168.33333333333334,"y":374.66666666666663,"w":65.99999999999997,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 14","identifier":"eager.14","value":null,"frame":{"x":168,"y":397,"w":66.33333333333331,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 15","identifier":"eager.15","value":null,"frame":{"x":168.33333333333334,"y":419.3333333333333,"w":65.99999999999997,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 16","identifier":"eager.16","value":null,"frame":{"x":168,"y":441.66666666666663,"w":66.33333333333331,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 17","identifier":"eager.17","value":null,"frame":{"x":168.66666666666666,"y":464,"w":65,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 18","identifier":"eager.18","value":null,"frame":{"x":168,"y":486.3333333333333,"w":66.33333333333331,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 19","identifier":"eager.19","value":null,"frame":{"x":168,"y":508.66666666666663,"w":66.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 20","identifier":"eager.20","value":null,"frame":{"x":167,"y":531,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 21","identifier":"eager.21","value":null,"frame":{"x":168.33333333333334,"y":553.3333333333333,"w":65.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 22","identifier":"eager.22","value":null,"frame":{"x":167,"y":575.6666666666666,"w":68.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 23","identifier":"eager.23","value":null,"frame":{"x":167,"y":598,"w":68.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 24","identifier":"eager.24","value":null,"frame":{"x":167,"y":620.3333333333333,"w":68.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 25","identifier":"eager.25","value":null,"frame":{"x":167,"y":642.6666666666666,"w":68.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 26","identifier":"eager.26","value":null,"frame":{"x":167,"y":665,"w":68.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 27","identifier":"eager.27","value":null,"frame":{"x":167.66666666666666,"y":687.3333333333333,"w":67.33333333333334,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 28","identifier":"eager.28","value":null,"frame":{"x":167,"y":709.6666666666666,"w":68.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 29","identifier":"eager.29","value":null,"frame":{"x":167,"y":732,"w":68.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 30","identifier":"eager.30","value":null,"frame":{"x":166.66666666666666,"y":754.3333333333333,"w":69.00000000000003,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 31","identifier":"eager.31","value":null,"frame":{"x":168.33333333333334,"y":776.6666666666666,"w":65.99999999999997,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 32","identifier":"eager.32","value":null,"frame":{"x":167,"y":799,"w":68.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 33","identifier":"eager.33","value":null,"frame":{"x":166.66666666666666,"y":821.3333333333333,"w":69.00000000000003,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 34","identifier":"eager.34","value":null,"frame":{"x":166.66666666666666,"y":843.6666666666666,"w":69.00000000000003,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 35","identifier":"eager.35","value":null,"frame":{"x":167,"y":866,"w":68.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 36","identifier":"eager.36","value":null,"frame":{"x":166.66666666666666,"y":888.3333333333333,"w":69.00000000000003,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 37","identifier":"eager.37","value":null,"frame":{"x":167,"y":910.6666666666666,"w":68.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 38","identifier":"eager.38","value":null,"frame":{"x":166.66666666666666,"y":933,"w":69.00000000000003,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 39","identifier":"eager.39","value":null,"frame":{"x":166.66666666666666,"y":955.3333333333333,"w":69.00000000000003,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 40","identifier":"eager.40","value":null,"frame":{"x":166.66666666666666,"y":977.6666666666666,"w":69.33333333333334,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 41","identifier":"eager.41","value":null,"frame":{"x":168,"y":1000,"w":66.33333333333331,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 42","identifier":"eager.42","value":null,"frame":{"x":167,"y":1022.3333333333333,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 43","identifier":"eager.43","value":null,"frame":{"x":166.66666666666666,"y":1044.6666666666665,"w":69.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 44","identifier":"eager.44","value":null,"frame":{"x":166.66666666666666,"y":1067,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 45","identifier":"eager.45","value":null,"frame":{"x":166.66666666666666,"y":1089.3333333333333,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 46","identifier":"eager.46","value":null,"frame":{"x":166.66666666666666,"y":1111.6666666666665,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 47","identifier":"eager.47","value":null,"frame":{"x":167,"y":1134,"w":68.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 48","identifier":"eager.48","value":null,"frame":{"x":166.66666666666666,"y":1156.3333333333333,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 49","identifier":"eager.49","value":null,"frame":{"x":166.66666666666666,"y":1178.6666666666665,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 50","identifier":"eager.50","value":null,"frame":{"x":167,"y":1201,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 51","identifier":"eager.51","value":null,"frame":{"x":168.33333333333334,"y":1223.3333333333333,"w":65.99999999999997,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 52","identifier":"eager.52","value":null,"frame":{"x":167,"y":1245.6666666666665,"w":68.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 53","identifier":"eager.53","value":null,"frame":{"x":167,"y":1268,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 54","identifier":"eager.54","value":null,"frame":{"x":166.66666666666666,"y":1290.3333333333333,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 55","identifier":"eager.55","value":null,"frame":{"x":167,"y":1312.6666666666665,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 56","identifier":"eager.56","value":null,"frame":{"x":166.66666666666666,"y":1335,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 57","identifier":"eager.57","value":null,"frame":{"x":167.33333333333334,"y":1357.3333333333333,"w":67.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 58","identifier":"eager.58","value":null,"frame":{"x":166.66666666666666,"y":1379.6666666666665,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 59","identifier":"eager.59","value":null,"frame":{"x":166.66666666666666,"y":1402,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 60","identifier":"eager.60","value":null,"frame":{"x":166.66666666666666,"y":1424.3333333333333,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 61","identifier":"eager.61","value":null,"frame":{"x":168,"y":1446.6666666666665,"w":66.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 62","identifier":"eager.62","value":null,"frame":{"x":167,"y":1469,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 63","identifier":"eager.63","value":null,"frame":{"x":166.66666666666666,"y":1491.3333333333333,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 64","identifier":"eager.64","value":null,"frame":{"x":166.66666666666666,"y":1513.6666666666665,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 65","identifier":"eager.65","value":null,"frame":{"x":166.66666666666666,"y":1536,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 66","identifier":"eager.66","value":null,"frame":{"x":166.66666666666666,"y":1558.3333333333333,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 67","identifier":"eager.67","value":null,"frame":{"x":167,"y":1580.6666666666665,"w":68.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 68","identifier":"eager.68","value":null,"frame":{"x":166.66666666666666,"y":1603,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 69","identifier":"eager.69","value":null,"frame":{"x":166.66666666666666,"y":1625.3333333333333,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 70","identifier":"eager.70","value":null,"frame":{"x":167.33333333333334,"y":1647.6666666666665,"w":67.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 71","identifier":"eager.71","value":null,"frame":{"x":168.66666666666666,"y":1670,"w":65,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 72","identifier":"eager.72","value":null,"frame":{"x":167.66666666666666,"y":1692.3333333333333,"w":67.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 73","identifier":"eager.73","value":null,"frame":{"x":167.33333333333334,"y":1714.6666666666665,"w":67.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 74","identifier":"eager.74","value":null,"frame":{"x":167.66666666666666,"y":1737,"w":67.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 75","identifier":"eager.75","value":null,"frame":{"x":167.33333333333334,"y":1759.3333333333333,"w":67.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 76","identifier":"eager.76","value":null,"frame":{"x":167.33333333333334,"y":1781.6666666666665,"w":67.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 77","identifier":"eager.77","value":null,"frame":{"x":167.66666666666666,"y":1804,"w":67.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 78","identifier":"eager.78","value":null,"frame":{"x":167,"y":1826.3333333333333,"w":68.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 79","identifier":"eager.79","value":null,"frame":{"x":167,"y":1848.6666666666665,"w":68.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 80","identifier":"eager.80","value":null,"frame":{"x":166.66666666666666,"y":1871,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 81","identifier":"eager.81","value":null,"frame":{"x":168,"y":1893.3333333333333,"w":66.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 82","identifier":"eager.82","value":null,"frame":{"x":167,"y":1915.6666666666665,"w":68.66666666666666,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 83","identifier":"eager.83","value":null,"frame":{"x":166.66666666666666,"y":1938,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 84","identifier":"eager.84","value":null,"frame":{"x":166.66666666666666,"y":1960.3333333333333,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 85","identifier":"eager.85","value":null,"frame":{"x":166.66666666666666,"y":1982.6666666666665,"w":69.00000000000003,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 86","identifier":"eager.86","value":null,"frame":{"x":166.66666666666666,"y":2005,"w":69.33333333333334,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 87","identifier":"eager.87","value":null,"frame":{"x":167,"y":2027.3333333333333,"w":68.33333333333331,"h":20.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 88","identifier":"eager.88","value":null,"frame":{"x":166.66666666666666,"y":2049.6666666666665,"w":69.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 89","identifier":"eager.89","value":null,"frame":{"x":166.66666666666666,"y":2072,"w":69.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 90","identifier":"eager.90","value":null,"frame":{"x":166.66666666666666,"y":2094.333333333333,"w":69.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 91","identifier":"eager.91","value":null,"frame":{"x":168,"y":2116.6666666666665,"w":66.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 92","identifier":"eager.92","value":null,"frame":{"x":167,"y":2139,"w":68.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 93","identifier":"eager.93","value":null,"frame":{"x":166.66666666666666,"y":2161.333333333333,"w":69.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 94","identifier":"eager.94","value":null,"frame":{"x":166.66666666666666,"y":2183.6666666666665,"w":69.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 95","identifier":"eager.95","value":null,"frame":{"x":166.66666666666666,"y":2206,"w":69.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 96","identifier":"eager.96","value":null,"frame":{"x":166.66666666666666,"y":2228.333333333333,"w":69.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 97","identifier":"eager.97","value":null,"frame":{"x":167.33333333333334,"y":2250.6666666666665,"w":67.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 98","identifier":"eager.98","value":null,"frame":{"x":166.66666666666666,"y":2273,"w":69.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 99","identifier":"eager.99","value":null,"frame":{"x":166.66666666666666,"y":2295.333333333333,"w":69.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 100","identifier":"eager.100","value":null,"frame":{"x":163,"y":2317.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 101","identifier":"eager.101","value":null,"frame":{"x":164.33333333333334,"y":2340,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 102","identifier":"eager.102","value":null,"frame":{"x":163,"y":2362.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 103","identifier":"eager.103","value":null,"frame":{"x":163,"y":2384.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 104","identifier":"eager.104","value":null,"frame":{"x":163,"y":2407,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 105","identifier":"eager.105","value":null,"frame":{"x":163,"y":2429.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 106","identifier":"eager.106","value":null,"frame":{"x":163,"y":2451.6666666666665,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 107","identifier":"eager.107","value":null,"frame":{"x":163.66666666666666,"y":2474,"w":75.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 108","identifier":"eager.108","value":null,"frame":{"x":163,"y":2496.333333333333,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 109","identifier":"eager.109","value":null,"frame":{"x":163,"y":2518.6666666666665,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 110","identifier":"eager.110","value":null,"frame":{"x":164.33333333333334,"y":2541,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 111","identifier":"eager.111","value":null,"frame":{"x":166,"y":2563.333333333333,"w":70.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 112","identifier":"eager.112","value":null,"frame":{"x":164.66666666666666,"y":2585.6666666666665,"w":73.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 113","identifier":"eager.113","value":null,"frame":{"x":164.33333333333334,"y":2608,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 114","identifier":"eager.114","value":null,"frame":{"x":164.33333333333334,"y":2630.333333333333,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 115","identifier":"eager.115","value":null,"frame":{"x":164.66666666666666,"y":2652.6666666666665,"w":73.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 116","identifier":"eager.116","value":null,"frame":{"x":164.33333333333334,"y":2675,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 117","identifier":"eager.117","value":null,"frame":{"x":165,"y":2697.333333333333,"w":72.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 118","identifier":"eager.118","value":null,"frame":{"x":164.33333333333334,"y":2719.6666666666665,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 119","identifier":"eager.119","value":null,"frame":{"x":164.33333333333334,"y":2742,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 120","identifier":"eager.120","value":null,"frame":{"x":163,"y":2764.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 121","identifier":"eager.121","value":null,"frame":{"x":164.66666666666666,"y":2786.6666666666665,"w":73.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 122","identifier":"eager.122","value":null,"frame":{"x":163.33333333333334,"y":2809,"w":75.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 123","identifier":"eager.123","value":null,"frame":{"x":163,"y":2831.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 124","identifier":"eager.124","value":null,"frame":{"x":163,"y":2853.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 125","identifier":"eager.125","value":null,"frame":{"x":163.33333333333334,"y":2876,"w":75.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 126","identifier":"eager.126","value":null,"frame":{"x":163,"y":2898.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 127","identifier":"eager.127","value":null,"frame":{"x":163.66666666666666,"y":2920.6666666666665,"w":75.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 128","identifier":"eager.128","value":null,"frame":{"x":163,"y":2943,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 129","identifier":"eager.129","value":null,"frame":{"x":163,"y":2965.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 130","identifier":"eager.130","value":null,"frame":{"x":163,"y":2987.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 131","identifier":"eager.131","value":null,"frame":{"x":164.33333333333334,"y":3010,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 132","identifier":"eager.132","value":null,"frame":{"x":163,"y":3032.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 133","identifier":"eager.133","value":null,"frame":{"x":163,"y":3054.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 134","identifier":"eager.134","value":null,"frame":{"x":163,"y":3077,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 135","identifier":"eager.135","value":null,"frame":{"x":163,"y":3099.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 136","identifier":"eager.136","value":null,"frame":{"x":163,"y":3121.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 137","identifier":"eager.137","value":null,"frame":{"x":163.66666666666666,"y":3144,"w":75.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 138","identifier":"eager.138","value":null,"frame":{"x":163,"y":3166.333333333333,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 139","identifier":"eager.139","value":null,"frame":{"x":163,"y":3188.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 140","identifier":"eager.140","value":null,"frame":{"x":163,"y":3211,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 141","identifier":"eager.141","value":null,"frame":{"x":164.33333333333334,"y":3233.333333333333,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 142","identifier":"eager.142","value":null,"frame":{"x":163,"y":3255.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 143","identifier":"eager.143","value":null,"frame":{"x":163,"y":3278,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 144","identifier":"eager.144","value":null,"frame":{"x":162.66666666666666,"y":3300.333333333333,"w":77.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 145","identifier":"eager.145","value":null,"frame":{"x":163,"y":3322.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 146","identifier":"eager.146","value":null,"frame":{"x":163,"y":3345,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 147","identifier":"eager.147","value":null,"frame":{"x":163.33333333333334,"y":3367.333333333333,"w":75.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 148","identifier":"eager.148","value":null,"frame":{"x":163,"y":3389.6666666666665,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 149","identifier":"eager.149","value":null,"frame":{"x":163,"y":3412,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 150","identifier":"eager.150","value":null,"frame":{"x":163,"y":3434.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 151","identifier":"eager.151","value":null,"frame":{"x":164.66666666666666,"y":3456.6666666666665,"w":73.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 152","identifier":"eager.152","value":null,"frame":{"x":163.33333333333334,"y":3479,"w":75.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 153","identifier":"eager.153","value":null,"frame":{"x":163,"y":3501.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 154","identifier":"eager.154","value":null,"frame":{"x":163,"y":3523.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 155","identifier":"eager.155","value":null,"frame":{"x":163,"y":3546,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 156","identifier":"eager.156","value":null,"frame":{"x":163,"y":3568.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 157","identifier":"eager.157","value":null,"frame":{"x":163.66666666666666,"y":3590.6666666666665,"w":75.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 158","identifier":"eager.158","value":null,"frame":{"x":163,"y":3613,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 159","identifier":"eager.159","value":null,"frame":{"x":163,"y":3635.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 160","identifier":"eager.160","value":null,"frame":{"x":163,"y":3657.6666666666665,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 161","identifier":"eager.161","value":null,"frame":{"x":164.33333333333334,"y":3680,"w":73.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 162","identifier":"eager.162","value":null,"frame":{"x":163,"y":3702.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 163","identifier":"eager.163","value":null,"frame":{"x":163,"y":3724.6666666666665,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 164","identifier":"eager.164","value":null,"frame":{"x":163,"y":3747,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 165","identifier":"eager.165","value":null,"frame":{"x":163,"y":3769.333333333333,"w":76.33333333333331,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 166","identifier":"eager.166","value":null,"frame":{"x":163,"y":3791.6666666666665,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 167","identifier":"eager.167","value":null,"frame":{"x":163.33333333333334,"y":3814,"w":75.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 168","identifier":"eager.168","value":null,"frame":{"x":163,"y":3836.333333333333,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 169","identifier":"eager.169","value":null,"frame":{"x":163,"y":3858.6666666666665,"w":76.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 170","identifier":"eager.170","value":null,"frame":{"x":163.66666666666666,"y":3881,"w":75.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 171","identifier":"eager.171","value":null,"frame":{"x":165,"y":3903.333333333333,"w":72.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 172","identifier":"eager.172","value":null,"frame":{"x":163.66666666666666,"y":3925.6666666666665,"w":75.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 173","identifier":"eager.173","value":null,"frame":{"x":163.66666666666666,"y":3948,"w":75.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 174","identifier":"eager.174","value":null,"frame":{"x":164,"y":3970.333333333333,"w":74.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 175","identifier":"eager.175","value":null,"frame":{"x":163.66666666666666,"y":3992.6666666666665,"w":75.00000000000003,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 176","identifier":"eager.176","value":null,"frame":{"x":163.66666666666666,"y":4015,"w":75.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 177","identifier":"eager.177","value":null,"frame":{"x":164,"y":4037.333333333333,"w":74.66666666666666,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 178","identifier":"eager.178","value":null,"frame":{"x":163.66666666666666,"y":4059.6666666666665,"w":75.33333333333334,"h":20.333333333333485},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 179","identifier":"eager.179","value":null,"frame":{"x":163.66666666666666,"y":4082,"w":75.33333333333334,"h":20.33333333333394},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 180","identifier":"eager.180","value":null,"frame":{"x":163,"y":4104.333333333333,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 181","identifier":"eager.181","value":null,"frame":{"x":164.33333333333334,"y":4126.666666666666,"w":73.66666666666666,"h":20.33333333333394},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 182","identifier":"eager.182","value":null,"frame":{"x":163,"y":4149,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 183","identifier":"eager.183","value":null,"frame":{"x":163,"y":4171.333333333333,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 184","identifier":"eager.184","value":null,"frame":{"x":163,"y":4193.666666666666,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 185","identifier":"eager.185","value":null,"frame":{"x":163,"y":4216,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 186","identifier":"eager.186","value":null,"frame":{"x":163,"y":4238.333333333333,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 187","identifier":"eager.187","value":null,"frame":{"x":163.33333333333334,"y":4260.666666666666,"w":75.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 188","identifier":"eager.188","value":null,"frame":{"x":163,"y":4283,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 189","identifier":"eager.189","value":null,"frame":{"x":163,"y":4305.333333333333,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 190","identifier":"eager.190","value":null,"frame":{"x":163,"y":4327.666666666666,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 191","identifier":"eager.191","value":null,"frame":{"x":164.33333333333334,"y":4350,"w":73.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 192","identifier":"eager.192","value":null,"frame":{"x":163,"y":4372.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 193","identifier":"eager.193","value":null,"frame":{"x":163,"y":4394.666666666666,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 194","identifier":"eager.194","value":null,"frame":{"x":163,"y":4417,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 195","identifier":"eager.195","value":null,"frame":{"x":163,"y":4439.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 196","identifier":"eager.196","value":null,"frame":{"x":163,"y":4461.666666666666,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 197","identifier":"eager.197","value":null,"frame":{"x":163.66666666666666,"y":4484,"w":75.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 198","identifier":"eager.198","value":null,"frame":{"x":163,"y":4506.333333333333,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 199","identifier":"eager.199","value":null,"frame":{"x":163,"y":4528.666666666666,"w":76.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 200","identifier":"eager.200","value":null,"frame":{"x":162,"y":4551,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 201","identifier":"eager.201","value":null,"frame":{"x":163,"y":4573.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 202","identifier":"eager.202","value":null,"frame":{"x":162,"y":4595.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 203","identifier":"eager.203","value":null,"frame":{"x":162,"y":4618,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 204","identifier":"eager.204","value":null,"frame":{"x":161.66666666666666,"y":4640.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 205","identifier":"eager.205","value":null,"frame":{"x":162,"y":4662.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 206","identifier":"eager.206","value":null,"frame":{"x":161.66666666666666,"y":4685,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 207","identifier":"eager.207","value":null,"frame":{"x":162.66666666666666,"y":4707.333333333333,"w":77.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 208","identifier":"eager.208","value":null,"frame":{"x":161.66666666666666,"y":4729.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 209","identifier":"eager.209","value":null,"frame":{"x":161.66666666666666,"y":4752,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 210","identifier":"eager.210","value":null,"frame":{"x":163,"y":4774.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 211","identifier":"eager.211","value":null,"frame":{"x":164.66666666666666,"y":4796.666666666666,"w":73.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 212","identifier":"eager.212","value":null,"frame":{"x":163.33333333333334,"y":4819,"w":75.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 213","identifier":"eager.213","value":null,"frame":{"x":163,"y":4841.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 214","identifier":"eager.214","value":null,"frame":{"x":163,"y":4863.666666666666,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 215","identifier":"eager.215","value":null,"frame":{"x":163.33333333333334,"y":4886,"w":75.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 216","identifier":"eager.216","value":null,"frame":{"x":163,"y":4908.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 217","identifier":"eager.217","value":null,"frame":{"x":163.66666666666666,"y":4930.666666666666,"w":75.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 218","identifier":"eager.218","value":null,"frame":{"x":163,"y":4953,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 219","identifier":"eager.219","value":null,"frame":{"x":163,"y":4975.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 220","identifier":"eager.220","value":null,"frame":{"x":162,"y":4997.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 221","identifier":"eager.221","value":null,"frame":{"x":163.33333333333334,"y":5020,"w":75.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 222","identifier":"eager.222","value":null,"frame":{"x":162,"y":5042.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 223","identifier":"eager.223","value":null,"frame":{"x":162,"y":5064.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 224","identifier":"eager.224","value":null,"frame":{"x":162,"y":5087,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 225","identifier":"eager.225","value":null,"frame":{"x":162,"y":5109.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 226","identifier":"eager.226","value":null,"frame":{"x":162,"y":5131.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 227","identifier":"eager.227","value":null,"frame":{"x":162.66666666666666,"y":5154,"w":77.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 228","identifier":"eager.228","value":null,"frame":{"x":162,"y":5176.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 229","identifier":"eager.229","value":null,"frame":{"x":162,"y":5198.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 230","identifier":"eager.230","value":null,"frame":{"x":162,"y":5221,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 231","identifier":"eager.231","value":null,"frame":{"x":163,"y":5243.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 232","identifier":"eager.232","value":null,"frame":{"x":162,"y":5265.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 233","identifier":"eager.233","value":null,"frame":{"x":162,"y":5288,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 234","identifier":"eager.234","value":null,"frame":{"x":161.66666666666666,"y":5310.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 235","identifier":"eager.235","value":null,"frame":{"x":162,"y":5332.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 236","identifier":"eager.236","value":null,"frame":{"x":161.66666666666666,"y":5355,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 237","identifier":"eager.237","value":null,"frame":{"x":162.33333333333334,"y":5377.333333333333,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 238","identifier":"eager.238","value":null,"frame":{"x":161.66666666666666,"y":5399.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 239","identifier":"eager.239","value":null,"frame":{"x":161.66666666666666,"y":5422,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 240","identifier":"eager.240","value":null,"frame":{"x":162,"y":5444.333333333333,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 241","identifier":"eager.241","value":null,"frame":{"x":163.33333333333334,"y":5466.666666666666,"w":75.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 242","identifier":"eager.242","value":null,"frame":{"x":162,"y":5489,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 243","identifier":"eager.243","value":null,"frame":{"x":162,"y":5511.333333333333,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 244","identifier":"eager.244","value":null,"frame":{"x":161.66666666666666,"y":5533.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 245","identifier":"eager.245","value":null,"frame":{"x":162,"y":5556,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 246","identifier":"eager.246","value":null,"frame":{"x":161.66666666666666,"y":5578.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 247","identifier":"eager.247","value":null,"frame":{"x":162.33333333333334,"y":5600.666666666666,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 248","identifier":"eager.248","value":null,"frame":{"x":161.66666666666666,"y":5623,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 249","identifier":"eager.249","value":null,"frame":{"x":161.66666666666666,"y":5645.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 250","identifier":"eager.250","value":null,"frame":{"x":162,"y":5667.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 251","identifier":"eager.251","value":null,"frame":{"x":163.33333333333334,"y":5690,"w":75.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 252","identifier":"eager.252","value":null,"frame":{"x":162,"y":5712.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 253","identifier":"eager.253","value":null,"frame":{"x":162,"y":5734.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 254","identifier":"eager.254","value":null,"frame":{"x":162,"y":5757,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 255","identifier":"eager.255","value":null,"frame":{"x":162,"y":5779.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 256","identifier":"eager.256","value":null,"frame":{"x":162,"y":5801.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 257","identifier":"eager.257","value":null,"frame":{"x":162.33333333333334,"y":5824,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 258","identifier":"eager.258","value":null,"frame":{"x":162,"y":5846.333333333333,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 259","identifier":"eager.259","value":null,"frame":{"x":162,"y":5868.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 260","identifier":"eager.260","value":null,"frame":{"x":161.66666666666666,"y":5891,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 261","identifier":"eager.261","value":null,"frame":{"x":163,"y":5913.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 262","identifier":"eager.262","value":null,"frame":{"x":162,"y":5935.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 263","identifier":"eager.263","value":null,"frame":{"x":161.66666666666666,"y":5958,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 264","identifier":"eager.264","value":null,"frame":{"x":161.66666666666666,"y":5980.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 265","identifier":"eager.265","value":null,"frame":{"x":162,"y":6002.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 266","identifier":"eager.266","value":null,"frame":{"x":161.66666666666666,"y":6025,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 267","identifier":"eager.267","value":null,"frame":{"x":162,"y":6047.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 268","identifier":"eager.268","value":null,"frame":{"x":161.66666666666666,"y":6069.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 269","identifier":"eager.269","value":null,"frame":{"x":161.66666666666666,"y":6092,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 270","identifier":"eager.270","value":null,"frame":{"x":162.33333333333334,"y":6114.333333333333,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 271","identifier":"eager.271","value":null,"frame":{"x":163.66666666666666,"y":6136.666666666666,"w":75.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 272","identifier":"eager.272","value":null,"frame":{"x":162.66666666666666,"y":6159,"w":77.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 273","identifier":"eager.273","value":null,"frame":{"x":162.66666666666666,"y":6181.333333333333,"w":77.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 274","identifier":"eager.274","value":null,"frame":{"x":162.66666666666666,"y":6203.666666666666,"w":77.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 275","identifier":"eager.275","value":null,"frame":{"x":162.66666666666666,"y":6226,"w":77.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 276","identifier":"eager.276","value":null,"frame":{"x":162.33333333333334,"y":6248.333333333333,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 277","identifier":"eager.277","value":null,"frame":{"x":162.66666666666666,"y":6270.666666666666,"w":77.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 278","identifier":"eager.278","value":null,"frame":{"x":162.33333333333334,"y":6293,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 279","identifier":"eager.279","value":null,"frame":{"x":162.33333333333334,"y":6315.333333333333,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 280","identifier":"eager.280","value":null,"frame":{"x":161.66666666666666,"y":6337.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 281","identifier":"eager.281","value":null,"frame":{"x":163,"y":6360,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 282","identifier":"eager.282","value":null,"frame":{"x":162,"y":6382.333333333333,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 283","identifier":"eager.283","value":null,"frame":{"x":161.66666666666666,"y":6404.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 284","identifier":"eager.284","value":null,"frame":{"x":161.66666666666666,"y":6427,"w":79.33333333333334,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 285","identifier":"eager.285","value":null,"frame":{"x":162,"y":6449.333333333333,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 286","identifier":"eager.286","value":null,"frame":{"x":161.66666666666666,"y":6471.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 287","identifier":"eager.287","value":null,"frame":{"x":162,"y":6494,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 288","identifier":"eager.288","value":null,"frame":{"x":161.66666666666666,"y":6516.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 289","identifier":"eager.289","value":null,"frame":{"x":161.66666666666666,"y":6538.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 290","identifier":"eager.290","value":null,"frame":{"x":161.66666666666666,"y":6561,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 291","identifier":"eager.291","value":null,"frame":{"x":163,"y":6583.333333333333,"w":76.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 292","identifier":"eager.292","value":null,"frame":{"x":162,"y":6605.666666666666,"w":78.33333333333331,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 293","identifier":"eager.293","value":null,"frame":{"x":161.66666666666666,"y":6628,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 294","identifier":"eager.294","value":null,"frame":{"x":161.66666666666666,"y":6650.333333333333,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 295","identifier":"eager.295","value":null,"frame":{"x":162,"y":6672.666666666666,"w":78.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 296","identifier":"eager.296","value":null,"frame":{"x":161.66666666666666,"y":6695,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 297","identifier":"eager.297","value":null,"frame":{"x":162.33333333333334,"y":6717.333333333333,"w":77.66666666666666,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 298","identifier":"eager.298","value":null,"frame":{"x":161.66666666666666,"y":6739.666666666666,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Eager 299","identifier":"eager.299","value":null,"frame":{"x":161.66666666666666,"y":6762,"w":79.00000000000003,"h":20.33333333333303},"enabled":true,"children":[]}]},{"role":"AXSlider","type":"Slider","label":"Vertical scroll bar, 10 pages","identifier":null,"value":"0%","frame":{"x":217.33333333333331,"y":50.666666666666664,"w":30,"h":740.3333333333334},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":244.33333333333331,"y":53.666666666666664,"w":3,"h":93.33333333333334},"enabled":true,"children":[]}]},{"role":"AXSlider","type":"Slider","label":"Vertical scroll bar, 10 pages","identifier":null,"value":"0%","frame":{"x":217.33333333333331,"y":50.666666666666664,"w":30,"h":740.3333333333334},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":244.33333333333331,"y":53.666666666666664,"w":3,"h":93.33333333333334},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":"Tab Bar","identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXRadioButton","type":"RadioButton","label":"Simple","identifier":null,"value":0,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"1.circle.fill","identifier":"1.circle.fill","value":null,"frame":{"x":60,"y":801.3333333333334,"w":27.33333333333333,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Heavy","identifier":null,"value":0,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"2.circle.fill","identifier":"2.circle.fill","value":null,"frame":{"x":144.66666666666669,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Deep","identifier":null,"value":0,"frame":{"x":194.33333333333334,"y":795,"w":98.00000000000003,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"3.circle.fill","identifier":"3.circle.fill","value":null,"frame":{"x":229.33333333333334,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Eager","identifier":null,"value":1,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"4.circle.fill","identifier":"4.circle.fill","value":null,"frame":{"x":314,"y":801.3333333333334,"w":27.333333333333314,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]}]}]}]}]}]}]}]}]}]}]}} diff --git a/skills/subtext-ios-sim/test/fixtures/describe-heavy.json b/skills/subtext-ios-sim/test/fixtures/describe-heavy.json new file mode 100644 index 0000000..f6707ee --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-heavy.json @@ -0,0 +1 @@ +{"screen":{"w":402,"h":874},"root":{"role":"AXApplication","type":"Application","label":"Bench","identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":"Heavy","value":null,"frame":{"x":0,"y":62,"w":402,"h":106},"enabled":true,"children":[{"role":"AXHeading","type":"Heading","label":"Heavy","identifier":null,"value":null,"frame":{"x":15.999999999999993,"y":119.66666666666667,"w":100.66666666666666,"h":40.66666666666667},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":-16,"y":-1580,"w":434,"h":1817.6666666666667},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":726.2,"w":402,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":null,"identifier":"AdditionalDimmingOverlay","value":null,"frame":{"x":-120,"y":731,"w":642,"h":286},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":726.2,"w":402,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":726.2,"w":402,"h":147.79999999999995},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":726.2,"w":402,"h":147.79999999999995},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":726.2,"w":402,"h":147.79999999999995},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":726.2,"w":402,"h":147.79999999999995},"enabled":true,"children":[]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":168,"w":370,"h":52},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":168,"w":370,"h":52},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":168,"w":370,"h":52},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"last:none n:0","identifier":"lastTapped","value":null,"frame":{"x":16,"y":168,"w":370,"h":52},"enabled":true,"children":[]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":-16,"y":237.66666666666606,"w":434,"h":27932.333333333336},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":255,"w":370,"h":66.66666666666669},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":255,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":255,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 0, Subtitle 0, $0.00","identifier":"row.0.buy-row.0.fav","value":null,"frame":{"x":16,"y":255,"w":370,"h":66.66666666666669},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":279.3333333333333,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 0","identifier":null,"value":null,"frame":{"x":62,"y":270,"w":49,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 0","identifier":null,"value":null,"frame":{"x":62,"y":292.3333333333333,"w":54.66666666666666,"h":14.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$0.00","identifier":null,"value":null,"frame":{"x":124.66666666666666,"y":292.8333333333333,"w":30.666666666666657,"h":13.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.0.buy","value":null,"frame":{"x":266,"y":271.3333333333333,"w":53.333333333333314,"h":34.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 0","identifier":"row.0.fav","value":null,"frame":{"x":327.3333333333333,"y":279.6666666666667,"w":20.666666666666686,"h":17.333333333333314},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":282.3333333333333,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":321.66666666666663,"w":370,"h":66.66666666666669},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":321.66666666666663,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":321.66666666666663,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 1, Subtitle 1, $1.00","identifier":"row.1.buy-row.1.fav","value":null,"frame":{"x":16,"y":321.66666666666663,"w":370,"h":66.66666666666669},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":345.99999999999994,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 1","identifier":null,"value":null,"frame":{"x":62,"y":336.66666666666663,"w":46,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 1","identifier":null,"value":null,"frame":{"x":62,"y":358.99999999999994,"w":52.66666666666666,"h":14.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$1.00","identifier":null,"value":null,"frame":{"x":122.66666666666666,"y":359.49999999999994,"w":29,"h":13.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.1.buy","value":null,"frame":{"x":266,"y":337.99999999999994,"w":53.333333333333314,"h":34.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 1","identifier":"row.1.fav","value":null,"frame":{"x":327.3333333333333,"y":346.3333333333333,"w":20.666666666666686,"h":17.333333333333314},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":348.99999999999994,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":320.66666666666663,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":388.33333333333337,"w":370,"h":66.66666666666669},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":388.33333333333337,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":388.33333333333337,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 2, Subtitle 2, $2.00","identifier":"row.2.buy-row.2.fav","value":null,"frame":{"x":16,"y":388.33333333333337,"w":370,"h":66.66666666666669},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":412.6666666666667,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 2","identifier":null,"value":null,"frame":{"x":62,"y":403.33333333333337,"w":48.33333333333333,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 2","identifier":null,"value":null,"frame":{"x":62,"y":425.6666666666667,"w":54.33333333333333,"h":14.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$2.00","identifier":null,"value":null,"frame":{"x":124.33333333333333,"y":426.1666666666667,"w":30.66666666666667,"h":13.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.2.buy","value":null,"frame":{"x":266,"y":404.6666666666667,"w":53.333333333333314,"h":34.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 2","identifier":"row.2.fav","value":null,"frame":{"x":327.3333333333333,"y":413.00000000000006,"w":20.666666666666686,"h":17.333333333333314},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":415.6666666666667,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":387.33333333333337,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":455,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":455,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":455,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 3, Subtitle 3, $3.00","identifier":"row.3.buy-row.3.fav","value":null,"frame":{"x":16,"y":455,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":479.3333333333333,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 3","identifier":null,"value":null,"frame":{"x":62,"y":470,"w":49,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 3","identifier":null,"value":null,"frame":{"x":62,"y":492.3333333333333,"w":54.66666666666666,"h":14.333333333333314},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$3.00","identifier":null,"value":null,"frame":{"x":124.66666666666666,"y":492.8333333333333,"w":30.666666666666657,"h":13.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.3.buy","value":null,"frame":{"x":266,"y":471.3333333333333,"w":53.333333333333314,"h":34.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 3","identifier":"row.3.fav","value":null,"frame":{"x":327.3333333333333,"y":479.6666666666667,"w":20.666666666666686,"h":17.333333333333314},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":482.3333333333333,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":454,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":521.6666666666667,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":521.6666666666667,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":521.6666666666667,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 4, Subtitle 4, $4.00","identifier":"row.4.buy-row.4.fav","value":null,"frame":{"x":16,"y":521.6666666666667,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":546.0000000000001,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 4","identifier":null,"value":null,"frame":{"x":62,"y":536.6666666666667,"w":49,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 4","identifier":null,"value":null,"frame":{"x":62,"y":559.0000000000001,"w":55,"h":14.333333333333258},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$4.00","identifier":null,"value":null,"frame":{"x":125,"y":559.5000000000001,"w":30.666666666666657,"h":13.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.4.buy","value":null,"frame":{"x":266,"y":538.0000000000001,"w":53.333333333333314,"h":34.33333333333326},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 4","identifier":"row.4.fav","value":null,"frame":{"x":327.3333333333333,"y":546.3333333333334,"w":20.666666666666686,"h":17.33333333333337},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":549.0000000000001,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":520.6666666666667,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":588.3333333333333,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":588.3333333333333,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":588.3333333333333,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 5, Subtitle 5, $5.00","identifier":"row.5.buy-row.5.fav","value":null,"frame":{"x":16,"y":588.3333333333333,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":612.6666666666666,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 5","identifier":null,"value":null,"frame":{"x":62,"y":603.3333333333333,"w":48.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 5","identifier":null,"value":null,"frame":{"x":62,"y":625.6666666666666,"w":54.66666666666666,"h":14.333333333333258},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$5.00","identifier":null,"value":null,"frame":{"x":124.66666666666666,"y":626.1666666666666,"w":30.666666666666657,"h":13.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.5.buy","value":null,"frame":{"x":266,"y":604.6666666666666,"w":53.333333333333314,"h":34.33333333333326},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 5","identifier":"row.5.fav","value":null,"frame":{"x":327.3333333333333,"y":612.9999999999999,"w":20.666666666666686,"h":17.33333333333337},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":615.6666666666666,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":587.3333333333333,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":655,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":655,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":655,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 6, Subtitle 6, $6.00","identifier":"row.6.buy-row.6.fav","value":null,"frame":{"x":16,"y":655,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":679.3333333333334,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 6","identifier":null,"value":null,"frame":{"x":62,"y":670,"w":49,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 6","identifier":null,"value":null,"frame":{"x":62,"y":692.3333333333334,"w":54.66666666666666,"h":14.333333333333258},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$6.00","identifier":null,"value":null,"frame":{"x":124.66666666666666,"y":692.8333333333334,"w":30.666666666666657,"h":13.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.6.buy","value":null,"frame":{"x":266,"y":671.3333333333334,"w":53.333333333333314,"h":34.33333333333326},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 6","identifier":"row.6.fav","value":null,"frame":{"x":327.3333333333333,"y":679.6666666666666,"w":20.666666666666686,"h":17.33333333333337},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":682.3333333333334,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":654,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":721.6666666666666,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":721.6666666666666,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":721.6666666666666,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 7, Subtitle 7, $7.00","identifier":"row.7.buy-row.7.fav","value":null,"frame":{"x":16,"y":721.6666666666666,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":746,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 7","identifier":null,"value":null,"frame":{"x":62,"y":736.6666666666666,"w":47.66666666666666,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 7","identifier":null,"value":null,"frame":{"x":62,"y":759,"w":54,"h":14.333333333333258},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$7.00","identifier":null,"value":null,"frame":{"x":124,"y":759.5,"w":29.333333333333343,"h":13.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.7.buy","value":null,"frame":{"x":266,"y":738,"w":53.333333333333314,"h":34.33333333333326},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 7","identifier":"row.7.fav","value":null,"frame":{"x":327.3333333333333,"y":746.3333333333333,"w":20.666666666666686,"h":17.33333333333337},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":749,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":720.6666666666666,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":788.3333333333334,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":788.3333333333334,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":788.3333333333334,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 8, Subtitle 8, $8.00","identifier":"row.8.buy-row.8.fav","value":null,"frame":{"x":16,"y":788.3333333333334,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":812.6666666666667,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 8","identifier":null,"value":null,"frame":{"x":62,"y":803.3333333333334,"w":49.33333333333333,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 8","identifier":null,"value":null,"frame":{"x":62,"y":825.6666666666667,"w":54.66666666666666,"h":14.333333333333258},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$8.00","identifier":null,"value":null,"frame":{"x":124.66666666666666,"y":826.1666666666667,"w":30.666666666666657,"h":13.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.8.buy","value":null,"frame":{"x":266,"y":804.6666666666667,"w":53.333333333333314,"h":34.33333333333326},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 8","identifier":"row.8.fav","value":null,"frame":{"x":327.3333333333333,"y":813,"w":20.666666666666686,"h":17.33333333333337},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":815.6666666666667,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":787.3333333333334,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":855,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":855,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":855,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXButton","type":"Button","label":"Row 9, Subtitle 9, $9.00","identifier":"row.9.buy-row.9.fav","value":null,"frame":{"x":16,"y":855,"w":370,"h":66.66666666666663},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"Favorite","identifier":"star.fill","value":null,"frame":{"x":33.66666666666667,"y":879.3333333333334,"w":18.666667938232422,"h":18},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Row 9","identifier":null,"value":null,"frame":{"x":62,"y":870,"w":49,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"Subtitle 9","identifier":null,"value":null,"frame":{"x":62,"y":892.3333333333334,"w":54.66666666666666,"h":14.333333333333258},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"$9.00","identifier":null,"value":null,"frame":{"x":124.66666666666666,"y":892.8333333333334,"w":30.666666666666657,"h":13.333333333333258},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Buy","identifier":"row.9.buy","value":null,"frame":{"x":266,"y":871.3333333333334,"w":53.333333333333314,"h":34.33333333333326},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Favorite 9","identifier":"row.9.fav","value":null,"frame":{"x":327.3333333333333,"y":879.6666666666666,"w":20.666666666666686,"h":17.33333333333337},"enabled":true,"children":[]},{"role":"AXImage","type":"Image","label":"","identifier":"chevron.forward","value":null,"frame":{"x":360.6666666666667,"y":882.3333333333334,"w":7,"h":12},"enabled":true,"children":[]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":62,"y":854,"w":308,"h":1},"enabled":true,"children":[]},{"role":"AXSlider","type":"Slider","label":"Vertical scroll bar, 43 pages","identifier":null,"value":"0%","frame":{"x":369,"y":116,"w":30,"h":675},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":396,"y":781,"w":3,"h":7},"enabled":true,"children":[]}]},{"role":"AXSlider","type":"Slider","label":"Vertical scroll bar, 43 pages","identifier":null,"value":"0%","frame":{"x":369,"y":116,"w":30,"h":675},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":396,"y":781,"w":3,"h":7},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[]}]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":"Tab Bar","identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXRadioButton","type":"RadioButton","label":"Simple","identifier":null,"value":0,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"1.circle.fill","identifier":"1.circle.fill","value":null,"frame":{"x":60,"y":801.3333333333334,"w":27.33333333333333,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Heavy","identifier":null,"value":1,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"2.circle.fill","identifier":"2.circle.fill","value":null,"frame":{"x":144.66666666666669,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Deep","identifier":null,"value":0,"frame":{"x":194.33333333333334,"y":795,"w":98.00000000000003,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"3.circle.fill","identifier":"3.circle.fill","value":null,"frame":{"x":229.33333333333334,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Eager","identifier":null,"value":0,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"4.circle.fill","identifier":"4.circle.fill","value":null,"frame":{"x":314,"y":801.3333333333334,"w":27.333333333333314,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]}]}]}]}]}]}]}]}]}]}]}} diff --git a/skills/subtext-ios-sim/test/fixtures/describe-simple.json b/skills/subtext-ios-sim/test/fixtures/describe-simple.json new file mode 100644 index 0000000..32a7c0f --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-simple.json @@ -0,0 +1 @@ +{"screen":{"w":402,"h":874},"root":{"role":"AXApplication","type":"Application","label":"Bench","identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":16,"y":314.1666564941406,"w":370,"h":224.50003051757812},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"Simple screen","identifier":null,"value":null,"frame":{"x":116.16666666666667,"y":314.1666666666667,"w":169.66666666666663,"h":33.666666666666686},"enabled":true,"children":[]},{"role":"AXStaticText","type":"StaticText","label":"last:none n:0","identifier":"lastTapped","value":null,"frame":{"x":151.83333333333334,"y":363.83333333333337,"w":98.33333333333334,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Primary","identifier":"simple.primary","value":null,"frame":{"x":171.66666666666666,"y":400.3333333333333,"w":58.66666666666666,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXButton","type":"Button","label":"Secondary","identifier":"simple.secondary","value":null,"frame":{"x":160,"y":436.66666666666663,"w":82,"h":20.333333333333314},"enabled":true,"children":[]},{"role":"AXTextField","type":"TextField","label":null,"identifier":"simple.email","value":"Email","frame":{"x":16,"y":473,"w":370,"h":22},"enabled":true,"children":[]},{"role":"AXCheckBox","type":"CheckBox","label":"Notifications","identifier":"simple.toggle","value":"1","frame":{"x":16,"y":511,"w":370,"h":27.66668701171875},"enabled":true,"children":[{"role":"AXStaticText","type":"StaticText","label":"Notifications","identifier":null,"value":null,"frame":{"x":16,"y":514.6666666666666,"w":96,"h":20.33333333333337},"enabled":true,"children":[]},{"role":"AXCheckBox","type":"CheckBox","label":null,"identifier":null,"value":"1","frame":{"x":325,"y":511,"w":63,"h":28},"enabled":true,"children":[],"subrole":"AXSwitch"}],"subrole":"AXSwitch"}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":"Tab Bar","identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":21,"y":791,"w":360,"h":62},"enabled":true,"children":[{"role":"AXRadioButton","type":"RadioButton","label":"Simple","identifier":"1.circle","value":1,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"1.circle.fill","identifier":"1.circle.fill","value":null,"frame":{"x":60,"y":801.3333333333334,"w":27.33333333333333,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Heavy","identifier":"2.circle","value":0,"frame":{"x":109.66666666666669,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"2.circle.fill","identifier":"2.circle.fill","value":null,"frame":{"x":144.66666666666669,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Deep","identifier":"3.circle","value":0,"frame":{"x":194.33333333333334,"y":795,"w":98.00000000000003,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"3.circle.fill","identifier":"3.circle.fill","value":null,"frame":{"x":229.33333333333334,"y":801.3333333333334,"w":27.333333333333343,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"},{"role":"AXRadioButton","type":"RadioButton","label":"Eager","identifier":"4.circle","value":0,"frame":{"x":279,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXImage","type":"Image","label":"4.circle.fill","identifier":"4.circle.fill","value":null,"frame":{"x":314,"y":801.3333333333334,"w":27.333333333333314,"h":27.33333333333337},"enabled":true,"children":[]}],"subrole":"AXTabButton"}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":25,"y":795,"w":98,"h":54},"enabled":true,"children":[]}]}]}]}]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]},{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[{"role":"AXGroup","type":"Group","label":null,"identifier":null,"value":null,"frame":{"x":0,"y":791,"w":402,"h":83},"enabled":true,"children":[]}]}]}]}]}]}]}]}]}]}]}]}]}} diff --git a/skills/subtext-ios-sim/test/fixtures/parity-corpus.json b/skills/subtext-ios-sim/test/fixtures/parity-corpus.json new file mode 100644 index 0000000..1779560 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/parity-corpus.json @@ -0,0 +1,101 @@ +{ + "globals": [ + { + "name": "Row", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"])" + ] + }, + { + "name": "Buy", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"]) button[identifier$=\".buy\"]" + ], + "parentChain": [ + "Row" + ] + }, + { + "name": "RowText", + "selectors": [ + "button:has(\u003e button[identifier$=\".buy\"]) text:not([text^=\"$\"])" + ], + "parentChain": [ + "Row" + ] + }, + { + "name": "TabItem", + "selectors": [ + "other[text=\"Tab Bar\"] tab:not([text=\"Deep\"])" + ] + }, + { + "name": "DeepTab", + "selectors": [ + "tab[text=\"Deep\"]" + ] + }, + { + "name": "EagerTeen", + "selectors": [ + "[identifier^=\"eager.1\"]" + ] + }, + { + "name": "ChildText", + "selectors": [ + "scrollview \u003e text" + ] + }, + { + "name": "ScrollOrHeading", + "selectors": [ + ":is(slider, heading)" + ] + }, + { + "name": "LastTapped", + "selectors": [ + "#lastTapped" + ] + }, + { + "name": "NamedImage", + "selectors": [ + "image[identifier]" + ] + }, + { + "name": "OnSwitch", + "selectors": [ + "switch[value=\"1\"]" + ] + }, + { + "name": "DeepLeaf", + "selectors": [ + "[identifier^=\"deepc.\"] [identifier^=\"deepc.\"] button" + ] + }, + { + "name": "DeepLabel", + "selectors": [ + "other:where([identifier$=\".3\"]) \u003e text", + "[identifier*=\"ep.1\"]" + ] + }, + { + "name": "Field", + "selectors": [ + "textfield[identifier=\"simple.email\"]" + ] + }, + { + "name": "CatchAllButton", + "selectors": [ + "button" + ] + } + ] +} diff --git a/skills/subtext-ios-sim/test/fixtures/sightmap-parity/components.yaml b/skills/subtext-ios-sim/test/fixtures/sightmap-parity/components.yaml new file mode 100644 index 0000000..324a9b6 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/sightmap-parity/components.yaml @@ -0,0 +1,33 @@ +version: 1 +components: + - name: Row + selector: 'button:has(> button[identifier$=".buy"])' + children: + - name: Buy + selector: 'button[identifier$=".buy"]' + - name: RowText + selector: 'text:not([text^="$"])' + - name: TabItem + selector: 'other[text="Tab Bar"] tab:not([text="Deep"])' + - name: DeepTab + selector: 'tab[text="Deep"]' + - name: EagerTeen + selector: '[identifier^="eager.1"]' + - name: ChildText + selector: 'scrollview > text' + - name: ScrollOrHeading + selector: ':is(slider, heading)' + - name: LastTapped + selector: '#lastTapped' + - name: NamedImage + selector: 'image[identifier]' + - name: OnSwitch + selector: 'switch[value="1"]' + - name: DeepLeaf + selector: '[identifier^="deepc."] [identifier^="deepc."] button' + - name: DeepLabel + selector: 'other:where([identifier$=".3"]) > text, [identifier*="ep.1"]' + - name: Field + selector: 'textfield[identifier="simple.email"]' + - name: CatchAllButton + selector: 'button' diff --git a/skills/subtext-ios-sim/test/fixtures/sightmap/components.yaml b/skills/subtext-ios-sim/test/fixtures/sightmap/components.yaml new file mode 100644 index 0000000..83aad07 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/sightmap/components.yaml @@ -0,0 +1,19 @@ +version: 1 +memory: + - The tab bar is present on every screen. +components: + - name: TabBar + selector: 'other[text="Tab Bar"]' + children: + - name: Tab + selector: 'tab' + properties: + - name: title + extract: text + - name: selected + extract: attr=value + - name: LastTapped + selector: '[identifier="lastTapped"]' + properties: + - name: text + extract: text diff --git a/skills/subtext-ios-sim/test/fixtures/sightmap/views/eager.yaml b/skills/subtext-ios-sim/test/fixtures/sightmap/views/eager.yaml new file mode 100644 index 0000000..18ff216 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/sightmap/views/eager.yaml @@ -0,0 +1,10 @@ +version: 1 +views: + - name: Eager + route: "/eager" + components: + - name: EagerButton + selector: 'button[identifier^="eager."]' + properties: + - name: key + extract: attr=identifier diff --git a/skills/subtext-ios-sim/test/fixtures/sightmap/views/heavy.yaml b/skills/subtext-ios-sim/test/fixtures/sightmap/views/heavy.yaml new file mode 100644 index 0000000..1c8b760 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/sightmap/views/heavy.yaml @@ -0,0 +1,33 @@ +version: 1 +views: + - name: Heavy + route: "/heavy" + memory: + - Rows lazy-load; scroll to reach rows past the first screen. + components: + - name: ScreenTitle + selector: 'heading' + - name: ProductRow + selector: 'button:has(> button[identifier$=".buy"])' + properties: + - name: title + extract: Title.text + - name: price + extract: Price.text + - name: favoritable + extract: exists:FavoriteButton + children: + - name: Title + selector: 'text[text^="Row"]' + properties: + - name: text + extract: text + - name: Price + selector: 'text[text^="$"]' + properties: + - name: text + extract: text + - name: BuyButton + selector: 'button[identifier$=".buy"]' + - name: FavoriteButton + selector: 'button[identifier$=".fav"]' diff --git a/skills/subtext-ios-sim/test/fixtures/sightmap/views/simple.yaml b/skills/subtext-ios-sim/test/fixtures/sightmap/views/simple.yaml new file mode 100644 index 0000000..235e647 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/sightmap/views/simple.yaml @@ -0,0 +1,20 @@ +version: 1 +views: + - name: Simple + route: "/simple" + components: + - $ref: TabBar + - name: PrimaryButton + selector: 'button[identifier="simple.primary"]' + - name: SecondaryButton + selector: '[text="Secondary"]' + - name: EmailField + selector: 'textfield[identifier="simple.email"]' + properties: + - name: value + extract: attr=value + - name: NotificationsToggle + selector: 'switch[identifier="simple.toggle"]' + properties: + - name: on + extract: attr=value From 19e7fb5288c4180ce917d19bb4e9dec273609c95 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 14:44:49 -0400 Subject: [PATCH 02/12] subtext-ios-sim: fixes from clean-machine install dry run - README: lead with the HTTPS marketplace URL (shorthand may clone over SSH) - SampleApp build.sh honors SUBTEXT_SIM_CACHE - helper build.sh keeps SwiftPM's artifact cache under SUBTEXT_SIM_CACHE, so a fresh cache dir is a truly cold, self-contained build - console list hides Apple framework errors by default; --all shows them Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/README.md | 12 +++--------- skills/subtext-ios-sim/SKILL.md | 2 +- skills/subtext-ios-sim/fixtures/SampleApp/build.sh | 7 ++++--- skills/subtext-ios-sim/helper/build.sh | 4 ++-- skills/subtext-ios-sim/lib/cli.mjs | 6 +++--- skills/subtext-ios-sim/lib/daemon.mjs | 4 ++-- skills/subtext-ios-sim/lib/logs.mjs | 5 +++-- skills/subtext-ios-sim/test/cli.test.mjs | 3 ++- 8 files changed, 20 insertions(+), 23 deletions(-) diff --git a/skills/subtext-ios-sim/README.md b/skills/subtext-ios-sim/README.md index 5ab5ee8..2a87b3a 100644 --- a/skills/subtext-ios-sim/README.md +++ b/skills/subtext-ios-sim/README.md @@ -14,17 +14,11 @@ Drive and inspect an iOS app in the local iOS Simulator from a coding agent. The The plugin's marketplace (`subtext-marketplace`) is this repo, so add it pinned to the branch and install `subtext` from it: ```sh -claude plugin marketplace add 'fullstorydev/subtext#chandler/ios-live-loop' +claude plugin marketplace add 'https://github.com/fullstorydev/subtext.git#chandler/ios-live-loop' claude plugin install subtext@subtext-marketplace ``` -Or in one step: - -```sh -claude plugin install subtext --marketplace 'fullstorydev/subtext#chandler/ios-live-loop' -``` - -`https://github.com/fullstorydev/subtext.git#chandler/ios-live-loop` works too. Restart Claude Code afterwards. +Use the HTTPS URL as shown. The `fullstorydev/subtext#…` shorthand also works, but it may clone over SSH, which fails without GitHub SSH keys. Restart Claude Code afterwards. **Already have Subtext installed from `main`?** The marketplace name is already taken by the `main` source, and `add` refuses a different source for it. Remove it first (this uninstalls the plugin), then run the commands above: @@ -91,7 +85,7 @@ test/e2e.sh --device "" # drives the sample app on that Every control has an `accessibilityIdentifier` (`login-email`, `login-submit`, `tab-network`, `product-row-42`, `product-detail-add`, `network-send`…). -`build.sh` needs only Xcode: it compiles with `swiftc` and assembles the bundle by hand, so there's no Xcode project. It writes to `~/Library/Caches/subtext-sim/SampleApp/` (set `OUT_DIR` to change that) and prints the `.app` path on stdout. +`build.sh` needs only Xcode: it compiles with `swiftc` and assembles the bundle by hand, so there's no Xcode project. It writes to `$SUBTEXT_SIM_CACHE/SampleApp/`, default `~/Library/Caches/subtext-sim/SampleApp/` (set `OUT_DIR` to change just the app's location) and prints the `.app` path on stdout. To capture sessions with Fullstory, build with your org ID: diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md index 21f83a6..be05233 100644 --- a/skills/subtext-ios-sim/SKILL.md +++ b/skills/subtext-ios-sim/SKILL.md @@ -62,7 +62,7 @@ subtext-sim stop # --shutdown also shuts the simul | `scroll [--in ] [--until ] [--max N]` | One page, or until a target is tappable. `--in` scrolls inside that element. | | `swipe x1 y1 x2 y2 [--duration S]`, `tap-xy x y`, `type `, `key `, `button `, `openurl ` | Raw input in screen points. Keys: `return delete tab escape space` and arrows. Buttons: `home lock siri side`. | | `screenshot --out F.png` | PNG of the screen. | -| `console list [--limit N] [--grep S] [--level L]` / `console get ` | The app's os_log/Logger output (Apple frameworks only at error/fault) and its stdout/stderr (levels `stdout`, `stderr`). | +| `console list [--limit N] [--grep S] [--level L] [--all]` / `console get ` | The app's own os_log/Logger output and its stdout/stderr (levels `stdout`, `stderr`). `--all` adds Apple framework errors and faults, which are hidden by default as noise. | | `session-url` | The Fullstory session URL from the app's logs. | | `run new [--label L]` / `run current` | Start a new run log / print the current run's id and path. | | `run tail [--follow] [--since SEQ] [--run ID]` | Print `.sightmap/.sim-runs/.jsonl` entries; `--follow` keeps streaming. | diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/build.sh b/skills/subtext-ios-sim/fixtures/SampleApp/build.sh index 25f08a7..eab5ff4 100755 --- a/skills/subtext-ios-sim/fixtures/SampleApp/build.sh +++ b/skills/subtext-ios-sim/fixtures/SampleApp/build.sh @@ -5,12 +5,13 @@ # ./build.sh # plain build # FS_ORG_ID= ./build.sh # embed the public Fullstory iOS SDK and record on start # -# Env: OUT_DIR (default ~/Library/Caches/subtext-sim/SampleApp), +# Env: SUBTEXT_SIM_CACHE (default ~/Library/Caches/subtext-sim), OUT_DIR (default $SUBTEXT_SIM_CACHE/SampleApp), # FS_SDK_VERSION (tag of fullstorydev/fullstory-swift-package-ios, default 1.76.0). set -euo pipefail here="$(cd "$(dirname "$0")" && pwd)" -out_dir="${OUT_DIR:-$HOME/Library/Caches/subtext-sim/SampleApp}" +cache_root="${SUBTEXT_SIM_CACHE:-$HOME/Library/Caches/subtext-sim}" +out_dir="${OUT_DIR:-$cache_root/SampleApp}" fs_version="${FS_SDK_VERSION:-1.76.0}" app="$out_dir/SimSample.app" log() { echo "build.sh: $*" >&2; } @@ -25,7 +26,7 @@ cp "$here/Info.plist" "$app/Info.plist" swift_flags=() if [[ -n "${FS_ORG_ID:-}" ]]; then - fs_cache="$HOME/Library/Caches/subtext-sim/fullstory-$fs_version" + fs_cache="$cache_root/fullstory-$fs_version" framework="$fs_cache/FullStory.xcframework/ios-arm64_x86_64-simulator/FullStory.framework" if [[ ! -d "$framework" ]]; then log "fetching Fullstory iOS SDK $fs_version" diff --git a/skills/subtext-ios-sim/helper/build.sh b/skills/subtext-ios-sim/helper/build.sh index 5c356dc..c427613 100755 --- a/skills/subtext-ios-sim/helper/build.sh +++ b/skills/subtext-ios-sim/helper/build.sh @@ -60,10 +60,10 @@ fi SCRATCH="$CACHE_ROOT/.build" echo "build.sh: building subtext-sim-helper $VERSION (first build downloads idb v1.7.7 frameworks, ~10 MB)" >&2 started=$SECONDS -xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" >&2 \ +xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --cache-path "$CACHE_ROOT/.swiftpm" >&2 \ || fail "swift build failed; see the output above" -BUILT="$(xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --show-bin-path)/subtext-sim-helper" +BUILT="$(xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --cache-path "$CACHE_ROOT/.swiftpm" --show-bin-path)/subtext-sim-helper" [[ -x "$BUILT" ]] || fail "build finished but $BUILT is missing" mkdir -p "$OUT_DIR" diff --git a/skills/subtext-ios-sim/lib/cli.mjs b/skills/subtext-ios-sim/lib/cli.mjs index d06b32e..0792fb7 100644 --- a/skills/subtext-ios-sim/lib/cli.mjs +++ b/skills/subtext-ios-sim/lib/cli.mjs @@ -12,7 +12,7 @@ import { latestRunPath, readRun, RUNS_DIR } from './runlog.mjs'; const BIN = fileURLToPath(new URL('../bin/subtext-sim', import.meta.url)); -const BOOLEAN_FLAGS = new Set(['detach', 'json', 'include-offscreen', 'clear', 'help', 'shutdown', 'fix', 'follow']); +const BOOLEAN_FLAGS = new Set(['detach', 'json', 'include-offscreen', 'clear', 'help', 'shutdown', 'fix', 'follow', 'all']); const VALUE_FLAGS = new Set(['device', 'app', 'bundle-id', 'view', 'out', 'tree-out', 'in', 'until', 'max', 'limit', 'grep', 'level', 'duration', 'sightmap-dir', 'label', 'since', 'run']); export const USAGE = `subtext-sim: drive an app in the local iOS Simulator (sightmap-browser style) @@ -28,7 +28,7 @@ Session Observe snapshot [--view V] [--include-offscreen] [--out F] [--tree-out F] [--json] screenshot --out F.png - console list [--limit N] [--grep S] [--level L[,L]] levels: debug info default error fault stdout stderr + console list [--limit N] [--grep S] [--level L[,L]] [--all] levels: debug info default error fault stdout stderr; --all includes Apple frameworks console get session-url Fullstory session URL from the app's logs @@ -141,7 +141,7 @@ export function toRequest({ command, positionals: p, flags }, cwd = process.cwd( return { cmd: 'sessionUrl', args: {} }; case 'console': { const [sub, index] = p; - if (sub === 'list') return { cmd: 'consoleList', args: { limit: flags.limit ? num(flags.limit, '--limit') : 50, grep: flags.grep, level: flags.level } }; + if (sub === 'list') return { cmd: 'consoleList', args: { limit: flags.limit ? num(flags.limit, '--limit') : 50, grep: flags.grep, level: flags.level, all: !!flags.all } }; if (sub === 'get') { need(p, 2, 'console get '); return { cmd: 'consoleGet', args: { index: num(index, 'index') } }; diff --git a/skills/subtext-ios-sim/lib/daemon.mjs b/skills/subtext-ios-sim/lib/daemon.mjs index 7afd16e..b70d905 100644 --- a/skills/subtext-ios-sim/lib/daemon.mjs +++ b/skills/subtext-ios-sim/lib/daemon.mjs @@ -430,8 +430,8 @@ export class SimSession { return this.commands.relaunch.call(this); }, - consoleList({ limit = 50, grep, level }) { - return this.buffer.list({ limit, grep, level }); + consoleList({ limit = 50, grep, level, all }) { + return this.buffer.list({ limit, grep, level, all }); }, consoleGet({ index }) { diff --git a/skills/subtext-ios-sim/lib/logs.mjs b/skills/subtext-ios-sim/lib/logs.mjs index c2f0372..65d87cf 100644 --- a/skills/subtext-ios-sim/lib/logs.mjs +++ b/skills/subtext-ios-sim/lib/logs.mjs @@ -57,11 +57,12 @@ export class ConsoleBuffer { return full; } - list({ limit = 50, grep, level } = {}) { + list({ limit = 50, grep, level, all = false } = {}) { const levels = level ? new Set(String(level).split(',').map((l) => l.trim().toLowerCase())) : null; const needle = grep ? String(grep).toLowerCase() : null; const hits = this.entries.filter((e) => - (!levels || levels.has(e.level)) && (!needle || e.message.toLowerCase().includes(needle) + (all || !(e.subsystem ?? '').startsWith('com.apple.')) + && (!levels || levels.has(e.level)) && (!needle || e.message.toLowerCase().includes(needle) || (e.subsystem ?? '').toLowerCase().includes(needle))); return { total: hits.length, entries: limit > 0 ? hits.slice(-limit) : hits }; } diff --git a/skills/subtext-ios-sim/test/cli.test.mjs b/skills/subtext-ios-sim/test/cli.test.mjs index 0a55157..151fded 100644 --- a/skills/subtext-ios-sim/test/cli.test.mjs +++ b/skills/subtext-ios-sim/test/cli.test.mjs @@ -32,7 +32,8 @@ test('toRequest maps commands onto daemon requests', () => { assert.deepEqual(req(['click', 'SignInButton']), { cmd: 'click', args: { target: 'SignInButton' } }); assert.deepEqual(req(['tap-xy', '10', '20.5']), { cmd: 'tapXY', args: { x: 10, y: 20.5 } }); assert.deepEqual(req(['screenshot', '--out', 'shots/a.png']), { cmd: 'screenshot', args: { out: '/proj/shots/a.png' } }); - assert.deepEqual(req(['console', 'list', '--limit', '5', '--level', 'error']), { cmd: 'consoleList', args: { limit: 5, grep: undefined, level: 'error' } }); + assert.deepEqual(req(['console', 'list', '--limit', '5', '--level', 'error']), { cmd: 'consoleList', args: { limit: 5, grep: undefined, level: 'error', all: false } }); + assert.equal(req(['console', 'list', '--all']).args.all, true); assert.deepEqual(req(['console', 'get', '7']), { cmd: 'consoleGet', args: { index: 7 } }); assert.deepEqual(req(['scroll', 'down', '--until', 'far', '--max', '3']), { cmd: 'scroll', args: { direction: 'down', within: undefined, until: 'far', max: 3 } }); assert.deepEqual(req(['run', 'new', '--label', 'checkout']), { cmd: 'runNew', args: { label: 'checkout' } }); From 022774113f1c5e019623fa7b2062768fc6df9802 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 15:08:07 -0400 Subject: [PATCH 03/12] subtext-ios-sim: doctor flags Fullstory mobile network capture as a manual check Network capture is off by default for mobile and isn't settable via API, so it's the main onboarding roadblock for network data; doctor now says where to turn it on until it can read the setting itself. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/lib/doctor.mjs | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/skills/subtext-ios-sim/lib/doctor.mjs b/skills/subtext-ios-sim/lib/doctor.mjs index 8058ec9..212f768 100644 --- a/skills/subtext-ios-sim/lib/doctor.mjs +++ b/skills/subtext-ios-sim/lib/doctor.mjs @@ -112,23 +112,38 @@ async function checkHelper() { return pass('helper', r.stdout.split('\n').at(-1) || 'built'); } +// TODO: read the org's mobile recording features (`GET /settings/recording/v1/features?platform=mobile`, +// field `ajaxWatcher`) once the Subtext MCP or a configured API key can reach it, and turn this into a +// real pass/fail. Network capture is off by default for mobile, which leaves consumer agents with no requests. +function checkMobileNetworkCapture() { + return { + name: 'fs-network-capture', + ok: null, + todo: true, + detail: 'not checked automatically: Fullstory mobile network capture must be on for network data to reach agents', + fix: 'Fullstory app → Settings → Data Capture → Mobile → turn on network data capture (and add header allowlists / body rules if you need headers or bodies)', + }; +} + const ICON = { true: 'ok ', false: 'FAIL', null: 'PEND' }; function report(checks) { const width = Math.max(...checks.map((c) => c.name.length)); const lines = checks.map((c) => { - const head = `[${ICON[c.ok]}] ${c.name.padEnd(width)} ${c.detail}`; + const head = `[${c.todo ? 'TODO' : ICON[c.ok]}] ${c.name.padEnd(width)} ${c.detail}`; return c.fix ? `${head}\n ${' '.repeat(width)} fix: ${c.fix}` : head; }); const failed = checks.filter((c) => c.ok === false).length; - const pending = checks.filter((c) => c.ok === null).length; - lines.push('', failed ? `${failed} check(s) failed.` : `All checks passed${pending ? ` (${pending} pending)` : ''}.`); + const pending = checks.filter((c) => c.ok === null && !c.todo).length; + const manual = checks.filter((c) => c.todo).length; + const notes = [pending && `${pending} pending`, manual && `${manual} to verify manually`].filter(Boolean).join(', '); + lines.push('', failed ? `${failed} check(s) failed.` : `All checks passed${notes ? ` (${notes})` : ''}.`); console.log(lines.join('\n')); } async function runChecks() { const prerequisites = await Promise.all([checkXcodeSelect(), checkXcodeVersion(), checkSimulators(), checkSwift(), checkArch(), checkMacOS(), checkNode(), checkSightmap()]); - return [...prerequisites.flat(), await checkHelper()]; + return [...prerequisites.flat(), await checkHelper(), checkMobileNetworkCapture()]; } /** Runs every check, prints a report, and returns `[{name, ok, detail, fix}]` (`ok: null` means pending). With `fix`, runs the unprivileged fixes once and re-checks. */ From e293ceebabbb211b9ff6616aedcccd69023fe463 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 15:15:52 -0400 Subject: [PATCH 04/12] subtext-ios-sim: drive React Native apps MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a React Native sample app (fixtures/RNSampleApp: login, React Navigation tabs and stack, a 200-row FlatList, the SwiftUI fixture's testIDs) whose build.sh generates the native project from the pinned RN template and builds a Release .app with the JS bundle embedded. test/e2e.sh takes --fixture to run the same flow against it. Fixes found running it: - RN accessible Views arrive as GenericElement: map them to `other`, and count ones with a testID as interactive so readiness and coverage work. - React Navigation tabs ("Products, tab, 1 of 3" buttons) normalize to `tab "Products"`; their row stands in for the missing tab bar container when checking whether a target is covered. - Controlled TextInputs dropped keys from a burst of HID events; pause 50 ms between keys, and let --clear's deletions settle before typing. - iOS's "Save Password?" sheet lives in another process and emptied the frontmost tree; describe the process under the screen's center instead and render it as an [Overlay] whose buttons can be clicked. - LogBox toasts (Debug builds) are reported with the tap-xy for their ✕ and refuse taps underneath. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- .changeset/ios-sim-react-native.md | 5 + skills/subtext-ios-sim/README.md | 20 +- skills/subtext-ios-sim/SKILL.md | 24 +- .../fixtures/RNSampleApp/.sightmap/.gitignore | 2 + .../RNSampleApp/.sightmap/components.yaml | 9 + .../RNSampleApp/.sightmap/views/login.yaml | 12 + .../RNSampleApp/.sightmap/views/products.yaml | 15 ++ .../fixtures/RNSampleApp/App.js | 206 ++++++++++++++++++ .../fixtures/RNSampleApp/app.json | 4 + .../fixtures/RNSampleApp/babel.config.js | 3 + .../fixtures/RNSampleApp/build.sh | 121 ++++++++++ .../fixtures/RNSampleApp/fullstory.js | 4 + .../fixtures/RNSampleApp/index.js | 9 + .../RNSampleApp/ios/SceneDelegate.swift | 56 +++++ .../fixtures/RNSampleApp/metro.config.js | 11 + .../fixtures/RNSampleApp/package.json | 30 +++ .../Sources/SubtextSimHelper/Describe.swift | 34 ++- .../Sources/SubtextSimHelper/Keyboard.swift | 8 + skills/subtext-ios-sim/lib/daemon.mjs | 50 ++++- skills/subtext-ios-sim/lib/errors.mjs | 2 +- skills/subtext-ios-sim/lib/normalize.mjs | 37 +++- .../subtext-ios-sim/lib/sightmap-bridge.mjs | 12 +- skills/subtext-ios-sim/test/e2e.sh | 44 +++- .../test/fixtures/describe-rn.json | 1 + skills/subtext-ios-sim/test/rn.test.mjs | 73 +++++++ 25 files changed, 760 insertions(+), 32 deletions(-) create mode 100644 .changeset/ios-sim-react-native.md create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/.gitignore create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/App.js create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/app.json create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/babel.config.js create mode 100755 skills/subtext-ios-sim/fixtures/RNSampleApp/build.sh create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/fullstory.js create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/index.js create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/ios/SceneDelegate.swift create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/metro.config.js create mode 100644 skills/subtext-ios-sim/fixtures/RNSampleApp/package.json create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-rn.json create mode 100644 skills/subtext-ios-sim/test/rn.test.mjs diff --git a/.changeset/ios-sim-react-native.md b/.changeset/ios-sim-react-native.md new file mode 100644 index 0000000..2070636 --- /dev/null +++ b/.changeset/ios-sim-react-native.md @@ -0,0 +1,5 @@ +--- +"subtext": minor +--- + +`subtext-ios-sim` now drives React Native apps: role-less `Pressable`s with a `testID` count as interactive, React Navigation's tabs read as `tab` elements and its JS tab bar blocks taps like a native one, typing keeps up with controlled `TextInput`s, and system sheets such as "Save Password?" show up in the snapshot as an `[Overlay]` you can dismiss. Adds a React Native sample app and an e2e run against it (`test/e2e.sh --fixture RNSampleApp`). diff --git a/skills/subtext-ios-sim/README.md b/skills/subtext-ios-sim/README.md index 5ab5ee8..c216432 100644 --- a/skills/subtext-ios-sim/README.md +++ b/skills/subtext-ios-sim/README.md @@ -75,12 +75,13 @@ node ../../bin/subtext-sim stop ```sh node --test test/*.test.mjs # unit tests; a fake helper stands in for the simulator -test/e2e.sh --device "" # drives the sample app on that simulator, then shuts it down +test/e2e.sh --device "" # drives the SwiftUI sample app on that simulator, then shuts it down +test/e2e.sh --device "" --fixture RNSampleApp # the React Native sample app ``` -`e2e.sh` boots the simulator if needed, signs in, scrolls to Product 150, adds it to the cart, and checks the console, a screenshot and the run log. Set `E2E_KEEP_BOOTED=1` to leave the simulator running. +`e2e.sh` boots the simulator if needed, signs in (dismissing iOS's "Save Password?" sheet if it appears), scrolls to Product 150, adds it to the cart, and checks the console, a screenshot and the run log. The React Native run also switches tabs and sends a request. Set `E2E_KEEP_BOOTED=1` to leave the simulator running. Set `SIGHTMAP_BIN` to a sightmap CLI to run the unit tests that need one. -## Sample app +## Sample apps `fixtures/SampleApp/` is a small SwiftUI app, "Subtext Sim Sample" (`dev.subtext.SimSample`), with: @@ -100,3 +101,16 @@ FS_ORG_ID= fixtures/SampleApp/build.sh ``` This downloads the public Fullstory iOS SDK (the binary from [`fullstorydev/fullstory-swift-package-ios`](https://github.com/fullstorydev/fullstory-swift-package-ios), checksum-verified; set `FS_SDK_VERSION` to pick a release), embeds it, adds a `Fullstory` dictionary with `OrgId` and `RecordOnStart` to `Info.plist`, and logs `FULLSTORY_SESSION_URL ` when a session starts. That line is what `subtext-sim session-url` reads. + +### React Native + +`fixtures/RNSampleApp/` is the same app in React Native 0.88 ("Subtext RN Sample", `dev.subtext.RNSample`): React Navigation bottom tabs with a native stack inside the Products tab, a 200-row `FlatList`, and the same `testID`s, so its `.sightmap/` differs from the SwiftUI one only in role-free selectors. Product rows and Add to Cart are `Pressable`s without `accessibilityRole`, like most RN code, so they show up as `other`. + +Requirements on top of Xcode: Node 22.11+, npm, CocoaPods, and network access the first time. Only the JS sources are checked in. `build.sh` generates the native iOS project from the React Native template for the pinned version, then runs `npm install`, `pod install` and `xcodebuild` in `~/Library/Caches/subtext-sim/RNSampleApp/` (set `OUT_DIR` to change that). It prints the `.app` path on stdout. The first build takes several minutes; after a JS change, a rebuild takes about 15 s. + +- `./build.sh` builds Release with the JS bundle embedded, so `subtext-sim start --app` works without Metro. `ios/SceneDelegate.swift` replaces the template's to keep `console.log` in Release builds (`RCTSetLogThreshold`). +- `CONFIGURATION=Debug ./build.sh` builds for Metro. Start Metro from the generated project (the build prints the command) before launching. +- `FS_ORG_ID= ./build.sh` adds `@fullstory/react-native` (its Babel plugins, the `FullStory` pod and a `FullStory` `Info.plist` dictionary) and logs `FULLSTORY_SESSION_URL ` from `FullStory.onReady()`. `FS_RN_VERSION` and `FS_SDK_VERSION` pick the plugin and iOS SDK versions. + +`build.sh` uses target builds rather than `xcodebuild -scheme`, and drops the template's launch storyboard and asset catalog. Both need Xcode's iOS platform component for the SDK version, which a machine with only a Simulator runtime installed doesn't have. + diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md index 21f83a6..f57c9c1 100644 --- a/skills/subtext-ios-sim/SKILL.md +++ b/skills/subtext-ios-sim/SKILL.md @@ -114,10 +114,30 @@ Prefer `[identifier=…]`; fall back to `[text=…]` only for copy that won't ch **SwiftUI gotcha:** `.accessibilityIdentifier` on a container overwrites the identifiers of its children. Put identifiers on the leaf controls (buttons, fields, rows), not on the stack around them. If every child of a view shows the same identifier, that's why. Tab bar items take the identifier set on the `Label` inside `.tabItem`. -## Sample app +## React Native apps + +The loop is the same; RN maps onto the iOS accessibility tree like this: + +- **`testID` is the identifier.** Target it directly (`click login-submit`) and select on it (`[identifier="login-submit"]`). +- **Role-less `Pressable`s show up as `other`**, not `button`; only `accessibilityRole="button"` makes a `button`. Write role-free selectors (`'[identifier^="product-row-"]'`), not `button[identifier=…]`. Accessible elements with a `testID` count as interactive. +- **`` doesn't reach the tree** on the New Architecture: the text shows up, its identifier doesn't. Put the `testID` on an `accessible` `View` around the `Text`, or select on `[text=…]`. +- **React Navigation tabs** become `tab "Products"` (the "tab, 1 of 3" suffix is stripped), so `tab` selectors and `TabBarItem[title="Network"]` work. Rows behind its JS tab bar are refused as covered, like a native one. +- **Labels join child texts with ", "**: a row with "Product 1" and "$6.37" reads `"Product 1, $6.37"`. + +**Embedded bundle vs Metro.** A Release build embeds `main.jsbundle` and runs without Metro; that's the most predictable for driving. A Debug build loads JS from Metro, so start Metro (`npx react-native start` in the app's directory) before `subtext-sim start`; without it the snapshot shows a red "No script URL provided" box. If port 8081 is taken, run Metro with `--port N` and point the app at it: `xcrun simctl spawn defaults write RCT_jsLocation localhost:N`, then `relaunch`. Release builds drop `console.log` below error unless the app calls `RCTSetLogThreshold(.info)` natively. + +**Overlays.** +- In Debug, a `console.error` or `console.warn` raises a LogBox toast over the bottom of the screen. The snapshot gets a `[LogBox]` section with the `tap-xy` that hits the toast's ✕ (the ✕ has no element of its own), and `click` refuses targets under it. Tapping the toast itself opens the full-screen LogBox inspector. To keep toasts away entirely, call `LogBox.ignoreAllLogs()` in the app. +- A system sheet from another process (e.g. iOS's "Save Password?" a moment after a login form goes away) replaces the snapshot with an `[Overlay]` section listing the sheet's buttons; `click 'button[text="Not Now"]'` dismisses it. + +**`console.log`** reaches `console list` as `info` lines from subsystem `com.facebook.react.log` (`console list --grep `). + +## Sample apps `/fixtures/SampleApp/build.sh` builds a small SwiftUI app (login form, 200-row product list, tab bar, a JSON POST button) and prints the `.app` path. Set `FS_ORG_ID=` to build it with the Fullstory SDK, which enables `session-url` and network. +`/fixtures/RNSampleApp/build.sh` builds the same app in React Native (React Navigation tabs and stack, a `FlatList`, the same `testID`s) with its JS bundle embedded. It needs Node 22.11+ and CocoaPods, and takes several minutes the first time. `FS_ORG_ID` adds the Fullstory React Native SDK; `CONFIGURATION=Debug` builds for Metro instead. + ## Known limits - Simulator only, Apple Silicon only. No physical devices. @@ -125,4 +145,4 @@ Prefer `[identifier=…]`; fall back to `[text=…]` only for copy that won't ch - Network isn't live, and needs the Fullstory SDK in the app plus network capture enabled for the org. - The helper uses Apple private frameworks; a new Xcode release may need a plugin update. `doctor` says so when the helper fails to build. - On iOS 26 a SwiftUI tab bar shows up as one `other "Tab Bar"` element without its tabs; switch tabs with `tap-xy` from a screenshot. -- Scrolling a long list to a far row takes about 3 s per screenful. +- Scrolling a long list to a far row takes about 3 s per screenful (4–5 s for a React Native `FlatList`, whose tree is larger). diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/.gitignore b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/.gitignore new file mode 100644 index 0000000..a47a5b8 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/.gitignore @@ -0,0 +1,2 @@ +.sim-session +.sim-runs/ diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml new file mode 100644 index 0000000..9bb8fb1 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml @@ -0,0 +1,9 @@ +version: 1 +memory: + - Any email containing "@" and any non-empty password signs in. +components: + - name: TabBarItem + selector: 'tab' + properties: + - name: title + extract: text diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml new file mode 100644 index 0000000..c2ac1c1 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml @@ -0,0 +1,12 @@ +version: 1 +views: + - name: Login + components: + - name: EmailField + selector: '[identifier="login-email"]' + - name: PasswordField + selector: '[identifier="login-password"]' + - name: SignInButton + selector: '[identifier="login-submit"]' + - name: LoginError + selector: '[identifier="login-error"]' diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml new file mode 100644 index 0000000..2eb9233 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml @@ -0,0 +1,15 @@ +version: 1 +views: + - name: Products + components: + - name: ProductRow + selector: '[identifier^="product-row-"]' + properties: + - name: title + extract: text + - name: ProductDetail + components: + - name: ProductName + selector: '[identifier="product-detail-name"]' + - name: AddToCartButton + selector: '[identifier="product-detail-add"]' diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/App.js b/skills/subtext-ios-sim/fixtures/RNSampleApp/App.js new file mode 100644 index 0000000..0846971 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/App.js @@ -0,0 +1,206 @@ +import React, { createContext, useContext, useState } from 'react'; +import { FlatList, Pressable, StyleSheet, Text, TextInput, View } from 'react-native'; +import { NavigationContainer } from '@react-navigation/native'; +import { createBottomTabNavigator } from '@react-navigation/bottom-tabs'; +import { createNativeStackNavigator } from '@react-navigation/native-stack'; +import { SafeAreaProvider } from 'react-native-safe-area-context'; +import { logSessionUrl } from './fullstory'; + +const Session = createContext(null); +const Tabs = createBottomTabNavigator(); +const ProductStack = createNativeStackNavigator(); + +const products = Array.from({ length: 200 }, (_, i) => { + const id = i + 1; + const cents = String((id * 37) % 100).padStart(2, '0'); + return { id, name: `Product ${id}`, price: `$${5 + (id % 95)}.${cents}` }; +}); + +console.log('Subtext RN Sample launched'); +logSessionUrl(); + +export default function App() { + const [email, setEmail] = useState(null); + return ( + + + {email ? : } + + + ); +} + +function LoginScreen() { + const { setEmail } = useContext(Session); + const [email, setEmailText] = useState(''); + const [password, setPassword] = useState(''); + const [error, setError] = useState(null); + + const signIn = () => { + if (!email.includes('@') || !password) { + setError('Enter an email and password'); + console.error('sign-in rejected: missing email or password'); + return; + } + console.log(`signed in as ${email}`); + setEmail(email); + }; + + return ( + + + + + {error && } + + Sign In + + + ); +} + +function MainTabs() { + return ( + + + + + + + + ); +} + +function ProductsStack() { + return ( + + + ({ title: route.params.product.name })} /> + + ); +} + +// Rows and the Add to Cart button deliberately have no accessibilityRole, like +// most RN apps: they show up as `other`, so their selectors match on identifier. +function ProductList({ navigation }) { + return ( + String(p.id)} + renderItem={({ item }) => ( + navigation.navigate('ProductDetail', { product: item })} + testID={`product-row-${item.id}`}> + {item.name} + {item.price} + + )} + /> + ); +} + +function ProductDetail({ route }) { + const { product } = route.params; + const [inCart, setInCart] = useState(false); + const add = () => { + setInCart(true); + console.log(`added ${product.name} to cart`); + }; + return ( + + + + + {inCart ? 'Added' : 'Add to Cart'} + + + ); +} + +// With the New Architecture, a 's testID never reaches the accessibility +// tree (its text is exposed through separate elements); an accessible View carries both. +function Label({ testID, style, children }) { + return ( + + {children} + + ); +} + +let requestCount = 0; + +function NetworkScreen() { + const [status, setStatus] = useState('Idle'); + const send = async () => { + requestCount += 1; + const path = `subtext-sim/order-${requestCount}`; + setStatus('Sending…'); + console.log(`POST /anything/${path}`); + try { + const res = await fetch(`https://httpbin.org/anything/${path}`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ order: requestCount, items: ['Product 1', 'Product 2'] }), + }); + setStatus(`HTTP ${res.status}`); + } catch (e) { + setStatus(`failed: ${e.message}`); + } + console.log(`POST /anything/${path} done`); + }; + return ( + + + Send Request + + + + ); +} + +function AccountScreen() { + const { email, setEmail } = useContext(Session); + return ( + + + { console.log('signed out'); setEmail(null); }} accessibilityRole="button" + testID="account-sign-out"> + Sign Out + + + ); +} + +const styles = StyleSheet.create({ + screen: { flex: 1, backgroundColor: 'white', padding: 24, gap: 16 }, + centered: { justifyContent: 'center', alignItems: 'stretch' }, + title: { fontSize: 30, fontWeight: 'bold', textAlign: 'center' }, + price: { fontSize: 22, textAlign: 'center' }, + input: { borderWidth: 1, borderColor: '#ccc', borderRadius: 8, padding: 10, fontSize: 17 }, + error: { color: 'red' }, + button: { backgroundColor: '#007aff', borderRadius: 10, padding: 12, alignItems: 'center' }, + disabled: { opacity: 0.4 }, + buttonText: { color: 'white', fontSize: 17, fontWeight: '600' }, + link: { color: '#007aff', fontSize: 17, textAlign: 'center' }, + row: { flexDirection: 'row', justifyContent: 'space-between', padding: 16, backgroundColor: 'white', + borderBottomWidth: StyleSheet.hairlineWidth, borderColor: '#ccc' }, + secondary: { color: '#888' }, +}); diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/app.json b/skills/subtext-ios-sim/fixtures/RNSampleApp/app.json new file mode 100644 index 0000000..6f58f00 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/app.json @@ -0,0 +1,4 @@ +{ + "name": "RNSampleApp", + "displayName": "Subtext RN Sample" +} diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/babel.config.js b/skills/subtext-ios-sim/fixtures/RNSampleApp/babel.config.js new file mode 100644 index 0000000..f7b3da3 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/babel.config.js @@ -0,0 +1,3 @@ +module.exports = { + presets: ['module:@react-native/babel-preset'], +}; diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/build.sh b/skills/subtext-ios-sim/fixtures/RNSampleApp/build.sh new file mode 100755 index 0000000..3e1e20f --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/build.sh @@ -0,0 +1,121 @@ +#!/bin/bash +# Builds the Subtext RN Sample app (React Native) for the iOS Simulator and prints the .app path. +# +# ./build.sh # Release: JS bundle embedded, runs without Metro +# CONFIGURATION=Debug ./build.sh # Debug: loads JS from Metro (start it as printed), dev menu + LogBox on +# FS_ORG_ID= ./build.sh # also add the Fullstory React Native SDK and record on start +# +# Needs Node 22.11+, CocoaPods, Xcode and network access on the first run. Only the +# JS sources live here: the native project comes from the React Native template for +# the pinned version, generated into the work dir along with node_modules and Pods. +# +# Env: OUT_DIR (default ~/Library/Caches/subtext-sim/RNSampleApp), +# FS_RN_VERSION (@fullstory/react-native, default 1.10.3), FS_SDK_VERSION (Fullstory iOS SDK, default 1.76.0). +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +work="${OUT_DIR:-$HOME/Library/Caches/subtext-sim/RNSampleApp}" +configuration="${CONFIGURATION:-Release}" +project="$work/project" +log() { echo "build.sh: $*" >&2; } +export LANG="${LANG:-en_US.UTF-8}" LC_ALL="${LC_ALL:-en_US.UTF-8}" + +for tool in node npm pod xcodebuild; do + command -v "$tool" >/dev/null || { log "$tool not found (needs Node 22.11+, CocoaPods and Xcode)"; exit 1; } +done + +pkg() { node -p "require('$here/package.json').$1"; } +rn_version="$(pkg 'dependencies["react-native"]')" +cli_version="$(pkg 'devDependencies["@react-native-community/cli"]')" + +if [[ "$(cat "$project/.rn-version" 2>/dev/null)" != "$rn_version" ]]; then + log "generating the React Native $rn_version iOS project" + rm -rf "$project" "$work/template" + mkdir -p "$work/template" + (cd "$work/template" && npx -y "@react-native-community/cli@$cli_version" init RNSampleApp \ + --version "$rn_version" --skip-git-init --skip-install --install-pods false >/dev/null) + mkdir -p "$project" + mv "$work/template/RNSampleApp/ios" "$project/ios" + rm -rf "$work/template" + # ibtool and actool need Xcode's iOS platform component, not just a Simulator + # runtime; a plist launch screen and no app icon avoid both. + sed -i '' -e '/LaunchScreen.storyboard in Resources/d' -e '/Images.xcassets in Resources/d' \ + "$project/ios/RNSampleApp.xcodeproj/project.pbxproj" + /usr/libexec/PlistBuddy \ + -c "Delete :UILaunchStoryboardName" \ + -c "Add :UILaunchScreen dict" \ + -c "Set :CFBundleDisplayName Subtext RN Sample" \ + "$project/ios/RNSampleApp/Info.plist" >/dev/null + echo "$rn_version" > "$project/.rn-version" +fi + +cp "$here"/{App.js,fullstory.js,index.js,app.json,babel.config.js,metro.config.js} "$project/" +cp "$here/ios/SceneDelegate.swift" "$project/ios/RNSampleApp/SceneDelegate.swift" + +package_json="$(cat "$here/package.json")" +babel_config="$(cat "$here/babel.config.js")" +if [[ -n "${FS_ORG_ID:-}" ]]; then + package_json="$(node -e ' + const p = JSON.parse(require("fs").readFileSync(0, "utf8")); + p.dependencies["@fullstory/react-native"] = process.env.FS_RN_VERSION || "1.10.3"; + console.log(JSON.stringify(p, null, 2));' <<<"$package_json")" + cat > "$project/fullstory.js" <<'JS' +import FullStory from '@fullstory/react-native'; + +export function logSessionUrl() { + FullStory.onReady().then(({ replayStartUrl }) => console.log(`FULLSTORY_SESSION_URL ${replayStartUrl}`)); +} +JS + babel_config="module.exports = { + presets: ['module:@react-native/babel-preset'], + plugins: ['@fullstory/react-native', ['@fullstory/annotate-react', { native: true }]], +};" +fi +[[ "$package_json" == "$(cat "$project/package.json" 2>/dev/null)" ]] || echo "$package_json" > "$project/package.json" +echo "$babel_config" > "$project/babel.config.js" + +cd "$project" +if [[ "$(cat node_modules/.package.json 2>/dev/null)" != "$package_json" ]]; then + log "npm install" + npm install --no-audit --no-fund --loglevel=error >&2 + echo "$package_json" > node_modules/.package.json +fi + +plist="ios/RNSampleApp/Info.plist" +/usr/libexec/PlistBuddy -c "Delete :FullStory" "$plist" >/dev/null 2>&1 || true +if [[ -n "${FS_ORG_ID:-}" ]]; then + /usr/libexec/PlistBuddy \ + -c "Add :FullStory dict" \ + -c "Add :FullStory:OrgId string $FS_ORG_ID" \ + -c "Add :FullStory:RecordOnStart bool true" \ + "$plist" >/dev/null + log "including the Fullstory React Native SDK" +fi + +podfile="$(grep -v "pod 'FullStory'" ios/Podfile)" +if [[ -n "${FS_ORG_ID:-}" ]]; then + fs_pod=" pod 'FullStory', :http => 'https://ios-releases.fullstory.com/fullstory-${FS_SDK_VERSION:-1.76.0}-xcframework.tar.gz'" + podfile="$(awk -v pod="$fs_pod" '{ print } /use_native_modules!/ { print pod }' <<<"$podfile")" +fi +[[ "$podfile" == "$(cat ios/Podfile)" ]] || echo "$podfile" > ios/Podfile + +if ! diff -q ios/Podfile.lock ios/Pods/Manifest.lock >/dev/null 2>&1 \ + || [[ package.json -nt ios/Pods/Manifest.lock || ios/Podfile -nt ios/Pods/Manifest.lock ]]; then + log "pod install" + (cd ios && RCT_NEW_ARCH_ENABLED=1 pod install >&2) +fi + +log "xcodebuild $configuration (the first build takes several minutes)" +# Target builds instead of `-scheme`: scheme builds also refuse to start when the +# iOS platform component matching the SDK isn't installed, even for the Simulator. +products="$work/build" +build=(-configuration "$configuration" -sdk iphonesimulator ARCHS="$(uname -m)" ONLY_ACTIVE_ARCH=YES + SYMROOT="$products" DISABLE_MANUAL_TARGET_ORDER_BUILD_WARNING=YES -quiet build) +xcodebuild -project ios/Pods/Pods.xcodeproj -alltargets "${build[@]}" >&2 +xcodebuild -project ios/RNSampleApp.xcodeproj -target RNSampleApp \ + PRODUCT_BUNDLE_IDENTIFIER=dev.subtext.RNSample "${build[@]}" >&2 + +if [[ "$configuration" == Debug ]]; then + log "Debug build: start Metro before launching the app: (cd '$project' && npx react-native start)" +fi +echo "$products/$configuration-iphonesimulator/RNSampleApp.app" diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/fullstory.js b/skills/subtext-ios-sim/fixtures/RNSampleApp/fullstory.js new file mode 100644 index 0000000..9825ed4 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/fullstory.js @@ -0,0 +1,4 @@ +// build.sh replaces this with a version that imports @fullstory/react-native when FS_ORG_ID is set. +export function logSessionUrl() { + console.log('built without the Fullstory SDK'); +} diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/index.js b/skills/subtext-ios-sim/fixtures/RNSampleApp/index.js new file mode 100644 index 0000000..9b73932 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/index.js @@ -0,0 +1,9 @@ +/** + * @format + */ + +import { AppRegistry } from 'react-native'; +import App from './App'; +import { name as appName } from './app.json'; + +AppRegistry.registerComponent(appName, () => App); diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/ios/SceneDelegate.swift b/skills/subtext-ios-sim/fixtures/RNSampleApp/ios/SceneDelegate.swift new file mode 100644 index 0000000..4f9ebff --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/ios/SceneDelegate.swift @@ -0,0 +1,56 @@ +import React +import React_RCTAppDelegate +import ReactAppDependencyProvider +import UIKit + +class SceneDelegate: RCTDefaultReactNativeFactoryDelegate, UIWindowSceneDelegate { + var window: UIWindow? + var reactNativeFactory: RCTReactNativeFactory? + + func scene( + _ scene: UIScene, + willConnectTo session: UISceneSession, + options connectionOptions: UIScene.ConnectionOptions + ) { + guard let windowScene = scene as? UIWindowScene else { + return + } + + // Release builds drop console.log/info/warn (below RCTLogLevelError); keep them for `console list`. + RCTSetLogThreshold(.info) + dependencyProvider = RCTAppDependencyProvider() + reactNativeFactory = RCTReactNativeFactory(delegate: self) + window = UIWindow(windowScene: windowScene) + + #if DEBUG + let devMenuConfiguration = RCTDevMenuConfiguration( + devMenuEnabled: true, + shakeGestureEnabled: true, + keyboardShortcutsEnabled: true + ) + reactNativeFactory?.devMenuConfiguration = devMenuConfiguration + #endif + + reactNativeFactory?.startReactNative( + withModuleName: "RNSampleApp", + in: window, + connectionOptions: connectionOptions + ) + } + + func scene(_ scene: UIScene, openURLContexts URLContexts: Set) { + RCTLinkingManager.scene(scene, openURLContexts: URLContexts) + } + + func scene(_ scene: UIScene, continue userActivity: NSUserActivity) { + RCTLinkingManager.scene(scene, continue: userActivity) + } + + override func bundleURL() -> URL? { + #if DEBUG + RCTBundleURLProvider.sharedSettings().jsBundleURL(forBundleRoot: "index") + #else + Bundle.main.url(forResource: "main", withExtension: "jsbundle") + #endif + } +} diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/metro.config.js b/skills/subtext-ios-sim/fixtures/RNSampleApp/metro.config.js new file mode 100644 index 0000000..2a0a21c --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/metro.config.js @@ -0,0 +1,11 @@ +const { getDefaultConfig, mergeConfig } = require('@react-native/metro-config'); + +/** + * Metro configuration + * https://reactnative.dev/docs/metro + * + * @type {import('@react-native/metro-config').MetroConfig} + */ +const config = {}; + +module.exports = mergeConfig(getDefaultConfig(__dirname), config); diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/package.json b/skills/subtext-ios-sim/fixtures/RNSampleApp/package.json new file mode 100644 index 0000000..c3bdfe2 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/package.json @@ -0,0 +1,30 @@ +{ + "name": "rn-sample-app", + "version": "0.0.1", + "private": true, + "scripts": { + "start": "react-native start", + "build": "./build.sh" + }, + "dependencies": { + "@react-navigation/bottom-tabs": "7.21.0", + "@react-navigation/native": "7.5.1", + "@react-navigation/native-stack": "7.21.0", + "react": "19.3.0", + "react-native": "0.88.1", + "react-native-safe-area-context": "5.11.0", + "react-native-screens": "4.30.1" + }, + "devDependencies": { + "@babel/core": "^7.25.2", + "@babel/preset-env": "^7.25.3", + "@babel/runtime": "^7.25.0", + "@react-native-community/cli": "20.2.0", + "@react-native-community/cli-platform-ios": "20.2.0", + "@react-native/babel-preset": "0.88.1", + "@react-native/metro-config": "0.88.1" + }, + "engines": { + "node": ">= 22.11.0" + } +} diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift index b15aacc..e4d3ba8 100644 --- a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Describe.swift @@ -13,17 +13,33 @@ enum Describe { static func run(_ connection: SimulatorConnection, udid: String) async throws -> [String: Any] { let options = AccessibilityRequestOptions(format: .nested, keys: keys) let response = try await connection.automation.describe(.frontmost, options: options) - let elements = response.elements.elements - let root: [String: Any] - if elements.count == 1 { - root = node(elements[0]) - } else { - root = syntheticRoot(children: elements.map(node)) + let root = tree(response.elements.elements) + if hasChildren(root) { + return ["screen": screen(connection.simulator, root: root), "root": root] } - if (root["children"] as? [Any])?.isEmpty ?? true { - throw HelperError(code: "ax_disabled", message: String(format: accessibilityDisabledMessage, udid)) + // A system sheet from another process (e.g. "Save Password?") leaves the + // frontmost app's tree empty; describe the process owning the screen's center. + let size = screen(connection.simulator, root: root) + let center = CGPoint(x: (size["w"] as? Double ?? 0) / 2, y: (size["h"] as? Double ?? 0) / 2) + let probe = AccessibilityRequestOptions(format: .nested, keys: [.pid]) + if let hit = try? await connection.automation.describe(.point(center), options: probe).elements.elements.first, + let pid = hit.pid.flatMap({ $0 }), + let overlay = try? await connection.automation.describe(.application(pid: pid_t(pid)), options: options) + { + let overlayRoot = tree(overlay.elements.elements) + if hasChildren(overlayRoot) { + return ["screen": size, "root": overlayRoot, "overlay": true] + } } - return ["screen": screen(connection.simulator, root: root), "root": root] + throw HelperError(code: "ax_disabled", message: String(format: accessibilityDisabledMessage, udid)) + } + + private static func tree(_ elements: [AccessibilityDocumentElement]) -> [String: Any] { + elements.count == 1 ? node(elements[0]) : syntheticRoot(children: elements.map(node)) + } + + private static func hasChildren(_ root: [String: Any]) -> Bool { + !((root["children"] as? [Any])?.isEmpty ?? true) } private static func node(_ element: AccessibilityDocumentElement) -> [String: Any] { diff --git a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift index 552d1d1..6e2b5a4 100644 --- a/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift +++ b/skills/subtext-ios-sim/helper/Sources/SubtextSimHelper/Keyboard.swift @@ -1,9 +1,14 @@ import FBSimulatorControl +import Foundation /// USB HID usage IDs (keyboard page 0x07) for a US keyboard layout. enum Keyboard { static let leftShift: UInt32 = 225 + /// Controlled React Native TextInputs echo every keystroke through JS and drop + /// keys that arrive while an echo is in flight; this pause lets each one land. + static let interKeyDelay: TimeInterval = 0.05 + static let namedKeys: [String: UInt32] = [ "return": 40, "enter": 40, "escape": 41, "esc": 41, @@ -41,6 +46,9 @@ enum Keyboard { static func events(typing text: String) throws -> [SimulatorHIDEvent] { var events: [SimulatorHIDEvent] = [] for character in text { + if !events.isEmpty { + events.append(.delay(interKeyDelay)) + } if let code = unshifted[character] { events.append(.keyboard(direction: .down, keyCode: code)) events.append(.keyboard(direction: .up, keyCode: code)) diff --git a/skills/subtext-ios-sim/lib/daemon.mjs b/skills/subtext-ios-sim/lib/daemon.mjs index 7afd16e..a17338d 100644 --- a/skills/subtext-ios-sim/lib/daemon.mjs +++ b/skills/subtext-ios-sim/lib/daemon.mjs @@ -5,7 +5,7 @@ import { tmpdir } from 'node:os'; import { dirname, join, resolve as resolvePath } from 'node:path'; import { createInterface } from 'node:readline'; import { BridgeError, annotate, centerOf, render, resolve as resolveQuery, toComponentTree, writeSnap } from './sightmap-bridge.mjs'; -import { normalize } from './normalize.mjs'; +import { logBoxToast, normalize } from './normalize.mjs'; import { SimError, toWire } from './errors.mjs'; import { ConsoleBuffer, LogCollector } from './logs.mjs'; import { HelperClient } from './helper-client.mjs'; @@ -76,6 +76,24 @@ const OBSTRUCTING_LABELS = new Set(['Tab Bar', 'Toolbar']); const obstructs = (o) => OBSTRUCTING.has(o.tag) || (o.tag === 'other' && OBSTRUCTING_LABELS.has(o.label)); const barName = (o) => (OBSTRUCTING.has(o.tag) ? o.tag : o.label.toLowerCase()); +/** + * A JS tab bar (React Navigation) has no container element, only its tabs; the + * band from the top of a bottom row of tabs to the screen's bottom stands in. + */ +function tabBand({ nodes, screen }) { + const tabs = nodes.filter((n) => n.tag === 'tab' && n.visible && !hasAncestorTag(n, 'tabbar')); + if (!tabs.length) return null; + const y = Math.min(...tabs.map((t) => t.frame.y)); + if (y < screen.h / 2) return null; + return { tag: 'tabbar', label: null, identifier: null, value: null, enabled: true, visible: true, children: [], + frame: { x: 0, y, w: screen.w, h: screen.h - y } }; +} + +function hasAncestorTag(node, tag) { + for (let p = node.parent; p; p = p.parent) if (p.tag === tag) return true; + return false; +} + /** Throws unless the node's center is on screen, uncovered by bars or the keyboard, and enabled. */ export function assertActionable(annotated, node, target) { const { screen } = annotated; @@ -87,8 +105,15 @@ export function assertActionable(annotated, node, target) { `run \`subtext-sim scroll ${dir} --until ${quote(target)}\`, then retry`); } const c = centerOf(node); - const cover = annotated.nodes.find((o) => obstructs(o) && o.visible && o !== node && !isAncestor(o, node) && !isAncestor(node, o) - && c.x >= o.frame.x && c.x <= o.frame.x + o.frame.w && c.y >= o.frame.y && c.y <= o.frame.y + o.frame.h); + const band = node.tag === 'tab' ? null : tabBand(annotated); + const logBox = logBoxToast(annotated); + const covers = (o) => c.x >= o.frame.x && c.x <= o.frame.x + o.frame.w && c.y >= o.frame.y && c.y <= o.frame.y + o.frame.h; + if (logBox && logBox.node !== node && covers(logBox.node)) { + throw new SimError('covered', `${desc} is covered by a React Native LogBox toast (${describeNode(logBox.node)})`, + `dismiss it with \`subtext-sim tap-xy ${logBox.dismiss.x} ${logBox.dismiss.y}\` (its ✕), then retry`); + } + const cover = [...annotated.nodes, ...(band ? [band] : [])].find((o) => obstructs(o) && o.visible && o !== node && !isAncestor(o, node) && !isAncestor(node, o) + && covers(o)); if (cover) { const hint = cover.tag === 'keyboard' ? 'dismiss the keyboard (`subtext-sim key return`) or scroll, then retry' @@ -142,7 +167,7 @@ const ACTION_BY_CMD = { export class SimSession { constructor({ udid, deviceName = null, bundleId, appPath = null, processName, sightmapDir, runDir, helper, simctl = realSimctl, logs = null, buffer = new ConsoleBuffer(), corpus = new CorpusCache(sightmapDir), log = () => {}, timing = {} }) { Object.assign(this, { udid, deviceName, bundleId, appPath, processName, sightmapDir, runDir, helper, simctl, logs, buffer, corpus, log }); - this.timing = { settleMs: 350, scrollSettleMs: 150, fillReadbackMs: 2000, readyTimeoutMs: 20_000, ...timing }; + this.timing = { settleMs: 350, scrollSettleMs: 150, overlaySettleMs: 800, fillReadbackMs: 2000, readyTimeoutMs: 20_000, ...timing }; this.last = null; this.lastView = null; this.lastTarget = null; @@ -164,9 +189,22 @@ export class SimSession { return this.helper.request('describe', { udid: this.udid }); } + /** A system sheet animating in leaves no readable tree for a moment; once it settles, describe shows it. */ + async describeSettled() { + try { + const result = await this.describe(); + this.readable = true; + return result; + } catch (err) { + if (err.code !== 'ax_disabled' || !this.readable) throw err; + await sleep(this.timing.overlaySettleMs); + return this.describe(); + } + } + async fresh({ view } = {}) { try { - const snap = annotate(normalize(await this.describe()), this.corpus.get(), { view }); + const snap = annotate(normalize(await this.describeSettled()), this.corpus.get(), { view }); this.lastView = snap.view; return snap; } catch (err) { @@ -195,6 +233,7 @@ export class SimSession { async launchApp({ terminate = true } = {}) { if (terminate) await this.simctl.enableAccessibility(this.udid); this.appPid = await this.simctl.launch(this.udid, this.bundleId, { stdout: this.stdoutPath, stderr: this.stderrPath, terminate }); + this.readable = false; this.last = null; return { bundleId: this.bundleId, pid: this.appPid }; } @@ -313,6 +352,7 @@ export class SimSession { const count = (field.value ?? '').length; for (let i = 0; i < count; i++) await this.input('key', { key: 'forward_delete' }); for (let i = 0; i < count; i++) await this.input('key', { key: 'delete' }); + if (count) await sleep(this.timing.settleMs); } if (text) await this.input('type', { text }); const stuck = (n) => SECURE.has(n.tag) || !text || (n.value ?? '').toLowerCase().includes(text.toLowerCase()); diff --git a/skills/subtext-ios-sim/lib/errors.mjs b/skills/subtext-ios-sim/lib/errors.mjs index 5973c0e..fbf6d90 100644 --- a/skills/subtext-ios-sim/lib/errors.mjs +++ b/skills/subtext-ios-sim/lib/errors.mjs @@ -11,7 +11,7 @@ const HINTS = { daemon_unreachable: 'run `subtext-sim start --detach` to start a new session', device_not_found: 'list devices with `xcrun simctl list devices available`, then pass --device', not_booted: 'run `subtext-sim start` to boot the simulator', - ax_disabled: 'run `subtext-sim relaunch` (it re-enables app accessibility before launching)', + ax_disabled: 'snapshot again in a moment (a system sheet such as "Save Password?" may be animating in); if it persists, run `subtext-sim relaunch` (it re-enables app accessibility before launching)', ax_unavailable: 'the app may still be launching; retry, or run `subtext-sim relaunch`', ax_error: 'the app may not be in the foreground; run `subtext-sim launch`, or `subtext-sim relaunch`', hid_error: 'retry; if it keeps failing, run `subtext-sim stop` then `subtext-sim start`', diff --git a/skills/subtext-ios-sim/lib/normalize.mjs b/skills/subtext-ios-sim/lib/normalize.mjs index 6253fa5..c5d8c73 100644 --- a/skills/subtext-ios-sim/lib/normalize.mjs +++ b/skills/subtext-ios-sim/lib/normalize.mjs @@ -2,6 +2,7 @@ const TAG_BY_TYPE = { application: 'application', window: 'window', group: 'other', + genericelement: 'other', other: 'other', button: 'button', statictext: 'text', @@ -73,8 +74,23 @@ function isWrapper(raw, tag) { return tag === 'other' && !present(raw.label) && !present(raw.identifier) && !present(raw.value); } +// React Native reports accessibilityRole="tab" (React Navigation's tab buttons) +// as a button whose label carries VoiceOver's "tab, N of M" suffix. +const RN_TAB_LABEL = /^(.*), tab, \d+ of \d+$/; + +function rnTabTitle(raw) { + return typeof raw.label === 'string' ? RN_TAB_LABEL.exec(raw.label)?.[1] ?? null : null; +} + +// React Native's accessible Views (role-less Pressables included) are +// GenericElements; with a testID they are as likely to be tappable as not. +function isIdentifiedGeneric(raw) { + const type = String(raw.type || raw.role || '').replace(/^AX/, '').toLowerCase(); + return type === 'genericelement' && !!present(raw.identifier); +} + function collapse(raw, isRoot) { - const tag = tagFor(raw.type, raw.role, raw.subrole); + const tag = rnTabTitle(raw) !== null ? 'tab' : tagFor(raw.type, raw.role, raw.subrole); let children = (raw.children ?? []).flatMap((c) => collapse(c, false)); if (!isRoot && isWrapper(raw, tag)) { if (children.length <= 1) return children; @@ -112,14 +128,14 @@ export function normalize(describeResult) { tag, type: raw.type ?? null, role: raw.role ?? null, - label: present(raw.label), + label: present(rnTabTitle(raw) ?? raw.label), identifier: present(raw.identifier), value: present(raw.value), enabled: raw.enabled !== false, frame, visible: intersects(frame, screen), onScreen: centerOnScreen(frame, screen), - interactive: INTERACTIVE_TAGS.has(tag), + interactive: INTERACTIVE_TAGS.has(tag) || isIdentifiedGeneric(raw), ignored: isWrapper(raw, tag), nthChild, text: '', @@ -133,7 +149,20 @@ export function normalize(describeResult) { }; const [top] = collapse(rawRoot, true); const root = build(top, null, 1); - return { screen, root, nodes, byId: new Map(nodes.map((n) => [n.id, n])) }; + return { screen, root, nodes, byId: new Map(nodes.map((n) => [n.id, n])), overlay: !!describeResult.overlay }; +} + +// A React Native LogBox toast (Debug builds) is one accessible element labeled +// with its icon or count first; its ✕ has no element of its own. +const LOGBOX_LABEL = /^(!|\d+), /; + +/** The LogBox toast at the bottom of the screen and the point of its ✕, or null. */ +export function logBoxToast({ nodes, screen }) { + const toast = nodes.find((n) => n.tag === 'other' && !n.identifier && n.visible && n.frame.y > screen.h / 2 + && LOGBOX_LABEL.test(n.label ?? '')); + if (!toast) return null; + const { x, y, w, h } = toast.frame; + return { node: toast, dismiss: { x: Math.round(x + w - 24), y: Math.round(y + h / 2) } }; } const CSS_IDENT = /^-?[A-Za-z_][A-Za-z0-9_-]*$/; diff --git a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs index d015c9b..38f0d6a 100644 --- a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs +++ b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs @@ -1,6 +1,6 @@ import { mkdirSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; -import { attrsOf, cssId, normalize } from './normalize.mjs'; +import { attrsOf, cssId, logBoxToast, normalize } from './normalize.mjs'; import { compareValue, matches } from './selector.mjs'; import { loadCorpus } from './corpus.mjs'; @@ -185,7 +185,17 @@ export function render(annotated, { includeOffscreen = false, showIdentifiers = if (annotated.view) lines.push(`[View: ${annotated.view}]${annotated.viewInferred ? ' (inferred)' : ''}${where}`); else lines.push(`[No view matched]${where}`); lines.push(''); + if (annotated.overlay) { + const owner = annotated.root.label ? ` (${quote(annotated.root.label)})` : ''; + lines.push('[Overlay]', `A system sheet from another process${owner} is in front of the app; the tree below is that sheet.`, + 'Dismiss it with one of its buttons, e.g. `click \'button[text="Not Now"]\'`, then snapshot again.', ''); + } + const logBox = logBoxToast(annotated); + if (logBox) { + lines.push('[LogBox]', `A React Native LogBox toast (#${logBox.node.id}) covers the bottom of the screen; tapping it opens the LogBox inspector.`, + `Dismiss it with \`tap-xy ${logBox.dismiss.x} ${logBox.dismiss.y}\` (its ✕), or call LogBox.ignoreAllLogs() in the app.`, ''); + } if (annotated.guide.size) { lines.push('[Guide]'); const entries = [...annotated.guide].sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])); diff --git a/skills/subtext-ios-sim/test/e2e.sh b/skills/subtext-ios-sim/test/e2e.sh index 757d7ac..7aabe29 100755 --- a/skills/subtext-ios-sim/test/e2e.sh +++ b/skills/subtext-ios-sim/test/e2e.sh @@ -1,7 +1,8 @@ #!/usr/bin/env bash -# End-to-end run of subtext-sim against the sample app on a real simulator. +# End-to-end run of subtext-sim against a sample app on a real simulator. # -# test/e2e.sh --device # required; the device is shut down at the end +# test/e2e.sh --device # SwiftUI fixtures/SampleApp; the device is shut down at the end +# test/e2e.sh --device --fixture RNSampleApp # the React Native fixture (needs Node and CocoaPods) # # Env: E2E_KEEP_BOOTED=1 leaves the simulator running; OUT (default: a temp dir) collects artifacts. set -euo pipefail @@ -10,13 +11,16 @@ here="$(cd "$(dirname "$0")" && pwd)" skill="$(cd "$here/.." && pwd)" sim=(node "$skill/bin/subtext-sim") device="" +fixture=SampleApp +usage() { echo "usage: $0 --device [--fixture SampleApp|RNSampleApp]" >&2; exit 2; } while [[ $# -gt 0 ]]; do case "$1" in --device) device="$2"; shift 2 ;; - *) echo "usage: $0 --device " >&2; exit 2 ;; + --fixture) fixture="$2"; shift 2 ;; + *) usage ;; esac done -[[ -n "$device" ]] || { echo "usage: $0 --device " >&2; exit 2; } +[[ -n "$device" && -x "$skill/fixtures/$fixture/build.sh" ]] || usage out="${OUT:-$(mktemp -d "${TMPDIR:-/tmp}/subtext-sim-e2e.XXXXXX")}" mkdir -p "$out" @@ -38,13 +42,14 @@ step "build helper" bash "$skill/helper/build.sh" >/dev/null took -step "build sample app" -app="$(bash "$skill/fixtures/SampleApp/build.sh" 2>/dev/null | tail -1)" +step "build $fixture" +app="$(bash "$skill/fixtures/$fixture/build.sh" 2>/dev/null | tail -1)" +[[ -d "$app" ]] || fail "$fixture/build.sh did not produce an app; run it directly to see why" echo " $app" took # The sample app's .sightmap/ lives next to it, so run from there. -cd "$skill/fixtures/SampleApp" +cd "$skill/fixtures/$fixture" trap cleanup EXIT step "start --detach" @@ -77,6 +82,21 @@ sleep 1 step "snapshot (products)" snap="$("${sim[@]}" snapshot)" took +# iOS may offer to save the password a moment after the login form goes away. +for _ in 1 2 3; do + grep -qF '[Overlay]' <<<"$snap" && break + sleep 1 + snap="$("${sim[@]}" snapshot)" +done +if grep -qF '[Overlay]' <<<"$snap"; then + echo "$snap" | head -8 + "${sim[@]}" click 'button[text="Not Now"]' + for _ in 1 2 3 4 5; do + sleep 1 + snap="$("${sim[@]}" snapshot)" + grep -qF '[Overlay]' <<<"$snap" || break + done +fi echo "$snap" | head -12 expect '[View: Products]' "$snap" @@ -107,6 +127,16 @@ echo "$logs" expect 'added Product 150 to cart' "$logs" expect 'signed in as agent@example.com' "$("${sim[@]}" console list --grep 'signed in')" +if [[ "$fixture" == RNSampleApp ]]; then + step "switch tabs by TabBarItem and send a request" + "${sim[@]}" click 'TabBarItem[title="Network"]' + sleep 1 + "${sim[@]}" click network-send + sleep 1 + expect 'POST /anything/subtext-sim/order-1' "$("${sim[@]}" console list --grep 'POST')" + took +fi + step "screenshot" "${sim[@]}" screenshot --out "$out/product-150.png" file "$out/product-150.png" | grep -q 'PNG image' || fail "screenshot is not a PNG" diff --git a/skills/subtext-ios-sim/test/fixtures/describe-rn.json b/skills/subtext-ios-sim/test/fixtures/describe-rn.json new file mode 100644 index 0000000..026c828 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-rn.json @@ -0,0 +1 @@ +{"screen":{"w":402,"h":874},"root":{"role":"AXApplication","subrole":null,"type":"Application","label":"Subtext RN Sample","identifier":null,"value":null,"frame":{"x":0,"y":0,"w":402,"h":874},"enabled":true,"traits":["None"],"children":[{"role":"AXHeading","subrole":null,"type":"Heading","label":"Products","identifier":null,"value":null,"frame":{"x":165,"y":74,"w":72,"h":21},"enabled":true,"traits":["Header"],"children":[]},{"role":"AXGenericElement","subrole":null,"type":"GenericElement","label":"Product 1, $6.37","identifier":"product-row-1","value":null,"frame":{"x":0,"y":116,"w":402,"h":49},"enabled":true,"traits":[],"children":[]},{"role":"AXGenericElement","subrole":null,"type":"GenericElement","label":"Product 14, $19.18","identifier":"product-row-14","value":null,"frame":{"x":0,"y":757,"w":402,"h":49},"enabled":true,"traits":[],"children":[]},{"role":"AXGenericElement","subrole":null,"type":"GenericElement","label":"Product 15, $20.55","identifier":"product-row-15","value":null,"frame":{"x":0,"y":807,"w":402,"h":49},"enabled":true,"traits":[],"children":[]},{"role":"AXGenericElement","subrole":null,"type":"GenericElement","label":"Unidentified text","identifier":null,"value":null,"frame":{"x":0,"y":300,"w":402,"h":49},"enabled":true,"traits":[],"children":[]},{"role":"AXButton","subrole":null,"type":"Button","label":"Products, tab, 1 of 3","identifier":"tab-products","value":null,"frame":{"x":0,"y":791,"w":134,"h":49},"enabled":true,"traits":["Button","Selected"],"children":[]},{"role":"AXButton","subrole":null,"type":"Button","label":"Network, tab, 2 of 3","identifier":"tab-network","value":null,"frame":{"x":134,"y":791,"w":134,"h":49},"enabled":true,"traits":["Button"],"children":[]},{"role":"AXButton","subrole":null,"type":"Button","label":"Account, tab, 3 of 3","identifier":"tab-account","value":null,"frame":{"x":268,"y":791,"w":134,"h":49},"enabled":true,"traits":["Button"],"children":[]}]}} \ No newline at end of file diff --git a/skills/subtext-ios-sim/test/rn.test.mjs b/skills/subtext-ios-sim/test/rn.test.mjs new file mode 100644 index 0000000..85c4fbb --- /dev/null +++ b/skills/subtext-ios-sim/test/rn.test.mjs @@ -0,0 +1,73 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { normalize } from '../lib/normalize.mjs'; +import { annotate, resolve } from '../lib/sightmap-bridge.mjs'; +import { parseCorpus } from '../lib/corpus.mjs'; +import { assertActionable } from '../lib/daemon.mjs'; + +const here = dirname(fileURLToPath(import.meta.url)); +const describeRN = () => JSON.parse(readFileSync(join(here, 'fixtures', 'describe-rn.json'), 'utf8')); + +const corpus = parseCorpus({ + globals: [{ name: 'TabBarItem', selectors: ['tab'], properties: [{ name: 'title', extract: 'text' }] }], + views: [{ name: 'Products', components: [{ name: 'ProductRow', selectors: ['[identifier^="product-row-"]'] }] }], +}); + +const byIdentifier = (snap, identifier) => snap.nodes.find((n) => n.identifier === identifier); + +test('React Navigation tab buttons normalize to tabs titled without the VoiceOver suffix', () => { + const snap = annotate(normalize(describeRN()), corpus); + const tab = byIdentifier(snap, 'tab-network'); + assert.equal(tab.tag, 'tab'); + assert.equal(tab.label, 'Network'); + assert.equal(resolve(snap, 'TabBarItem[title="Network"]'), tab); +}); + +test('identified React Native GenericElements are interactive, unidentified ones are not', () => { + const { nodes } = normalize(describeRN()); + const row = nodes.find((n) => n.identifier === 'product-row-1'); + assert.equal(row.tag, 'other'); + assert.equal(row.interactive, true); + assert.equal(nodes.find((n) => n.label === 'Unidentified text').interactive, false); +}); + +test('a row under a JS tab bar is refused as covered; rows above it and the tabs are not', () => { + const snap = annotate(normalize(describeRN()), corpus); + assert.throws(() => assertActionable(snap, byIdentifier(snap, 'product-row-15'), 'product-row-15'), + (err) => err.code === 'covered' && /tabbar/.test(err.message)); + assert.doesNotThrow(() => assertActionable(snap, byIdentifier(snap, 'product-row-14'), 'product-row-14')); + assert.doesNotThrow(() => assertActionable(snap, byIdentifier(snap, 'tab-account'), 'tab-account')); +}); + +test('a system sheet from another process renders as an overlay with a way out', () => { + const sheet = { + screen: { w: 402, h: 874 }, + overlay: true, + root: { + type: 'Application', role: 'AXApplication', label: 'Safari', frame: { x: 0, y: 0, w: 402, h: 874 }, + children: [ + { type: 'StaticText', role: 'AXStaticText', label: 'Save Password?', frame: { x: 71, y: 417, w: 260, h: 20 } }, + { type: 'Button', role: 'AXButton', label: 'Not Now', frame: { x: 57, y: 523, w: 140, h: 48 } }, + ], + }, + }; + const snap = annotate(normalize(sheet), corpus); + assert.match(snap.text, /\[Overlay\]\nA system sheet from another process \("Safari"\)/); + assert.equal(resolve(snap, 'button[text="Not Now"]').label, 'Not Now'); + assert.doesNotMatch(annotate(normalize(describeRN()), corpus).text, /\[Overlay\]/); +}); + +test('a LogBox toast is reported with its ✕ and refuses taps underneath', () => { + const describe = describeRN(); + describe.root.children.push({ + type: 'GenericElement', role: 'AXGenericElement', label: '!, sign-in rejected', identifier: null, + frame: { x: 10, y: 788, w: 382, h: 48 }, + }); + const snap = annotate(normalize(describe), corpus); + assert.match(snap.text, /\[LogBox\]\n.*\nDismiss it with `tap-xy 368 812`/); + assert.throws(() => assertActionable(snap, byIdentifier(snap, 'product-row-15'), 'product-row-15'), + (err) => err.code === 'covered' && /LogBox/.test(err.message) && /tap-xy 368 812/.test(err.hint)); +}); From c38b875e7fe30a1aa0d5aae8047cd258dc43f739 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 15:23:45 -0400 Subject: [PATCH 05/12] subtext-ios-sim: build SampleApp with xcodebuild The swiftc hand-assembled bundle lacked the simulated entitlements Xcode embeds in simulator apps, so the Fullstory SDK's background uploads all failed and no session reached Fullstory. Build a committed SimSample.xcodeproj instead; FS_ORG_ID injects the SDK, OrgId and VerboseLogging at build time, so no org ID lives in the repo. Verified: e2e passes, and with FS_ORG_ID the app's POST is readable via review-open / review-zoom (status, timing, response headers). Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/README.md | 4 +- skills/subtext-ios-sim/SKILL.md | 2 + .../SimSample.xcodeproj/project.pbxproj | 158 ++++++++++++++++++ .../fixtures/SampleApp/build.sh | 56 ++++--- 4 files changed, 192 insertions(+), 28 deletions(-) create mode 100644 skills/subtext-ios-sim/fixtures/SampleApp/SimSample.xcodeproj/project.pbxproj diff --git a/skills/subtext-ios-sim/README.md b/skills/subtext-ios-sim/README.md index a8973fc..037fee9 100644 --- a/skills/subtext-ios-sim/README.md +++ b/skills/subtext-ios-sim/README.md @@ -86,12 +86,12 @@ test/e2e.sh --device "" --fixture RNSampleApp # the Re Every control has an `accessibilityIdentifier` (`login-email`, `login-submit`, `tab-network`, `product-row-42`, `product-detail-add`, `network-send`…). -`build.sh` needs only Xcode: it compiles with `swiftc` and assembles the bundle by hand, so there's no Xcode project. It writes to `$SUBTEXT_SIM_CACHE/SampleApp/`, default `~/Library/Caches/subtext-sim/SampleApp/` (set `OUT_DIR` to change just the app's location) and prints the `.app` path on stdout. +`build.sh` needs only Xcode: it builds `SimSample.xcodeproj` with `xcodebuild` (a target build for the simulator SDK). It must be a real Xcode build: Xcode embeds simulated entitlements in simulator apps, and without them the Fullstory SDK's background uploads fail, so no session data reaches Fullstory. Apply the same rule to your own app. It writes to `$SUBTEXT_SIM_CACHE/SampleApp/`, default `~/Library/Caches/subtext-sim/SampleApp/` (set `OUT_DIR` to change just the app's location) and prints the `.app` path on stdout. To capture sessions with Fullstory, build with your org ID: ```sh -FS_ORG_ID= fixtures/SampleApp/build.sh +FS_ORG_ID= fixtures/SampleApp/build.sh # also sets Fullstory VerboseLogging=2 (FS_VERBOSE_LOGGING to change) ``` This downloads the public Fullstory iOS SDK (the binary from [`fullstorydev/fullstory-swift-package-ios`](https://github.com/fullstorydev/fullstory-swift-package-ios), checksum-verified; set `FS_SDK_VERSION` to pick a release), embeds it, adds a `Fullstory` dictionary with `OrgId` and `RecordOnStart` to `Info.plist`, and logs `FULLSTORY_SESSION_URL ` when a session starts. That line is what `subtext-sim session-url` reads. diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md index 5bde629..26aa64c 100644 --- a/skills/subtext-ios-sim/SKILL.md +++ b/skills/subtext-ios-sim/SKILL.md @@ -136,6 +136,8 @@ The loop is the same; RN maps onto the iOS accessibility tree like this: `/fixtures/SampleApp/build.sh` builds a small SwiftUI app (login form, 200-row product list, tab bar, a JSON POST button) and prints the `.app` path. Set `FS_ORG_ID=` to build it with the Fullstory SDK, which enables `session-url` and network. +**Apps you build yourself must be built by Xcode** (`xcodebuild` or the Xcode app), not assembled by hand. Without the simulated entitlements Xcode embeds, the Fullstory SDK's uploads fail in the simulator, and the consumer sees no session. After signing in, iOS may show a "Save Password?" sheet over the app for a moment; if a snapshot reports a system sheet, snapshot again. + `/fixtures/RNSampleApp/build.sh` builds the same app in React Native (React Navigation tabs and stack, a `FlatList`, the same `testID`s) with its JS bundle embedded. It needs Node 22.11+ and CocoaPods, and takes several minutes the first time. `FS_ORG_ID` adds the Fullstory React Native SDK; `CONFIGURATION=Debug` builds for Metro instead. ## Known limits diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/SimSample.xcodeproj/project.pbxproj b/skills/subtext-ios-sim/fixtures/SampleApp/SimSample.xcodeproj/project.pbxproj new file mode 100644 index 0000000..c0653e0 --- /dev/null +++ b/skills/subtext-ios-sim/fixtures/SampleApp/SimSample.xcodeproj/project.pbxproj @@ -0,0 +1,158 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 56; + objects = { + +/* Begin PBXBuildFile section */ + A10000000000000000000001 /* SampleApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = A20000000000000000000001; }; + A10000000000000000000002 /* FullstoryBridge.swift in Sources */ = {isa = PBXBuildFile; fileRef = A20000000000000000000002; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + A20000000000000000000001 /* SampleApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SampleApp.swift; sourceTree = ""; }; + A20000000000000000000002 /* FullstoryBridge.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FullstoryBridge.swift; sourceTree = ""; }; + A20000000000000000000004 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; + A20000000000000000000005 /* SimSample.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = SimSample.app; sourceTree = BUILT_PRODUCTS_DIR; }; +/* End PBXFileReference section */ + +/* Begin PBXGroup section */ + A40000000000000000000001 = { + isa = PBXGroup; + children = ( + A40000000000000000000003, + A20000000000000000000004, + A40000000000000000000002, + ); + sourceTree = ""; + }; + A40000000000000000000002 /* Products */ = { + isa = PBXGroup; + children = ( + A20000000000000000000005, + ); + name = Products; + sourceTree = ""; + }; + A40000000000000000000003 /* Sources */ = { + isa = PBXGroup; + children = ( + A20000000000000000000001, + A20000000000000000000002, + ); + path = Sources; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + A50000000000000000000001 /* SimSample */ = { + isa = PBXNativeTarget; + buildConfigurationList = A70000000000000000000002; + buildPhases = ( + A30000000000000000000003, + ); + buildRules = ( + ); + dependencies = ( + ); + name = SimSample; + productName = SimSample; + productReference = A20000000000000000000005; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + A60000000000000000000001 /* Project object */ = { + isa = PBXProject; + attributes = { + BuildIndependentTargetsInParallel = 1; + LastUpgradeCheck = 2600; + }; + buildConfigurationList = A70000000000000000000001; + compatibilityVersion = "Xcode 14.0"; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + en, + Base, + ); + mainGroup = A40000000000000000000001; + productRefGroup = A40000000000000000000002; + projectDirPath = ""; + projectRoot = ""; + targets = ( + A50000000000000000000001, + ); + }; +/* End PBXProject section */ + +/* Begin PBXSourcesBuildPhase section */ + A30000000000000000000003 /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + A10000000000000000000001, + A10000000000000000000002, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin XCBuildConfiguration section */ + A80000000000000000000001 /* Debug (project) */ = { + isa = XCBuildConfiguration; + buildSettings = { + IPHONEOS_DEPLOYMENT_TARGET = 17.0; + ONLY_ACTIVE_ARCH = YES; + SDKROOT = iphoneos; + SUPPORTED_PLATFORMS = "iphoneos iphonesimulator"; + SWIFT_VERSION = 5.0; + }; + name = Debug; + }; + A80000000000000000000002 /* Debug (target) */ = { + isa = XCBuildConfiguration; + buildSettings = { + CODE_SIGN_IDENTITY = "-"; + CODE_SIGN_STYLE = Manual; + DEVELOPMENT_TEAM = ""; + GENERATE_INFOPLIST_FILE = NO; + INFOPLIST_FILE = Info.plist; + LD_RUNPATH_SEARCH_PATHS = ( + "$(inherited)", + "@executable_path/Frameworks", + ); + PRODUCT_BUNDLE_IDENTIFIER = dev.subtext.SimSample; + PRODUCT_NAME = SimSample; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + TARGETED_DEVICE_FAMILY = 1; + }; + name = Debug; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + A70000000000000000000001 /* project */ = { + isa = XCConfigurationList; + buildConfigurations = ( + A80000000000000000000001, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Debug; + }; + A70000000000000000000002 /* target */ = { + isa = XCConfigurationList; + buildConfigurations = ( + A80000000000000000000002, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Debug; + }; +/* End XCConfigurationList section */ + }; + rootObject = A60000000000000000000001; +} diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/build.sh b/skills/subtext-ios-sim/fixtures/SampleApp/build.sh index eab5ff4..102c013 100755 --- a/skills/subtext-ios-sim/fixtures/SampleApp/build.sh +++ b/skills/subtext-ios-sim/fixtures/SampleApp/build.sh @@ -1,33 +1,36 @@ #!/bin/bash -# Builds the Subtext Sim Sample app for the iOS Simulator with only Xcode installed -# (swiftc plus a hand-assembled bundle, no .xcodeproj) and prints the .app path. +# Builds the Subtext Sim Sample app for the iOS Simulator with xcodebuild (SimSample.xcodeproj) +# and prints the .app path. Only Xcode is required. # # ./build.sh # plain build # FS_ORG_ID= ./build.sh # embed the public Fullstory iOS SDK and record on start # # Env: SUBTEXT_SIM_CACHE (default ~/Library/Caches/subtext-sim), OUT_DIR (default $SUBTEXT_SIM_CACHE/SampleApp), -# FS_SDK_VERSION (tag of fullstorydev/fullstory-swift-package-ios, default 1.76.0). +# FS_SDK_VERSION (tag of fullstorydev/fullstory-swift-package-ios, default 1.76.0), +# FS_VERBOSE_LOGGING (Fullstory VerboseLogging level with FS_ORG_ID, default 2). +# +# It must be an xcodebuild build: Xcode embeds simulated entitlements (application-identifier) in +# simulator builds, and without them the SDK's background uploads fail, so no session data arrives. set -euo pipefail here="$(cd "$(dirname "$0")" && pwd)" cache_root="${SUBTEXT_SIM_CACHE:-$HOME/Library/Caches/subtext-sim}" out_dir="${OUT_DIR:-$cache_root/SampleApp}" +build_dir="$cache_root/SampleApp-build" fs_version="${FS_SDK_VERSION:-1.76.0}" app="$out_dir/SimSample.app" log() { echo "build.sh: $*" >&2; } -arch="$(uname -m)" -target="${arch}-apple-ios17.0-simulator" -sdk="$(xcrun --sdk iphonesimulator --show-sdk-path)" - -rm -rf "$app" -mkdir -p "$app" -cp "$here/Info.plist" "$app/Info.plist" +mkdir -p "$build_dir" "$out_dir" +plist="$build_dir/Info.plist" +cp "$here/Info.plist" "$plist" -swift_flags=() +settings=(INFOPLIST_FILE="$plist" ARCHS="$(uname -m)") +framework="" if [[ -n "${FS_ORG_ID:-}" ]]; then fs_cache="$cache_root/fullstory-$fs_version" - framework="$fs_cache/FullStory.xcframework/ios-arm64_x86_64-simulator/FullStory.framework" + slice="$fs_cache/FullStory.xcframework/ios-arm64_x86_64-simulator" + framework="$slice/FullStory.framework" if [[ ! -d "$framework" ]]; then log "fetching Fullstory iOS SDK $fs_version" manifest="$(curl -fsSL "https://raw.githubusercontent.com/fullstorydev/fullstory-swift-package-ios/$fs_version/Package.swift")" @@ -43,29 +46,30 @@ if [[ -n "${FS_ORG_ID:-}" ]]; then unzip -q -o "$fs_cache/sdk.zip" -d "$fs_cache" rm "$fs_cache/sdk.zip" fi - swift_flags+=(-D SUBTEXT_FULLSTORY -F "$(dirname "$framework")" -framework FullStory - -Xlinker -rpath -Xlinker @executable_path/Frameworks) - mkdir -p "$app/Frameworks" - cp -R "$framework" "$app/Frameworks/" - plist="$app/Info.plist" /usr/libexec/PlistBuddy \ -c "Add :Fullstory dict" \ -c "Add :Fullstory:OrgId string $FS_ORG_ID" \ -c "Add :Fullstory:RecordOnStart bool true" \ + -c "Add :Fullstory:VerboseLogging integer ${FS_VERBOSE_LOGGING:-2}" \ "$plist" >/dev/null + settings+=(SWIFT_ACTIVE_COMPILATION_CONDITIONS="DEBUG SUBTEXT_FULLSTORY" + FRAMEWORK_SEARCH_PATHS="$slice" OTHER_LDFLAGS="-framework FullStory") log "including Fullstory SDK $fs_version" fi -log "compiling for $target" -xcrun --sdk iphonesimulator swiftc \ - -target "$target" -sdk "$sdk" -O -parse-as-library \ - "${swift_flags[@]+"${swift_flags[@]}"}" \ - -o "$app/SimSample" \ - "$here"/Sources/*.swift +log "xcodebuild for the iOS Simulator" +xcodebuild -project "$here/SimSample.xcodeproj" -target SimSample -configuration Debug -sdk iphonesimulator \ + SYMROOT="$build_dir/products" OBJROOT="$build_dir/intermediates" "${settings[@]}" build >"$build_dir/xcodebuild.log" 2>&1 \ + || { tail -20 "$build_dir/xcodebuild.log" >&2; log "xcodebuild failed; full log: $build_dir/xcodebuild.log"; exit 1; } -if [[ -d "$app/Frameworks" ]]; then - codesign --force --sign - "$app"/Frameworks/*.framework >/dev/null 2>&1 +rm -rf "$app" +cp -R "$build_dir/products/Debug-iphonesimulator/SimSample.app" "$app" + +if [[ -n "$framework" ]]; then + mkdir -p "$app/Frameworks" + cp -R "$framework" "$app/Frameworks/" + codesign --force --sign - "$app/Frameworks/FullStory.framework" >/dev/null 2>&1 + codesign --force --sign - --preserve-metadata=entitlements "$app" >/dev/null 2>&1 fi -codesign --force --sign - "$app" >/dev/null 2>&1 echo "$app" From 441004492357551b415f4497d7151f02aa327320 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 15:32:25 -0400 Subject: [PATCH 06/12] subtext-ios-sim: build the helper without keychain or netrc credentials On a cold build SwiftPM downloads idb's public xcframeworks from github.com and, when the login keychain holds a github.com credential, shows a "swift-package wants to use your confidential information" keychain prompt. The downloads are public, so pass --disable-keychain and --disable-netrc and never ask. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/helper/build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/skills/subtext-ios-sim/helper/build.sh b/skills/subtext-ios-sim/helper/build.sh index c427613..cd26b3c 100755 --- a/skills/subtext-ios-sim/helper/build.sh +++ b/skills/subtext-ios-sim/helper/build.sh @@ -60,10 +60,10 @@ fi SCRATCH="$CACHE_ROOT/.build" echo "build.sh: building subtext-sim-helper $VERSION (first build downloads idb v1.7.7 frameworks, ~10 MB)" >&2 started=$SECONDS -xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --cache-path "$CACHE_ROOT/.swiftpm" >&2 \ +xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --cache-path "$CACHE_ROOT/.swiftpm" --disable-keychain --disable-netrc >&2 \ || fail "swift build failed; see the output above" -BUILT="$(xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --cache-path "$CACHE_ROOT/.swiftpm" --show-bin-path)/subtext-sim-helper" +BUILT="$(xcrun swift build -c release --package-path "$HELPER_DIR" --scratch-path "$SCRATCH" --cache-path "$CACHE_ROOT/.swiftpm" --disable-keychain --disable-netrc --show-bin-path)/subtext-sim-helper" [[ -x "$BUILT" ]] || fail "build finished but $BUILT is missing" mkdir -p "$OUT_DIR" From e9fb6b39c374302029d90fa18f67e6e31bf139ba Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 15:51:44 -0400 Subject: [PATCH 07/12] subtext-ios-sim: run producer and consumer concurrently; explain SDK delivery Spell out how Fullstory SDK data reaches agents (in-app capture, ~20 s bundle uploads, point-in-time review-open reads at 10 credits, network capture and Xcode-build requirements) and make the producer and consumer work at the same time: the orchestrator starts the consumer as soon as a session URL exists, the consumer re-reads every ~30 s within a credit budget, and asks the producer for coverage through a shared notes file. Ownership of .sightmap/ sections is split to avoid conflicting edits. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/SKILL.md | 62 +++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 19 deletions(-) diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md index 26aa64c..825327c 100644 --- a/skills/subtext-ios-sim/SKILL.md +++ b/skills/subtext-ios-sim/SKILL.md @@ -69,29 +69,53 @@ subtext-sim stop # --shutdown also shuts the simul Every command takes `--json`. Errors are one line with a `Next:` step. -## Network, and the producer / consumer split - -Network and Fullstory-captured console data come from the app's Fullstory SDK session, read through the Subtext MCP. It needs an app built with the Fullstory iOS SDK and network capture enabled for the org. That data arrives on the SDK's upload interval, about **20 s** behind the app. So don't make one agent tap and then wait; split the work into two subagents. - -**Producer:** drives the app and never waits on network data. -- `start` (opens a run), then work through flows with `snapshot` / `click` / `fill` / `scroll` / `openurl`. -- Every action is appended to the run log `.sightmap/.sim-runs/.jsonl` with its timestamp, target component and view. The session URL is logged as a `session` entry once the SDK reports it. -- It also fixes UI-side gaps as it goes: elements it can't target cleanly are candidates for new `components:`. -- When done, it reports the run id. - -**Consumer:** turns captured traffic into sightmap. -1. `run tail --follow` (or `--since SEQ`) to follow the producer's steps and pick up the `session` URL. -2. `review-open` with that URL, then `review-zoom` with `resolution={"network": "standard", "console": "standard"}`. Use `"detail"` on specific requests for headers and bodies. -3. Line up each request and log with the producer step just before it (by timestamp), then author or clean up: +## Network, and the producer / consumer collaboration + +### How the Fullstory SDK delivers data + +Network and Fullstory-captured console data come from the app's own Fullstory SDK session, read back through the Subtext MCP. Nothing is read from the device directly, so know how the data moves: + +- **Capture is in the app.** The SDK records interactions, navigation, network requests (method, URL, status, timing) and `FS.log` messages as they happen. Headers and bodies are captured only when the org's network header allowlists and body rules ask for them. +- **Upload is batched.** The SDK uploads a bundle about every **20 s**, and also when the app goes to the background or relaunches. Fullstory makes a bundle readable within seconds of receiving it, so any given request is visible roughly 5–30 s after it happens. +- **Reads are point-in-time.** Each `review-open` returns the session as uploaded so far and costs **10 credits**. `review-zoom` on the same client is free but never shows newer data. To see what arrived since, open the session again. +- **Requirements:** + - the Fullstory SDK in the app, and an app **built by Xcode** (hand-assembled apps can't upload from the simulator); + - **mobile network capture turned on** for the org (Settings → Data Capture → Mobile; `doctor` reminds you); + - the Subtext MCP authenticated to the same org. +- **Console:** only `FS.log` reaches Fullstory. `print` / `os_log` output is local only; read it live with `subtext-sim console list`. +- **Session URL:** `subtext-sim session-url` returns it, and the run log records it as a `session` entry. A relaunch can start a new session; when the run log gets a new `session` entry, follow the newest one. + +### Two agents, working at the same time + +Because data lags the app by up to ~30 s, one agent that taps, then waits, then reads would spend most of its time waiting. Split the work into two subagents that **run concurrently** for the whole run: the producer keeps driving while the consumer keeps reading. + +**Orchestrating agent:** +1. Start the **producer** as a background subagent. +2. Poll `subtext-sim session-url` every few seconds until it returns a URL, which is usually within seconds of `start`. Then start the **consumer** as a second background subagent, **without waiting for the producer to finish**. +3. Give both the run id and a credit budget for the consumer (default: at most 10 `review-open` calls, i.e. 100 credits, unless the user says otherwise). +4. When the producer reports it's done, tell the consumer, wait for its final pass, then reconcile and report. + +**Producer** drives the app and never waits on network data: +- It works through flows with `snapshot` / `click` / `fill` / `scroll` / `openurl`. Every action lands in the run log `.sightmap/.sim-runs/.jsonl` with its timestamp, target component and view. +- It owns `components:` and `views:` in `.sightmap/`: elements it can't target cleanly become new components. +- Between flows, it reads `.sightmap/.sim-runs/.consumer.md` for the consumer's requests (e.g. "re-run checkout; a POST /orders fired with no step that explains it") and works them into its next flows. +- When finished, it reports to the orchestrator: the run id, the flows it covered, and any consumer requests it couldn't do. It leaves the app running, so the last bundle uploads and the consumer can still re-check a screen. + +**Consumer** reads captured data while the producer is still working: +1. Follow the producer with `subtext-sim run tail --follow` (or poll with `--since SEQ`). +2. About every 30 s while new producer steps are appearing, read the newest data: `review-open` the session URL, then `review-zoom` with `t0_ms` set past what it has already processed, and `resolution={"network": "standard", "console": "standard"}`. Use `"detail"` only on the specific requests whose headers or bodies matter. Close each client with `review-close` before opening the next. +3. Match each request and `FS.log` message to the producer step just before it (by timestamp), then author: - `requests:` (route glob + method) for endpoints that mean something; - `properties:` for outcomes hidden in bodies; - - `messages:` for meaningful logs and errors; - - corrections to components and views the producer flagged. -4. Re-read about every 20 s, or once after the producer finishes. Each `review-open` costs credits, so don't poll faster than data can arrive. + - `messages:` for meaningful logs and errors. + + It owns those sections of `.sightmap/`. Component and view problems it notices go to the producer, not into the YAML. +4. When it needs more coverage (an endpoint it can't place, a flow that never fired a request, an error worth reproducing), it appends a short line to `.sightmap/.sim-runs/.consumer.md`. +5. When the producer is done, wait ~30 s for the last bundle, do one final read to cover the tail of the session, then report what it authored and what remains unmapped. -The orchestrating agent starts the producer, then the consumer once the run has a `session` entry. Run the producer alone when only UI or components work is needed. +**Budget and pacing:** never re-open faster than every ~30 s, since newer data can't arrive sooner. Stop at the credit budget and say so. If the producer goes idle, stop polling until new steps appear. -Local, live alternatives that need no SDK: `console list` (the app's os_log / print output) and `snapshot`. +Run the producer alone when only UI or components work is needed. For quick local signals that need no SDK and no credits, use `console list` and `snapshot`. ## Authoring `.sightmap/` for iOS From 61a37a844934d27cec1ee6b53f5d26ba4af2c523 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 17:08:00 -0400 Subject: [PATCH 08/12] subtext-ios-sim: fixes from React Native field test - Bridge accepts sightmap's object `extract` form (0.34+); npx pin moves to 0.36.1 and doctor reports the resolved sightmap version and range - session-url finds any Fullstory session URL in any console entry - Selectors: `testid` aliases `identifier`; descendant chains skip ancestors whose identifier no element carries (wrapper views aren't accessibility elements) - Skill and fixture YAML pass `sightmap validate`; validate runs in tests and the e2e - start errors when asked for a different app than the running session and reports what it installed/relaunched - tap-xy/swipe reject off-screen points (units are points) - Empty-tree errors list modal/hidden-root views first and drop the accessibility advice when it's already enabled - Skill: dev client + Metro is fine, inside-vs-outside capture, empty-tree causes, testIDs on accessible elements, Metro --clear, `machine` grain for naming endpoints; README: UTF-8 locale for pod install Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/README.md | 7 +- skills/subtext-ios-sim/SKILL.md | 60 ++++++++---- .../RNSampleApp/.sightmap/components.yaml | 2 +- .../RNSampleApp/.sightmap/views/login.yaml | 3 + .../RNSampleApp/.sightmap/views/products.yaml | 5 +- .../SampleApp/.sightmap/components.yaml | 2 +- .../SampleApp/.sightmap/views/login.yaml | 3 + .../SampleApp/.sightmap/views/products.yaml | 5 +- skills/subtext-ios-sim/lib/BRIDGE.md | 58 +++++++++++- skills/subtext-ios-sim/lib/cli.mjs | 27 ++++-- skills/subtext-ios-sim/lib/corpus.mjs | 22 ++++- skills/subtext-ios-sim/lib/daemon.mjs | 45 ++++++++- skills/subtext-ios-sim/lib/doctor.mjs | 26 ++++-- skills/subtext-ios-sim/lib/errors.mjs | 18 +++- skills/subtext-ios-sim/lib/logs.mjs | 7 ++ skills/subtext-ios-sim/lib/normalize.mjs | 24 ++++- skills/subtext-ios-sim/lib/selector.mjs | 53 ++++++++++- .../subtext-ios-sim/lib/sightmap-bridge.mjs | 77 ++++++++++++--- skills/subtext-ios-sim/lib/sightmap-cli.mjs | 78 +++++++++++++--- skills/subtext-ios-sim/test/bridge.test.mjs | 93 ++++++++++++++++++- skills/subtext-ios-sim/test/cli.test.mjs | 46 ++++++++- skills/subtext-ios-sim/test/e2e.sh | 16 ++++ skills/subtext-ios-sim/test/rn.test.mjs | 35 +++++++ skills/subtext-ios-sim/test/sightmap.test.mjs | 52 +++++++++++ 24 files changed, 676 insertions(+), 88 deletions(-) create mode 100644 skills/subtext-ios-sim/test/sightmap.test.mjs diff --git a/skills/subtext-ios-sim/README.md b/skills/subtext-ios-sim/README.md index 037fee9..610d35f 100644 --- a/skills/subtext-ios-sim/README.md +++ b/skills/subtext-ios-sim/README.md @@ -8,6 +8,7 @@ Drive and inspect an iOS app in the local iOS Simulator from a coding agent. The - Xcode 26 or later, opened once, with an iOS Simulator runtime (Xcode > Settings > Components) - `xcode-select -p` pointing at Xcode, not the Command Line Tools - Node 18 or later +- The sightmap CLI: nothing to install, `subtext-sim` runs the pinned `@sightmap/sightmap@0.36.1` through `npx`. A `sightmap` on your PATH (or `SIGHTMAP_BIN`) wins over the pin; `doctor` reports which one resolves and warns outside the tested range (0.32–0.36; the sample apps' `.sightmap/` need 0.34+). ## Install from the branch (Claude Code) @@ -73,7 +74,7 @@ test/e2e.sh --device "" # drives the SwiftUI sample app test/e2e.sh --device "" --fixture RNSampleApp # the React Native sample app ``` -`e2e.sh` boots the simulator if needed, signs in (dismissing iOS's "Save Password?" sheet if it appears), scrolls to Product 150, adds it to the cart, and checks the console, a screenshot and the run log. The React Native run also switches tabs and sends a request. Set `E2E_KEEP_BOOTED=1` to leave the simulator running. Set `SIGHTMAP_BIN` to a sightmap CLI to run the unit tests that need one. +`e2e.sh` boots the simulator if needed, signs in (dismissing iOS's "Save Password?" sheet if it appears), scrolls to Product 150, adds it to the cart, and checks the console, a screenshot and the run log. The React Native run also switches tabs and sends a request. Set `E2E_KEEP_BOOTED=1` to leave the simulator running. `e2e.sh` also runs `sightmap validate` on the fixture's `.sightmap/`. Set `SIGHTMAP_BIN` to a sightmap CLI to run the unit tests that need one (the parity tests with 0.32+, the `validate` tests with 0.34+). ## Sample apps @@ -100,10 +101,10 @@ This downloads the public Fullstory iOS SDK (the binary from [`fullstorydev/full `fixtures/RNSampleApp/` is the same app in React Native 0.88 ("Subtext RN Sample", `dev.subtext.RNSample`): React Navigation bottom tabs with a native stack inside the Products tab, a 200-row `FlatList`, and the same `testID`s, so its `.sightmap/` differs from the SwiftUI one only in role-free selectors. Product rows and Add to Cart are `Pressable`s without `accessibilityRole`, like most RN code, so they show up as `other`. -Requirements on top of Xcode: Node 22.11+, npm, CocoaPods, and network access the first time. Only the JS sources are checked in. `build.sh` generates the native iOS project from the React Native template for the pinned version, then runs `npm install`, `pod install` and `xcodebuild` in `~/Library/Caches/subtext-sim/RNSampleApp/` (set `OUT_DIR` to change that). It prints the `.app` path on stdout. The first build takes several minutes; after a JS change, a rebuild takes about 15 s. +Requirements on top of Xcode: Node 22.11+, npm, CocoaPods, and network access the first time. `pod install` needs a UTF-8 locale; `build.sh` sets one, but for your own app run `export LANG=en_US.UTF-8` first. Only the JS sources are checked in. `build.sh` generates the native iOS project from the React Native template for the pinned version, then runs `npm install`, `pod install` and `xcodebuild` in `~/Library/Caches/subtext-sim/RNSampleApp/` (set `OUT_DIR` to change that). It prints the `.app` path on stdout. The first build takes several minutes; after a JS change, a rebuild takes about 15 s. - `./build.sh` builds Release with the JS bundle embedded, so `subtext-sim start --app` works without Metro. `ios/SceneDelegate.swift` replaces the template's to keep `console.log` in Release builds (`RCTSetLogThreshold`). -- `CONFIGURATION=Debug ./build.sh` builds for Metro. Start Metro from the generated project (the build prints the command) before launching. +- `CONFIGURATION=Debug ./build.sh` builds for Metro. Start Metro from the generated project (the build prints the command) before launching. Debug with Metro works for the whole loop and is the cheaper choice while iterating on JS; if edits don't show up, restart Metro with `--clear`. - `FS_ORG_ID= ./build.sh` adds `@fullstory/react-native` (its Babel plugins, the `FullStory` pod and a `FullStory` `Info.plist` dictionary) and logs `FULLSTORY_SESSION_URL ` from `FullStory.onReady()`. `FS_RN_VERSION` and `FS_SDK_VERSION` pick the plugin and iOS SDK versions. `build.sh` uses target builds rather than `xcodebuild -scheme`, and drops the template's launch storyboard and asset catalog. Both need Xcode's iOS platform component for the SDK version, which a machine with only a Simulator runtime installed doesn't have. diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md index 825327c..2f7cf77 100644 --- a/skills/subtext-ios-sim/SKILL.md +++ b/skills/subtext-ios-sim/SKILL.md @@ -37,9 +37,9 @@ subtext-sim console list --grep error # app logs (print / os_log), live subtext-sim stop # --shutdown also shuts the simulator down ``` -1. **Start** once per session, always with `--detach` (without it, `start` holds the shell as a foreground daemon). `status` reports health; `start` again reinstalls (with `--app`) and relaunches the app. The first `start` on a simulator that has never had app accessibility enabled reboots it once. +1. **Start** once per session, always with `--detach` (without it, `start` holds the shell as a foreground daemon). `status` reports health; `start` again on a running session reinstalls (with `--app`) and relaunches the **same** app, and says which. It refuses a different app: `stop` first, then `start` the other one. The first `start` on a simulator that has never had app accessibility enabled reboots it once. 2. **Snapshot** before acting. Ids are per-snapshot, so act on the latest one. Prefer accessibility identifiers or component names over snapshot ids in anything you write down. -3. **Act** with `click` / `fill`. Both re-read the screen just before tapping, so a component query or identifier never goes stale. Lower level: `scroll`, `swipe`, `tap-xy x y`, `type`, `key return`, `button home`, `openurl `. +3. **Act** with `click` / `fill`. Both re-read the screen just before tapping, so a component query or identifier never goes stale. Lower level: `scroll`, `swipe`, `tap-xy x y`, `type`, `key return`, `button home`, `openurl `. `tap-xy` and `swipe` take screen **points**, not pixels (a 390×844 pt screen is 1170×2532 px in a screenshot); points off the screen are refused. 4. **Check** with `snapshot` (state), `console list` / `console get ` (logs), `screenshot --out shot.png` (visual evidence). **Targets** are a snapshot id (`12`), an exact accessibility identifier (`login-submit`), or a component query: `SignInButton`, `ProductRow[title^="Product 15"]`, `LoginForm SignInButton`, or a role with attributes for unmapped apps, `button[identifier="login-submit"]`. A query matching several elements errors and lists them; add a predicate or append `#N`. @@ -52,18 +52,18 @@ subtext-sim stop # --shutdown also shuts the simul | Command | What it does | |---|---| -| `doctor [--fix]` | Checks Xcode, simulators, Swift, Node, the sightmap CLI; builds the helper. | -| `start [--device NAME\|UDID] [--app PATH] [--bundle-id ID] [--label L] [--detach]` | Boots, enables app accessibility, installs, launches, starts the daemon and a new run log. | +| `doctor [--fix]` | Checks Xcode, simulators, Swift, Node, the sightmap CLI (the version that resolves: `SIGHTMAP_BIN`, then PATH, then the pinned npx package; warns outside the tested 0.32–0.36 range); builds the helper. | +| `start [--device NAME\|UDID] [--app PATH] [--bundle-id ID] [--label L] [--detach]` | Boots, enables app accessibility, installs, launches, starts the daemon and a new run log. On a running session: reinstalls and relaunches the same app; errors for a different one. | | `status` / `stop [--shutdown]` | Session health / stop the daemon and the app (and the simulator). | | `launch` / `relaunch` | Foreground the app / restart it (re-enables accessibility first). | | `snapshot [--view V] [--include-offscreen] [--out F] [--tree-out F] [--json]` | Annotated tree. `--out` writes the text plus a `.tree.json` that `sightmap coverage` / `explain --snap` read. | | `click ` | Re-resolves, checks it's on screen, uncovered and enabled, taps its center. | | `fill [--clear]` | Taps, types (appends unless `--clear`), and errors if the field doesn't show the text. | | `scroll [--in ] [--until ] [--max N]` | One page, or until a target is tappable. `--in` scrolls inside that element. | -| `swipe x1 y1 x2 y2 [--duration S]`, `tap-xy x y`, `type `, `key `, `button `, `openurl ` | Raw input in screen points. Keys: `return delete tab escape space` and arrows. Buttons: `home lock siri side`. | +| `swipe x1 y1 x2 y2 [--duration S]`, `tap-xy x y`, `type `, `key `, `button `, `openurl ` | Raw input in screen points (not pixels; off-screen points are refused). Keys: `return delete tab escape space` and arrows. Buttons: `home lock siri side`. | | `screenshot --out F.png` | PNG of the screen. | | `console list [--limit N] [--grep S] [--level L] [--all]` / `console get ` | The app's own os_log/Logger output and its stdout/stderr (levels `stdout`, `stderr`). `--all` adds Apple framework errors and faults, which are hidden by default as noise. | -| `session-url` | The Fullstory session URL from the app's logs. | +| `session-url` | The Fullstory session URL from the app's console: any `https://app…fullstory.com/ui//session/…` (or `client-session`) URL in any line, source or level; the newest wins. | | `run new [--label L]` / `run current` | Start a new run log / print the current run's id and path. | | `run tail [--follow] [--since SEQ] [--run ID]` | Print `.sightmap/.sim-runs/.jsonl` entries; `--follow` keeps streaming. | @@ -83,7 +83,8 @@ Network and Fullstory-captured console data come from the app's own Fullstory SD - **mobile network capture turned on** for the org (Settings → Data Capture → Mobile; `doctor` reminds you); - the Subtext MCP authenticated to the same org. - **Console:** only `FS.log` reaches Fullstory. `print` / `os_log` output is local only; read it live with `subtext-sim console list`. -- **Session URL:** `subtext-sim session-url` returns it, and the run log records it as a `session` entry. A relaunch can start a new session; when the run log gets a new `session` entry, follow the newest one. +- **Session URL:** `subtext-sim session-url` returns it, and the run log records it as a `session` entry. It's found in any console line that contains one (a React Native `console.log('session', url)` works, as does a `FULLSTORY_SESSION_URL ` line). A relaunch can start a new session; when the run log gets a new `session` entry, follow the newest one. +- **Recording isn't reading.** Fullstory records views from inside the app; `subtext-sim` reads the accessibility tree from outside it. A healthy replay doesn't mean `snapshot` can see the screen, and an empty snapshot doesn't mean recording broke. ### Two agents, working at the same time @@ -103,7 +104,8 @@ Because data lags the app by up to ~30 s, one agent that taps, then waits, then **Consumer** reads captured data while the producer is still working: 1. Follow the producer with `subtext-sim run tail --follow` (or poll with `--since SEQ`). -2. About every 30 s while new producer steps are appearing, read the newest data: `review-open` the session URL, then `review-zoom` with `t0_ms` set past what it has already processed, and `resolution={"network": "standard", "console": "standard"}`. Use `"detail"` only on the specific requests whose headers or bodies matter. Close each client with `review-close` before opening the next. +2. About every 30 s while new producer steps are appearing, read the newest data: `review-open` the session URL, then `review-zoom` with `t0_ms` set past what it has already processed, and `resolution={"network": "machine", "console": "standard"}`. Close each client with `review-close` before opening the next. + - **Network grain:** endpoint names (method + URL) show at `machine`. `standard` collapses traffic into counts, and `detail` adds response headers only, unless the org's capture rules record more. Use `{"network": "machine"}` to name endpoints, and `detail` only on the specific requests whose headers matter. 3. Match each request and `FS.log` message to the producer step just before it (by timestamp), then author: - `requests:` (route glob + method) for endpoints that mean something; - `properties:` for outcomes hidden in bodies; @@ -119,20 +121,36 @@ Run the producer alone when only UI or components work is needed. For quick loca ## Authoring `.sightmap/` for iOS -Use the `sightmap-authoring` workflow; only the selectors differ. Elements expose `role` (from the accessibility type), `identifier` (the `accessibilityIdentifier`), `text` (the label) and `value`: +Use the `sightmap-authoring` workflow; only the selectors and routes differ. Elements expose `role` (from the accessibility type), `identifier` (the `accessibilityIdentifier`; `testid` is an alias), `text` (the label) and `value`: ```yaml +version: 1 components: - - name: SignInButton - selector: 'button[identifier="login-submit"]' - - name: ProductRow - selector: 'button[identifier^="product-row-"]' + - name: TabBarItem + selector: 'tab' properties: - name: title - extract: text + extract: { from: dom.text } +views: + - name: Products + route: /products # required by sightmap; ignored by subtext-sim (see below) + components: + - name: ProductRow + selector: '[identifier^="product-row-"]' + properties: + - name: title + extract: { from: dom.text } + - name: number + extract: { from: dom.attr, path: identifier, pattern: 'product-row-(\d+)' } ``` -Views have no URL to route on: the snapshot infers the view whose own components match the most elements (`[View: Login] (inferred)`), or pass `--view`. `/fixtures/SampleApp/.sightmap/` is a small worked example. +Write `extract` as an object (`{ from: dom.text }`, `{ from: dom.attr, path: NAME }`, `{ from: component, path: Child.prop }`, `{ from: component.exists, path: Child }`, `{ from: dom.state, path: checked }`). The object form needs sightmap 0.34+; the older string forms (`extract: text`) still load but are deprecated. Run `sightmap validate` after editing. + +**Views and routes.** sightmap requires a `route` on every view, and native screens have no URL, so give each view a short placeholder (`/login`, `/products`). `subtext-sim` ignores routes: the snapshot infers the view whose own components match the most elements (`[View: Login] (inferred)`), or pass `--view`. `/fixtures/SampleApp/.sightmap/` is a small worked example. + +**Reusing a web or React Native session corpus.** Two rules make selectors written against a recorded session match in the simulator: +- `[testid="x"]` matches the element whose accessibility identifier is `x`. +- In a descendant selector (`A B`), an ancestor step that names an exact identifier (`[testid="x"]`, `[identifier="x"]`, `#x`) is skipped when **no element in the tree** carries it, as long as the last step has an attribute of its own. A `testID` on a plain wrapper view never reaches the iOS tree, so `[testid="login-form"] [testid="login-submit"]` still matches the button. An identifier that is on screen is never skipped, `>` steps are never skipped, and `[testid="login-form"] button` doesn't match (it would match every button). Prefer `[identifier=…]`; fall back to `[text=…]` only for copy that won't change or be localized. Snapshot after editing to confirm the components resolve. @@ -142,13 +160,19 @@ Prefer `[identifier=…]`; fall back to `[text=…]` only for copy that won't ch The loop is the same; RN maps onto the iOS accessibility tree like this: -- **`testID` is the identifier.** Target it directly (`click login-submit`) and select on it (`[identifier="login-submit"]`). +- **`testID` is the identifier.** Target it directly (`click login-submit`) and select on it (`[identifier="login-submit"]` or `[testid="login-submit"]`). +- **Put `testID`s on accessible elements.** Only accessible elements reach the tree: a `Pressable`, a `TextInput`, a `View` with `accessible`. A `testID` on a plain wrapper `View` is invisible to the simulator (selectors that use it as an ancestor still work, see above). - **Role-less `Pressable`s show up as `other`**, not `button`; only `accessibilityRole="button"` makes a `button`. Write role-free selectors (`'[identifier^="product-row-"]'`), not `button[identifier=…]`. Accessible elements with a `testID` count as interactive. - **`` doesn't reach the tree** on the New Architecture: the text shows up, its identifier doesn't. Put the `testID` on an `accessible` `View` around the `Text`, or select on `[text=…]`. - **React Navigation tabs** become `tab "Products"` (the "tab, 1 of 3" suffix is stripped), so `tab` selectors and `TabBarItem[title="Network"]` work. Rows behind its JS tab bar are refused as covered, like a native one. - **Labels join child texts with ", "**: a row with "Product 1" and "$6.37" reads `"Product 1, $6.37"`. -**Embedded bundle vs Metro.** A Release build embeds `main.jsbundle` and runs without Metro; that's the most predictable for driving. A Debug build loads JS from Metro, so start Metro (`npx react-native start` in the app's directory) before `subtext-sim start`; without it the snapshot shows a red "No script URL provided" box. If port 8081 is taken, run Metro with `--port N` and point the app at it: `xcrun simctl spawn defaults write RCT_jsLocation localhost:N`, then `relaunch`. Release builds drop `console.log` below error unless the app calls `RCTSetLogThreshold(.info)` natively. +**Debug with Metro is fine.** A Debug build (or an Expo dev client) loading JS from Metro works for the whole loop and is cheaper than Release, since JS edits reload without a native rebuild; Release isn't required. Start Metro (`npx react-native start`, or `npx expo start --dev-client`) before `subtext-sim start`; without it the snapshot shows a red "No script URL provided" box. If edits don't show up, restart Metro with `--clear`: it can serve stale modules, especially from a git worktree. If port 8081 is taken, run Metro with `--port N` and point the app at it: `xcrun simctl spawn defaults write RCT_jsLocation localhost:N`, then `relaunch`. A Release build embeds `main.jsbundle` and runs without Metro; it drops `console.log` below error unless the app calls `RCTSetLogThreshold(.info)` natively. + +**Empty or nearly empty tree.** When `snapshot` says the app exposes no accessibility elements, or shows only a small floating element (e.g. the Expo dev-menu button, which lives in its own window), the likely causes in order are: +1. A view that stays mounted hides the rest: `accessibilityViewIsModal = YES`, or `accessibilityElementsHidden` / `importantForAccessibility="no-hide-descendants"` on a root view. In RN this is often a full-window overlay (a toast host, a modal backdrop) that is invisible but still mounted. Unmount it, or set those props only while it's shown. +2. App accessibility is off: `subtext-sim relaunch` turns it on. The error leaves this out when it's already on. +3. A system sheet is animating in: snapshot again in a moment. **Overlays.** - In Debug, a `console.error` or `console.warn` raises a LogBox toast over the bottom of the screen. The snapshot gets a `[LogBox]` section with the `tap-xy` that hits the toast's ✕ (the ✕ has no element of its own), and `click` refuses targets under it. Tapping the toast itself opens the full-screen LogBox inspector. To keep toasts away entirely, call `LogBox.ignoreAllLogs()` in the app. @@ -162,7 +186,7 @@ The loop is the same; RN maps onto the iOS accessibility tree like this: **Apps you build yourself must be built by Xcode** (`xcodebuild` or the Xcode app), not assembled by hand. Without the simulated entitlements Xcode embeds, the Fullstory SDK's uploads fail in the simulator, and the consumer sees no session. After signing in, iOS may show a "Save Password?" sheet over the app for a moment; if a snapshot reports a system sheet, snapshot again. -`/fixtures/RNSampleApp/build.sh` builds the same app in React Native (React Navigation tabs and stack, a `FlatList`, the same `testID`s) with its JS bundle embedded. It needs Node 22.11+ and CocoaPods, and takes several minutes the first time. `FS_ORG_ID` adds the Fullstory React Native SDK; `CONFIGURATION=Debug` builds for Metro instead. +`/fixtures/RNSampleApp/build.sh` builds the same app in React Native (React Navigation tabs and stack, a `FlatList`, the same `testID`s) with its JS bundle embedded. It needs Node 22.11+ and CocoaPods (with a UTF-8 locale: `export LANG=en_US.UTF-8`, or `pod install` fails), and takes several minutes the first time. `FS_ORG_ID` adds the Fullstory React Native SDK; `CONFIGURATION=Debug` builds for Metro instead. ## Known limits diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml index 9bb8fb1..5bdcb72 100644 --- a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/components.yaml @@ -6,4 +6,4 @@ components: selector: 'tab' properties: - name: title - extract: text + extract: { from: dom.text } diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml index c2ac1c1..39150c5 100644 --- a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/login.yaml @@ -1,6 +1,9 @@ version: 1 views: + # sightmap requires a route on every view. Native screens have no URL, so this + # is a placeholder: subtext-sim ignores routes and infers the view from its components. - name: Login + route: /login components: - name: EmailField selector: '[identifier="login-email"]' diff --git a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml index 2eb9233..6d1bb25 100644 --- a/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml +++ b/skills/subtext-ios-sim/fixtures/RNSampleApp/.sightmap/views/products.yaml @@ -1,13 +1,16 @@ version: 1 views: + # Placeholder routes (native screens have no URL); see login.yaml. - name: Products + route: /products components: - name: ProductRow selector: '[identifier^="product-row-"]' properties: - name: title - extract: text + extract: { from: dom.text } - name: ProductDetail + route: /products/detail components: - name: ProductName selector: '[identifier="product-detail-name"]' diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml index 9bb8fb1..5bdcb72 100644 --- a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/components.yaml @@ -6,4 +6,4 @@ components: selector: 'tab' properties: - name: title - extract: text + extract: { from: dom.text } diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml index 8fa150d..c9d5f00 100644 --- a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/login.yaml @@ -1,6 +1,9 @@ version: 1 views: + # sightmap requires a route on every view. Native screens have no URL, so this + # is a placeholder: subtext-sim ignores routes and infers the view from its components. - name: Login + route: /login components: - name: EmailField selector: '[identifier="login-email"]' diff --git a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml index 7892dab..b41c38a 100644 --- a/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml +++ b/skills/subtext-ios-sim/fixtures/SampleApp/.sightmap/views/products.yaml @@ -1,13 +1,16 @@ version: 1 views: + # Placeholder routes (native screens have no URL); see login.yaml. - name: Products + route: /products components: - name: ProductRow selector: '[identifier^="product-row-"]' properties: - name: title - extract: text + extract: { from: dom.text } - name: ProductDetail + route: /products/detail components: - name: ProductName selector: '[identifier="product-detail-name"]' diff --git a/skills/subtext-ios-sim/lib/BRIDGE.md b/skills/subtext-ios-sim/lib/BRIDGE.md index c605e8e..ab572d8 100644 --- a/skills/subtext-ios-sim/lib/BRIDGE.md +++ b/skills/subtext-ios-sim/lib/BRIDGE.md @@ -7,7 +7,7 @@ like `sightmap snapshot`, resolves ids and component queries, and writes ## Decision: our own matcher (b), with the corpus loaded by `sightmap export` -Checked against sightmap v0.32.0: +Checked against sightmap v0.32.0 and re-checked on v0.36.1: - **No offline command annotates a tree.** `snapshot --json` writes `component` and `props` per node, but only for a live Chrome page. Offline, `explain --snap @@ -25,10 +25,58 @@ Checked against sightmap v0.32.0: So we match in Node (`selector.mjs`: tag, `*`, `#id`, `[a]`, `= ^= $= *= ~= |=` with an optional `i`, descendant and `>` combinators, selector lists, `:not`, -`:is`, `:where`, `:has`). We extract properties (`text`, `attr=`, `Child.prop`, -`exists:Child`) and apply the live snapshot's rules: view components before -globals, first match wins, and conflicts are reported. Live `sightmap snapshot` -behaves the same way, which we verified with a probe page. +`:is`, `:where`, `:has`). We extract properties and apply the live snapshot's +rules: view components before globals, first match wins, and conflicts are +reported. Live `sightmap snapshot` behaves the same way, which we verified with +a probe page. + +## Export shape across sightmap versions + +The bridge is tested with sightmap 0.32 through 0.36 (`SIGHTMAP_TESTED` in +`sightmap-cli.mjs`; `doctor` warns outside it). The npx fallback pins 0.36.1. + +- **`extract` (0.34+).** `sightmap export` writes every component property's + `extract` as an object, `{from, path?, pattern?, join?}`; 0.32/0.33 write the + strings `text`, `attr=NAME`, `Child.prop`, `exists:Child`. `parseCorpus` + lowers both to the object form. Supported sources: `dom.text` and + `dom.raw_text` (both read the label, or the joined child text), + `dom.attr`, `dom.state` (`disabled`; `selected` from the `Selected` trait; + `checked` from a switch's `1`/`0` value; `expanded` never resolves), + `component` (with `Name[]` steps and `join`) and `component.exists`. + `pattern` takes capture group 1, else the whole match. +- **YAML.** 0.34+ reads both forms in `.sightmap/`; 0.32 can't parse the object + form ("cannot unmarshal !!map into string"), and the snapshot error says to + upgrade. The sample apps' corpora use the object form. +- **Routes.** `sightmap validate` requires a `route` on every view (any + version). Native screens have none, so corpora give each view a placeholder + route; the bridge never reads it. +- Nothing else in the export changed between 0.32 and 0.36.1 for the fields the + bridge reads (`globals`, `views[].components`, `selectors`, `parentChain`, + `memory`). + +## iOS-specific selector rules + +Two rules let a corpus written against a recorded web or React Native session +match the simulator's tree: + +- **`testid` aliases `identifier`.** Session corpora write `[testid="x"]`; the + simulator exposes the same testID as the accessibility identifier. Every + identified element carries both attributes, in `attrsOf` and so in the + `.tree.json`, which keeps sightmap's offline tools in agreement. +- **Absent identified ancestors are skipped.** In a descendant chain, ancestor + step *k* (followed by whitespace, not `>`) may be skipped when its compound + names an exact identifier (`#x`, `[identifier="x"]`, `[testid="x"]`, `[id="x"]`) + and **no element in the whole tree** matches that compound, provided the + subject step has an attribute or `#id` of its own. A testID on a plain + wrapper view (not an accessibility element) never reaches the iOS tree, so + `[testid="login-form"] [testid="login-submit"]` would otherwise never match. + An identifier present anywhere on screen is never skipped (so a + `[testid="card"] [testid="close"]` selector doesn't leak to another card's + close button outside it), prefix and substring matches are never skipped, and + `[testid="list"] button` doesn't match (it would match every button). The rule + doesn't apply to relative selectors inside `:has()`. + `sightmap explain` / `coverage` don't apply this rule, so they can report + fewer matches than the snapshot for such selectors. **We stay compatible with sightmap.** The tests write our tree as `.snap.tree.json`, run `sightmap explain --snap` over a globals-only parity diff --git a/skills/subtext-ios-sim/lib/cli.mjs b/skills/subtext-ios-sim/lib/cli.mjs index 0792fb7..735560e 100644 --- a/skills/subtext-ios-sim/lib/cli.mjs +++ b/skills/subtext-ios-sim/lib/cli.mjs @@ -20,7 +20,8 @@ export const USAGE = `subtext-sim: drive an app in the local iOS Simulator (sigh Session doctor [--fix] check prerequisites; builds the helper start [--device NAME|UDID] [--app PATH] [--bundle-id ID] [--detach] [--label L] - boot, enable accessibility, install, launch, run the daemon + boot, enable accessibility, install, launch, run the daemon; + on a running session: reinstall (--app) and relaunch the same app status session health stop [--shutdown] stop the daemon and the app (--shutdown: the simulator too) launch | relaunch foreground the app | restart it (re-enables accessibility) @@ -41,8 +42,8 @@ Act (targets: snapshot id, accessibility identifier, or component query) click fill [--clear] scroll [--in ] [--until ] [--max N] - swipe [--duration S] - tap-xy + swipe [--duration S] screen points (not pixels), origin top-left + tap-xy screen points (not pixels); off-screen points are refused type key return delete tab escape space up down left right … button home lock siri side apple_pay volume_up volume_down @@ -116,7 +117,7 @@ export function toRequest({ command, positionals: p, flags }, cwd = process.cwd( need(p, 4, 'swipe [--duration S]'); return { cmd: 'swipe', args: { x1: num(p[0], 'x1'), y1: num(p[1], 'y1'), x2: num(p[2], 'x2'), y2: num(p[3], 'y2'), ...(flags.duration ? { duration: num(flags.duration, '--duration') } : {}) } }; case 'tap-xy': - need(p, 2, 'tap-xy '); + need(p, 2, 'tap-xy (screen points, not pixels)'); return { cmd: 'tapXY', args: { x: num(p[0], 'x'), y: num(p[1], 'y') } }; case 'type': need(p, 1, 'type '); @@ -184,8 +185,11 @@ export function formatResult(cmd, r) { case 'openurl': return `✓ opened ${r.url}`; case 'screenshot': return `✓ wrote ${r.out}`; case 'launch': return `✓ launched ${r.bundleId} (pid ${r.pid})`; - case 'relaunch': case 'restart': - return `✓ relaunched ${r.bundleId} (pid ${r.pid})${r.ready === false ? '; accessibility tree not readable yet, snapshot again shortly' : ''}`; + case 'relaunch': case 'restart': { + const what = r.installed ? `installed ${r.installed} and relaunched` : 'relaunched'; + const reuse = cmd === 'restart' ? ' in the running session' : ''; + return `✓ ${what} ${r.bundleId} (pid ${r.pid})${reuse}${r.ready === false ? '; accessibility tree not readable yet, snapshot again shortly' : ''}`; + } case 'sessionUrl': return r.url; case 'runNew': case 'runCurrent': return r.run ? `${r.run}\n${r.path}` : '(no run yet)'; @@ -263,7 +267,14 @@ async function start(flags, sightmapDir, out) { if (flags.device && flags.device !== existing.udid && flags.device !== existing.deviceName) { throw new SimError('session_running', `a session is already running on ${existing.deviceName} (${existing.udid})`, 'run `subtext-sim stop` first'); } - const r = await call(existing.socket, 'restart', { appPath: flags.app ? resolve(flags.app) : undefined }); + const appPath = flags.app ? resolve(flags.app) : undefined; + if (appPath && !existsSync(appPath)) throw new SimError('bad_app', `${appPath} does not exist`, 'build the app for the simulator and pass the .app path'); + const wanted = flags['bundle-id'] ?? (appPath ? await simctl.plistValue(appPath, 'CFBundleIdentifier') : existing.bundleId); + if (wanted !== existing.bundleId) { + throw new SimError('session_running', `a session for ${existing.bundleId} is already running on ${existing.deviceName ?? existing.udid}; it can't switch to ${wanted}`, + `run \`subtext-sim stop\`, then \`subtext-sim start ${appPath ? `--app ${JSON.stringify(appPath)}` : `--bundle-id ${wanted}`} --detach\``); + } + const r = await call(existing.socket, 'restart', { appPath }); return { cmd: 'restart', result: r }; } @@ -297,7 +308,7 @@ function formatStart(r, cwd) { const lines = [ `✓ simulator session started in ${r.seconds.toFixed(1)} s`, ` device: ${r.deviceName} (${r.udid})`, - ` app: ${r.bundleId}${r.appPath ? ` ${r.appPath}` : ''}`, + ` app: ${r.appPath ? `installed ${r.appPath} and launched ${r.bundleId}` : `launched ${r.bundleId} (already installed)`}`, ` daemon: pid ${r.pid}; session file ${relative(cwd, r.sessionFile) || r.sessionFile}`, ` run: ${r.run} (\`subtext-sim run tail --follow\` to follow)`, ]; diff --git a/skills/subtext-ios-sim/lib/corpus.mjs b/skills/subtext-ios-sim/lib/corpus.mjs index 93477f1..5e7ca34 100644 --- a/skills/subtext-ios-sim/lib/corpus.mjs +++ b/skills/subtext-ios-sim/lib/corpus.mjs @@ -32,11 +32,31 @@ export function loadCorpusFile(path) { return parseCorpus(JSON.parse(readFileSync(path, 'utf8'))); } +const EXTRACT_KEYS = ['from', 'path', 'pattern', 'join']; + +/** + * sightmap 0.34+ exports every `extract` as `{from, path, pattern, join}`; + * older releases (and legacy YAML) use the strings `text`, `attr=NAME`, + * `Child.prop` and `exists:Child`. Both become the object form. + */ +export function normalizeExtract(extract) { + if (extract == null || extract === '') return { from: 'dom.text' }; + if (typeof extract === 'object') { + return Object.fromEntries(EXTRACT_KEYS.filter((k) => extract[k] !== undefined && extract[k] !== null).map((k) => [k, extract[k]])); + } + const s = String(extract); + if (s === 'text') return { from: 'dom.text' }; + if (s === 'raw_text') return { from: 'dom.raw_text' }; + if (s.startsWith('attr=')) return { from: 'dom.attr', path: s.slice(5) }; + if (s.startsWith('exists:')) return { from: 'component.exists', path: s.slice(7) }; + return { from: 'component', path: s }; +} + export function parseCorpus(wire) { const component = (c) => ({ name: c.name, selectors: c.selectors ?? [], - properties: c.properties ?? [], + properties: (c.properties ?? []).map((p) => ({ ...p, extract: normalizeExtract(p.extract) })), parentChain: c.parentChain ?? [], memory: c.memory ?? [], }); diff --git a/skills/subtext-ios-sim/lib/daemon.mjs b/skills/subtext-ios-sim/lib/daemon.mjs index ecbf8cc..2a964c0 100644 --- a/skills/subtext-ios-sim/lib/daemon.mjs +++ b/skills/subtext-ios-sim/lib/daemon.mjs @@ -5,8 +5,8 @@ import { tmpdir } from 'node:os'; import { dirname, join, resolve as resolvePath } from 'node:path'; import { createInterface } from 'node:readline'; import { BridgeError, annotate, centerOf, render, resolve as resolveQuery, toComponentTree, writeSnap } from './sightmap-bridge.mjs'; -import { logBoxToast, normalize } from './normalize.mjs'; -import { SimError, toWire } from './errors.mjs'; +import { floatingOnly, logBoxToast, normalize } from './normalize.mjs'; +import { emptyTreeMessage, SimError, toWire } from './errors.mjs'; import { ConsoleBuffer, LogCollector } from './logs.mjs'; import { HelperClient } from './helper-client.mjs'; import { CorpusCache } from './sightmap-cli.mjs'; @@ -203,15 +203,34 @@ export class SimSession { } async fresh({ view } = {}) { + let described; try { - const snap = annotate(normalize(await this.describeSettled()), this.corpus.get(), { view }); + described = await this.describeSettled(); + } catch (err) { + if (err.code === 'ax_disabled') throw await this.emptyTreeError(); + throw err; + } + try { + const snap = annotate(normalize(described), this.corpus.get(), { view }); this.lastView = snap.view; + this.screen = snap.screen; return snap; } catch (err) { throw bridgeToSim(err); } } + async accessibilityEnabled() { + if (this.axEnabled === undefined) { + this.axEnabled = this.simctl.accessibilityEnabled ? await this.simctl.accessibilityEnabled(this.udid).catch(() => null) : null; + } + return this.axEnabled; + } + + async emptyTreeError() { + return new SimError('ax_disabled', emptyTreeMessage({ udid: this.udid, accessibilityEnabled: await this.accessibilityEnabled() })); + } + /** Re-describes until two reads agree on every element's frame, so a list's momentum has stopped. */ async settled({ timeoutMs = 4000 } = {}) { const layout = (snap) => snap.nodes.map((n) => `${n.tag}|${n.identifier}|${n.label}|${Math.round(n.frame.x)},${Math.round(n.frame.y)}`).join('\n'); @@ -233,6 +252,7 @@ export class SimSession { async launchApp({ terminate = true } = {}) { if (terminate) await this.simctl.enableAccessibility(this.udid); this.appPid = await this.simctl.launch(this.udid, this.bundleId, { stdout: this.stdoutPath, stderr: this.stderrPath, terminate }); + this.axEnabled = undefined; this.readable = false; this.last = null; return { bundleId: this.bundleId, pid: this.appPid }; @@ -326,6 +346,10 @@ export class SimSession { async snapshot({ view, includeOffscreen = false, out, treeOut, json = false }) { const snap = await this.fresh({ view }); + if (floatingOnly(snap)) { + Object.assign(snap, { udid: this.udid, accessibilityEnabled: await this.accessibilityEnabled() }); + snap.text = render(snap); + } this.last = snap; const text = includeOffscreen ? render(snap, { includeOffscreen: true }) : snap.text; if (out) writeSnap({ ...snap, text }, out); @@ -417,12 +441,14 @@ export class SimSession { }, async swipe({ x1, y1, x2, y2, duration }) { + await this.assertOnScreen([[x1, y1], [x2, y2]], 'swipe'); await this.input('swipe', { x1, y1, x2, y2, ...(duration ? { duration } : {}) }); this.last = null; return { x1, y1, x2, y2 }; }, async tapXY({ x, y }) { + await this.assertOnScreen([[x, y]], 'tap-xy'); await this.input('tap', { x, y }); return { x, y }; }, @@ -467,7 +493,7 @@ export class SimSession { await this.simctl.install(this.udid, appPath); this.appPath = appPath; } - return this.commands.relaunch.call(this); + return { ...(await this.commands.relaunch.call(this)), installed: appPath ?? null }; }, consoleList({ limit = 50, grep, level, all }) { @@ -489,6 +515,17 @@ export class SimSession { }, }; + /** Rejects points outside the screen, in points as `describe` reports it. Without a readable tree, the last known size applies. */ + async assertOnScreen(points, cmd) { + const screen = this.screen ?? (await this.fresh().catch(() => null))?.screen; + if (!screen?.w || !screen?.h) return; + const outside = points.find(([x, y]) => x < 0 || y < 0 || x > screen.w || y > screen.h); + if (outside) { + throw new SimError('off_screen_point', `(${outside[0]}, ${outside[1]}) is outside the ${screen.w}x${screen.h} pt screen; ${cmd} takes screen points (origin top-left), not pixels`, + `pick a point within 0–${screen.w} x 0–${screen.h}; divide screenshot pixel coordinates by the device scale (2 or 3)`); + } + } + /** Re-describes, re-resolves the target, checks it is tappable, and taps its center. */ async tap(target) { const numeric = /^\d+$/.test(String(target).trim()); diff --git a/skills/subtext-ios-sim/lib/doctor.mjs b/skills/subtext-ios-sim/lib/doctor.mjs index 212f768..218f91d 100644 --- a/skills/subtext-ios-sim/lib/doctor.mjs +++ b/skills/subtext-ios-sim/lib/doctor.mjs @@ -2,10 +2,10 @@ import { execFile } from 'node:child_process'; import { existsSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; +import { resolveSightmap, SIGHTMAP_PACKAGE, sightmapVersionWarning } from './sightmap-cli.mjs'; const SKILL_DIR = join(dirname(fileURLToPath(import.meta.url)), '..'); const HELPER_BUILD = join(SKILL_DIR, 'helper', 'build.sh'); -const SIGHTMAP_PACKAGE = '@sightmap/sightmap@0.32.0'; const MIN_XCODE = 26; const MIN_MACOS = 15; const MIN_NODE = 18; @@ -95,11 +95,19 @@ async function checkNode() { } async function checkSightmap() { - const onPath = await run('sightmap', ['version']); - if (onPath.ok) return pass('sightmap', `${onPath.stdout} (on PATH)`); - const viaNpx = await run('npx', ['-y', SIGHTMAP_PACKAGE, 'version'], { timeout: 180_000 }); - if (viaNpx.ok) return pass('sightmap', `${viaNpx.stdout} (via npx ${SIGHTMAP_PACKAGE})`); - return fail('sightmap', `sightmap not on PATH and npx ${SIGHTMAP_PACKAGE} failed: ${viaNpx.stderr.split('\n').at(-1)}`, `npm install -g ${SIGHTMAP_PACKAGE}`, ['npm', ['install', '-g', SIGHTMAP_PACKAGE]]); + let resolved; + try { + resolved = resolveSightmap(); + } catch (err) { + return fail('sightmap', err.message, `npm install -g ${SIGHTMAP_PACKAGE}`, ['npm', ['install', '-g', SIGHTMAP_PACKAGE]]); + } + const detail = `sightmap ${resolved.version} (${resolved.source})`; + const warning = sightmapVersionWarning(resolved.version); + if (!warning) return pass('sightmap', detail); + const fix = resolved.source.startsWith('SIGHTMAP_BIN') + ? `point SIGHTMAP_BIN at a sightmap in the tested range, or unset it to use npx ${SIGHTMAP_PACKAGE}` + : `npm install -g ${SIGHTMAP_PACKAGE} (a sightmap on PATH wins over the npx pin)`; + return { ...pass('sightmap', `${detail}; ${warning}`), warn: true, fix }; } async function checkHelper() { @@ -126,17 +134,19 @@ function checkMobileNetworkCapture() { } const ICON = { true: 'ok ', false: 'FAIL', null: 'PEND' }; +const iconOf = (c) => (c.todo ? 'TODO' : c.warn ? 'WARN' : ICON[c.ok]); function report(checks) { const width = Math.max(...checks.map((c) => c.name.length)); const lines = checks.map((c) => { - const head = `[${c.todo ? 'TODO' : ICON[c.ok]}] ${c.name.padEnd(width)} ${c.detail}`; + const head = `[${iconOf(c)}] ${c.name.padEnd(width)} ${c.detail}`; return c.fix ? `${head}\n ${' '.repeat(width)} fix: ${c.fix}` : head; }); const failed = checks.filter((c) => c.ok === false).length; const pending = checks.filter((c) => c.ok === null && !c.todo).length; const manual = checks.filter((c) => c.todo).length; - const notes = [pending && `${pending} pending`, manual && `${manual} to verify manually`].filter(Boolean).join(', '); + const warned = checks.filter((c) => c.warn).length; + const notes = [warned && `${warned} warning(s)`, pending && `${pending} pending`, manual && `${manual} to verify manually`].filter(Boolean).join(', '); lines.push('', failed ? `${failed} check(s) failed.` : `All checks passed${notes ? ` (${notes})` : ''}.`); console.log(lines.join('\n')); } diff --git a/skills/subtext-ios-sim/lib/errors.mjs b/skills/subtext-ios-sim/lib/errors.mjs index fbf6d90..2ea4281 100644 --- a/skills/subtext-ios-sim/lib/errors.mjs +++ b/skills/subtext-ios-sim/lib/errors.mjs @@ -11,7 +11,7 @@ const HINTS = { daemon_unreachable: 'run `subtext-sim start --detach` to start a new session', device_not_found: 'list devices with `xcrun simctl list devices available`, then pass --device', not_booted: 'run `subtext-sim start` to boot the simulator', - ax_disabled: 'snapshot again in a moment (a system sheet such as "Save Password?" may be animating in); if it persists, run `subtext-sim relaunch` (it re-enables app accessibility before launching)', + ax_disabled: 'check the causes listed, in order, then snapshot again', ax_unavailable: 'the app may still be launching; retry, or run `subtext-sim relaunch`', ax_error: 'the app may not be in the foreground; run `subtext-sim launch`, or `subtext-sim relaunch`', hid_error: 'retry; if it keeps failing, run `subtext-sim stop` then `subtext-sim start`', @@ -27,6 +27,22 @@ const HINTS = { usage: 'run `subtext-sim help`', }; +/** + * Why the app's accessibility tree can come back empty, most likely first. When + * app accessibility is known to be on, that cause is dropped. + */ +export function emptyTreeMessage({ udid, accessibilityEnabled = null }) { + const causes = [ + 'A view that stays mounted hides everything else: `accessibilityViewIsModal = YES`, or `accessibilityElementsHidden` / React Native `importantForAccessibility="no-hide-descendants"` on a root view (common in React Native, e.g. a full-window overlay that is invisible but still mounted). Check the app\'s root and overlay views.', + ...(accessibilityEnabled === true ? [] : [ + `App accessibility is off on the simulator. Run \`subtext-sim relaunch\` (it enables it first), or \`xcrun simctl spawn ${udid} defaults write com.apple.Accessibility ApplicationAccessibilityEnabled -bool true\` and relaunch the app.`, + ]), + 'A system sheet (e.g. "Save Password?") is animating in. Snapshot again in a moment.', + ]; + const note = accessibilityEnabled === true ? ' App accessibility is enabled on this simulator, so that is not the cause.' : ''; + return [`the app exposes no accessibility elements.${note} Likely causes, most likely first:`, ...causes.map((c, i) => ` ${i + 1}. ${c}`)].join('\n'); +} + export function toWire(err) { return { code: err.code ?? 'internal', message: err.message, hint: err.hint ?? HINTS[err.code] ?? null }; } diff --git a/skills/subtext-ios-sim/lib/logs.mjs b/skills/subtext-ios-sim/lib/logs.mjs index 65d87cf..9656404 100644 --- a/skills/subtext-ios-sim/lib/logs.mjs +++ b/skills/subtext-ios-sim/lib/logs.mjs @@ -26,9 +26,16 @@ export function parseLogLine(line) { }; } +// Any Fullstory app host (app.fullstory.com, app.eu1.fullstory.com, …), anywhere +// in the line: React Native logs it as one argument among several, often quoted. +// A query or fragment (a replay position) is dropped. +const FULLSTORY_SESSION = /https:\/\/app(?:\.[a-z0-9-]+)*\.fullstory\.com\/ui\/[^/\s"'`]+\/(?:client-)?session\/[^\s"'`<>()[\]{},?#]+/gi; const SESSION_URL = [/FULLSTORY_SESSION_URL\s+(\S+)/, /Beginning session:?\s*(https?:\/\/\S+)/i]; +/** The Fullstory session URL in a console message (the last one, if it has several), or null. */ export function sessionUrlIn(message) { + const fullstory = String(message).match(FULLSTORY_SESSION); + if (fullstory) return fullstory.at(-1).replace(/[.;:!?]+$/, ''); for (const re of SESSION_URL) { const m = re.exec(message); if (m) return m[1]; diff --git a/skills/subtext-ios-sim/lib/normalize.mjs b/skills/subtext-ios-sim/lib/normalize.mjs index c5d8c73..0274d7d 100644 --- a/skills/subtext-ios-sim/lib/normalize.mjs +++ b/skills/subtext-ios-sim/lib/normalize.mjs @@ -132,6 +132,7 @@ export function normalize(describeResult) { identifier: present(raw.identifier), value: present(raw.value), enabled: raw.enabled !== false, + traits: Array.isArray(raw.traits) ? raw.traits : [], frame, visible: intersects(frame, screen), onScreen: centerOnScreen(frame, screen), @@ -165,15 +166,36 @@ export function logBoxToast({ nodes, screen }) { return { node: toast, dismiss: { x: Math.round(x + w - 24), y: Math.round(y + h / 2) } }; } +/** + * When everything visible fits in a small box (e.g. only the React Native / Expo + * dev-menu button, which lives in a separate window), the app's own window is + * most likely hidden from accessibility. Returns the visible nodes, or null. + */ +export function floatingOnly({ nodes, root, screen, overlay }) { + if (overlay) return null; + const shown = nodes.filter((n) => n !== root && n.visible && n.tag !== 'window' && !n.ignored); + if (!shown.length) return null; + const x0 = Math.min(...shown.map((n) => n.frame.x)); + const y0 = Math.min(...shown.map((n) => n.frame.y)); + const x1 = Math.max(...shown.map((n) => n.frame.x + n.frame.w)); + const y1 = Math.max(...shown.map((n) => n.frame.y + n.frame.h)); + return (x1 - x0) * (y1 - y0) < 0.1 * screen.w * screen.h ? shown : null; +} + const CSS_IDENT = /^-?[A-Za-z_][A-Za-z0-9_-]*$/; export function cssId(node) { return node.identifier && CSS_IDENT.test(node.identifier) ? node.identifier : null; } +// Web and React Native session corpora select on `testid`; in the simulator the +// same testID arrives as the accessibility identifier. export function attrsOf(node) { const attrs = {}; - if (node.identifier) attrs.identifier = node.identifier; + if (node.identifier) { + attrs.identifier = node.identifier; + attrs.testid = node.identifier; + } if (cssId(node)) attrs.id = node.identifier; if (node.label) attrs.text = node.label; if (node.value) attrs.value = node.value; diff --git a/skills/subtext-ios-sim/lib/selector.mjs b/skills/subtext-ios-sim/lib/selector.mjs index 5c92386..665763a 100644 --- a/skills/subtext-ios-sim/lib/selector.mjs +++ b/skills/subtext-ios-sim/lib/selector.mjs @@ -238,6 +238,53 @@ function hasMatch(scope, list) { return false; } +const IDENTIFIER_ATTRS = new Set(['identifier', 'testid', 'id']); +const namesIdentifier = (c) => c.ids.length > 0 || c.attrs.some((a) => IDENTIFIER_ATTRS.has(a.name) && a.op === '='); +const discriminates = (c) => c.ids.length > 0 || c.attrs.some((a) => a.op !== null); + +function rootOf(node) { + let r = node; + while (r.parent) r = r.parent; + return r; +} + +const absentCache = new WeakMap(); + +/** True when no element of the tree that `node` belongs to matches the compound on its own. */ +function absentFromTree(node, compound) { + const root = rootOf(node); + let perRoot = absentCache.get(root); + if (!perRoot) absentCache.set(root, (perRoot = new Map())); + if (!perRoot.has(compound)) { + let found = matchCompound(root, compound); + for (const d of descendants(root)) if (found || (found = matchCompound(d, compound))) break; + perRoot.set(compound, !found); + } + return perRoot.get(compound); +} + +/** + * Ancestor step `k` of a descendant chain may be skipped when it names an exact + * identifier that no element of the tree carries, and the subject step has an + * attribute of its own: a testID on a plain wrapper view never reaches the iOS + * accessibility tree, so its descendants would otherwise never match, while a + * bare `[testid="list"] button` would match every button. + */ +function skippable(node, parts, k, scope) { + return !scope && k < parts.length - 1 && parts[k + 1].combinator === ' ' + && discriminates(parts.at(-1).compound) + && namesIdentifier(parts[k].compound) && absentFromTree(node, parts[k].compound); +} + +function matchAncestors(node, parts, k, scope) { + for (let a = node.parent; a; a = a.parent) { + if (scope && a === scope) return false; + if (matchFrom(a, parts, k, scope)) return true; + } + if (!skippable(node, parts, k, scope)) return false; + return k === 0 || matchAncestors(node, parts, k - 1, scope); +} + function matchFrom(node, parts, k, scope) { if (!matchCompound(node, parts[k].compound)) return false; const { combinator } = parts[k]; @@ -248,11 +295,7 @@ function matchFrom(node, parts, k, scope) { return false; } if (combinator === '>') return !!node.parent && matchFrom(node.parent, parts, k - 1, scope); - for (let a = node.parent; a; a = a.parent) { - if (scope && a === scope) return false; - if (matchFrom(a, parts, k - 1, scope)) return true; - } - return false; + return matchAncestors(node, parts, k - 1, scope); } function matchesList(node, list, scope = null) { diff --git a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs index 38f0d6a..6d47c87 100644 --- a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs +++ b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs @@ -1,6 +1,7 @@ import { mkdirSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; -import { attrsOf, cssId, logBoxToast, normalize } from './normalize.mjs'; +import { attrsOf, cssId, floatingOnly, logBoxToast, normalize } from './normalize.mjs'; +import { emptyTreeMessage } from './errors.mjs'; import { compareValue, matches } from './selector.mjs'; import { loadCorpus } from './corpus.mjs'; @@ -26,6 +27,10 @@ function firstDescendant(node, componentName) { return null; } +function allDescendants(node, componentName) { + return [...descendants(node)].filter((d) => d.component === componentName); +} + function matchedNodes(nodes, def) { return nodes.filter((n) => def.selectors.some((sel) => matches(n, sel))); } @@ -50,24 +55,63 @@ function extractProps(node, memo) { const props = {}; memo.set(node, props); for (const { name, extract } of node.componentDef.properties) { - const value = extractOne(node, extract ?? 'text', memo); + const value = extractOne(node, extract ?? { from: 'dom.text' }, memo); if (value !== undefined && value !== null && value !== '') props[name] = String(value); } return props; } -function extractOne(node, extract, memo) { - if (extract === 'text') return node.label ?? node.text; - if (extract.startsWith('attr=')) return attrsOf(node)[extract.slice(5).toLowerCase()]; - if (extract.startsWith('exists:')) return firstDescendant(node, extract.slice(7)) ? 'true' : undefined; - const path = extract.split('.'); - const prop = path.pop(); - let cur = node; - for (const name of path) { - cur = firstDescendant(cur, name); - if (!cur) return undefined; +/** Follows `Child.Grandchild` (a `Name[]` step takes every match) from a node to the components it names. */ +function componentsAt(node, names) { + let current = [node]; + for (const step of names) { + const every = step.endsWith('[]'); + const name = every ? step.slice(0, -2) : step; + current = current.flatMap((n) => (every ? allDescendants(n, name) : [firstDescendant(n, name)].filter(Boolean))); + } + return current; +} + +const SWITCH_STATE = { 1: 'true', 0: 'false' }; + +function stateOf(node, state) { + switch (state) { + case 'disabled': return node.enabled ? 'false' : 'true'; + case 'selected': return node.traits.includes('Selected') ? 'true' : 'false'; + case 'checked': return ['switch', 'checkbox'].includes(node.tag) ? SWITCH_STATE[node.value] : undefined; + default: return undefined; + } +} + +function readSource(node, { from, path, join }, memo) { + switch (from) { + case 'dom.text': + case 'dom.raw_text': + return node.label ?? node.text; + case 'dom.attr': + return path ? attrsOf(node)[path.toLowerCase()] : undefined; + case 'dom.state': + return stateOf(node, path); + case 'component.exists': + return path && componentsAt(node, path.split('.')).length ? 'true' : undefined; + case 'component': { + if (!path) return undefined; + const names = path.split('.'); + const prop = names.pop(); + const values = componentsAt(node, names).map((n) => extractProps(n, memo)[prop]).filter((v) => v !== undefined && v !== ''); + if (join !== undefined) return values.length ? values.join(join) : undefined; + return values[0]; + } + default: + return undefined; } - return extractProps(cur, memo)[prop]; +} + +function extractOne(node, extract, memo) { + const value = readSource(node, extract, memo); + if (value === undefined || value === null || !extract.pattern) return value; + const m = new RegExp(extract.pattern).exec(String(value)); + return m ? (m[1] ?? m[0]) : undefined; } function computeCoverage(nodes) { @@ -191,6 +235,13 @@ export function render(annotated, { includeOffscreen = false, showIdentifiers = 'Dismiss it with one of its buttons, e.g. `click \'button[text="Not Now"]\'`, then snapshot again.', ''); } + const floating = floatingOnly(annotated); + if (floating) { + const desc = floating.slice(0, 3).map((n) => `${n.tag}${n.label ? ` ${quote(n.label)}` : ''}`).join(', '); + lines.push('[Mostly empty]', `Only a small floating element is exposed (${desc}). It is probably a separate window, such as the React Native / Expo dev-menu button, and the app's own window shows nothing to accessibility.`, + emptyTreeMessage({ udid: annotated.udid ?? '', accessibilityEnabled: annotated.accessibilityEnabled ?? null }).split('\n').slice(1).join('\n'), ''); + } + const logBox = logBoxToast(annotated); if (logBox) { lines.push('[LogBox]', `A React Native LogBox toast (#${logBox.node.id}) covers the bottom of the screen; tapping it opens the LogBox inspector.`, diff --git a/skills/subtext-ios-sim/lib/sightmap-cli.mjs b/skills/subtext-ios-sim/lib/sightmap-cli.mjs index c39ecd6..1611444 100644 --- a/skills/subtext-ios-sim/lib/sightmap-cli.mjs +++ b/skills/subtext-ios-sim/lib/sightmap-cli.mjs @@ -5,28 +5,70 @@ import { dirname, join } from 'node:path'; import { parseCorpus } from './corpus.mjs'; import { SimError } from './errors.mjs'; -export const SIGHTMAP_PACKAGE = '@sightmap/sightmap@0.32.0'; +export const SIGHTMAP_VERSION = '0.36.1'; +export const SIGHTMAP_PACKAGE = `@sightmap/sightmap@${SIGHTMAP_VERSION}`; +/** The sightmap releases the bridge is tested with: [min, maxExclusive). */ +export const SIGHTMAP_TESTED = ['0.32.0', '0.37.0']; +/** The first release that reads `extract: { from, path }` objects in YAML. */ +export const SIGHTMAP_EXTRACT_OBJECT = '0.34.0'; export const CACHE_ROOT = process.env.SUBTEXT_SIM_CACHE || join(homedir(), 'Library', 'Caches', 'subtext-sim'); -const NPX_PATH_CACHE = join(CACHE_ROOT, `sightmap-${SIGHTMAP_PACKAGE.split('@').at(-1)}.path`); +const NPX_PATH_CACHE = join(CACHE_ROOT, `sightmap-${SIGHTMAP_VERSION}.path`); -const works = (bin) => spawnSync(bin, ['version'], { encoding: 'utf8', timeout: 30_000 }).status === 0; +/** `sightmap version` → "0.36.1", or null when the binary doesn't run. */ +export function sightmapVersion(bin) { + const r = spawnSync(bin, ['version'], { encoding: 'utf8', timeout: 30_000 }); + if (r.status !== 0) return null; + return /(\d+\.\d+\.\d+)/.exec(r.stdout)?.[1] ?? 'unknown'; +} + +export function compareVersions(a, b) { + const pa = a.split('.').map(Number); + const pb = b.split('.').map(Number); + for (let i = 0; i < 3; i++) if ((pa[i] ?? 0) !== (pb[i] ?? 0)) return (pa[i] ?? 0) - (pb[i] ?? 0); + return 0; +} -/** SIGHTMAP_BIN, then `sightmap` on PATH, then the pinned package through npx (its resolved path is cached). */ -export function resolveSightmapBin() { - if (process.env.SIGHTMAP_BIN) return process.env.SIGHTMAP_BIN; - if (works('sightmap')) return 'sightmap'; +/** Why a sightmap version may not work with the bridge, or null when it is in the tested range. */ +export function sightmapVersionWarning(version) { + if (!version || version === 'unknown') return 'could not read its version'; + const [min, max] = SIGHTMAP_TESTED; + if (compareVersions(version, min) < 0 || compareVersions(version, max) >= 0) { + return `outside the range subtext-sim is tested with (${min} to <${max})`; + } + if (compareVersions(version, SIGHTMAP_EXTRACT_OBJECT) < 0) { + return `it can't load \`extract: { from: … }\` objects (sightmap ${SIGHTMAP_EXTRACT_OBJECT}+), which the sample apps' .sightmap/ use`; + } + return null; +} + +/** + * SIGHTMAP_BIN, then `sightmap` on PATH, then the pinned package through npx + * (its resolved path is cached). Returns `{ bin, source, version }`. + */ +export function resolveSightmap() { + if (process.env.SIGHTMAP_BIN) { + const bin = process.env.SIGHTMAP_BIN; + return { bin, source: `SIGHTMAP_BIN=${bin}`, version: sightmapVersion(bin) }; + } + const onPath = sightmapVersion('sightmap'); + if (onPath) { + const where = spawnSync('/bin/sh', ['-c', 'command -v sightmap'], { encoding: 'utf8' }).stdout.trim(); + return { bin: 'sightmap', source: `on PATH: ${where || 'sightmap'}`, version: onPath }; + } if (existsSync(NPX_PATH_CACHE)) { const cached = readFileSync(NPX_PATH_CACHE, 'utf8').trim(); - if (cached && existsSync(cached)) return cached; + const version = cached && existsSync(cached) ? sightmapVersion(cached) : null; + if (version) return { bin: cached, source: `npx ${SIGHTMAP_PACKAGE}`, version }; } const r = spawnSync('npx', ['-y', '-p', SIGHTMAP_PACKAGE, 'sh', '-c', 'command -v sightmap'], { encoding: 'utf8', timeout: 180_000 }); const path = r.status === 0 ? r.stdout.trim().split('\n').at(-1) : ''; - if (!path || !works(path)) { + const version = path ? sightmapVersion(path) : null; + if (!version) { throw new SimError('corpus_unavailable', `sightmap CLI not found (no SIGHTMAP_BIN, not on PATH, and npx ${SIGHTMAP_PACKAGE} failed)`); } mkdirSync(dirname(NPX_PATH_CACHE), { recursive: true }); writeFileSync(NPX_PATH_CACHE, `${path}\n`); - return path; + return { bin: path, source: `npx ${SIGHTMAP_PACKAGE}`, version }; } function yamlFiles(dir) { @@ -51,7 +93,7 @@ export class CorpusCache { this.dir = sightmapDir; this.signature = null; this.corpus = EMPTY; - this.bin = null; + this.sightmap = null; } get() { @@ -61,15 +103,21 @@ export class CorpusCache { if (!files.length) { this.corpus = EMPTY; } else { - this.bin ??= resolveSightmapBin(); + this.sightmap ??= resolveSightmap(); let out; try { - out = execFileSync(this.bin, ['export', '--sightmap-dir', this.dir], { + out = execFileSync(this.sightmap.bin, ['export', '--sightmap-dir', this.dir], { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], maxBuffer: 64 * 1024 * 1024, }); } catch (err) { - const detail = String(err.stderr || err.message).trim().split('\n').at(-1); - throw new SimError('corpus_invalid', `could not load ${this.dir}: ${detail}`, 'fix the YAML under .sightmap/, then retry'); + const stderr = String(err.stderr || err.message).trim(); + const detail = stderr.split('\n').filter(Boolean).at(-1); + const { version, source } = this.sightmap; + if (/cannot unmarshal !!map into string/.test(stderr) && version && version !== 'unknown' && compareVersions(version, SIGHTMAP_EXTRACT_OBJECT) < 0) { + throw new SimError('corpus_invalid', `could not load ${this.dir}: it uses \`extract: { from: … }\` objects, which need sightmap ${SIGHTMAP_EXTRACT_OBJECT}+, and the sightmap in use is ${version} (${source})`, + `upgrade it (\`npm install -g ${SIGHTMAP_PACKAGE}\`) or point SIGHTMAP_BIN at a newer one, then retry`); + } + throw new SimError('corpus_invalid', `could not load ${this.dir}: ${detail}`, 'fix the YAML under .sightmap/ (`sightmap validate` shows every problem), then retry'); } this.corpus = parseCorpus(JSON.parse(out)); } diff --git a/skills/subtext-ios-sim/test/bridge.test.mjs b/skills/subtext-ios-sim/test/bridge.test.mjs index 8ad6ad5..bf8b993 100644 --- a/skills/subtext-ios-sim/test/bridge.test.mjs +++ b/skills/subtext-ios-sim/test/bridge.test.mjs @@ -194,7 +194,7 @@ test('toComponentTree emits sightmap ComponentNode shape', () => { assert.equal(n.component, undefined); } const email = all.find((n) => n.element.attrs?.identifier === 'simple.email'); - assert.deepEqual(email.element, { tag: 'textfield', attrs: { identifier: 'simple.email', value: 'Email' } }); + assert.deepEqual(email.element, { tag: 'textfield', attrs: { identifier: 'simple.email', testid: 'simple.email', value: 'Email' } }); const last = all.find((n) => n.element.attrs?.identifier === 'lastTapped'); assert.equal(last.element.id, 'lastTapped'); const annotated = toComponentTree(snap, { annotations: true }); @@ -259,3 +259,94 @@ test('sightmap coverage reads an iOS snap', needsSightmap, () => { assert.match(run.stdout, /9 interactive · \d+ direct T1/); assert.doesNotMatch(run.stdout + run.stderr, /parse tree JSON/); }); + +const LEGACY_TO_OBJECT = { + text: { from: 'dom.text' }, + 'attr=value': { from: 'dom.attr', path: 'value' }, + 'attr=identifier': { from: 'dom.attr', path: 'identifier' }, + 'Title.text': { from: 'component', path: 'Title.text' }, + 'Price.text': { from: 'component', path: 'Price.text' }, + 'exists:FavoriteButton': { from: 'component.exists', path: 'FavoriteButton' }, +}; + +test('extract objects (sightmap 0.34+) resolve like the legacy strings', () => { + const wire = JSON.parse(readFileSync(join(here, 'fixtures', 'corpus.json'), 'utf8')); + const asObjects = JSON.parse(JSON.stringify(wire, (key, value) => (key === 'extract' ? LEGACY_TO_OBJECT[value] : value))); + assert.deepEqual(parseCorpus(asObjects), corpus); + for (const name of ['simple', 'heavy', 'eager']) { + const legacy = snapshot(fixture(name), corpus); + const objects = snapshot(fixture(name), parseCorpus(asObjects)); + assert.deepEqual(objects.nodes.map((n) => n.props), legacy.nodes.map((n) => n.props)); + } +}); + +test('extract objects support pattern, join over Name[], and dom.state', () => { + const row = 'button:has(> button[identifier$=".buy"])'; + const objectCorpus = parseCorpus({ + globals: [ + { + name: 'ProductRow', selectors: [row], properties: [ + { name: 'cents', extract: { from: 'component', path: 'Price.text', pattern: '\\.(\\d+)$' } }, + { name: 'buttons', extract: { from: 'component', path: 'RowButton[].text', join: '|' } }, + { name: 'number', extract: { from: 'dom.attr', path: 'identifier', pattern: 'row\\.(\\d+)\\.buy' } }, + ], + }, + { name: 'Price', selectors: [`${row} text[text^="$"]`], parentChain: ['ProductRow'], properties: [{ name: 'text', extract: { from: 'dom.text' } }] }, + { name: 'RowButton', selectors: [`${row} button`], parentChain: ['ProductRow'], properties: [{ name: 'text', extract: { from: 'dom.text' } }] }, + ], + views: [], + }); + const snap = snapshot(fixture('heavy'), objectCorpus); + const row3 = snap.nodes.filter((n) => n.component === 'ProductRow')[3]; + assert.deepEqual(row3.props, { cents: '00', buttons: 'Buy|Favorite 3', number: '3' }); + + const toggleCorpus = parseCorpus({ + globals: [{ + name: 'Toggle', selectors: ['switch'], properties: [ + { name: 'on', extract: { from: 'dom.state', path: 'checked' } }, + { name: 'disabled', extract: { from: 'dom.state', path: 'disabled' } }, + ], + }], + }); + assert.deepEqual(byIdentifier(snapshot(fixture('simple'), toggleCorpus), 'simple.toggle').props, { on: 'true', disabled: 'false' }); +}); + +const node = (type, frame, extra = {}) => ({ role: `AX${type}`, type, label: null, identifier: null, value: null, frame, enabled: true, children: [], ...extra }); + +/** A React Native screen whose testID'd wrapper views never reach the accessibility tree. */ +const rnScreen = () => normalize({ + screen: { w: 390, h: 844 }, + root: node('Application', { x: 0, y: 0, w: 390, h: 844 }, { + label: 'App', + children: [ + node('Group', { x: 0, y: 0, w: 390, h: 844 }, { + children: [ + node('Button', { x: 20, y: 100, w: 200, h: 44 }, { label: 'Sign In', identifier: 'login-submit' }), + node('Button', { x: 20, y: 160, w: 200, h: 44 }, { label: 'Help' }), + node('GenericElement', { x: 20, y: 220, w: 200, h: 44 }, { label: 'Card', identifier: 'promo-card', + children: [node('Button', { x: 30, y: 230, w: 60, h: 20 }, { label: 'Close', identifier: 'promo-close' })] }), + ], + }), + ], + }), +}); + +test('selectors treat testid as the accessibility identifier', () => { + const { nodes } = rnScreen(); + const count = (sel) => nodes.filter((n) => matches(n, sel)).length; + assert.equal(count('[testid="login-submit"]'), 1); + assert.equal(count('button[testid^="login-"]'), 1); + assert.equal(count('[testid="nope"]'), 0); +}); + +test('descendant selectors skip ancestors whose exact identifier is absent from the tree', () => { + const { nodes } = rnScreen(); + const count = (sel) => nodes.filter((n) => matches(n, sel)).length; + assert.equal(count('[testid="login-form"] [testid="login-submit"]'), 1, 'absent wrapper is skipped'); + assert.equal(count('[testid="screen"] [testid="login-form"] [testid="login-submit"]'), 1, 'several absent wrappers are skipped'); + assert.equal(count('[testid="promo-card"] [testid="promo-close"]'), 1, 'present ancestors still match'); + assert.equal(count('[testid="promo-card"] [testid="login-submit"]'), 0, 'a present identifier is never skipped'); + assert.equal(count('[testid="login-form"] button'), 0, 'a subject without its own attribute is not widened'); + assert.equal(count('[testid^="login-form"] [testid="login-submit"]'), 0, 'only exact identifiers are skippable'); + assert.equal(count('[testid="login-form"] > [testid="login-submit"]'), 0, 'the child combinator is never skipped'); +}); diff --git a/skills/subtext-ios-sim/test/cli.test.mjs b/skills/subtext-ios-sim/test/cli.test.mjs index 151fded..f61672d 100644 --- a/skills/subtext-ios-sim/test/cli.test.mjs +++ b/skills/subtext-ios-sim/test/cli.test.mjs @@ -6,7 +6,7 @@ import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; import { formatResult, parseArgs, toRequest } from '../lib/cli.mjs'; import { call, readSession } from '../lib/client.mjs'; -import { runDaemon } from '../lib/daemon.mjs'; +import { runDaemon, SimSession } from '../lib/daemon.mjs'; import { HelperClient } from '../lib/helper-client.mjs'; import { ConsoleBuffer, parseLogLine } from '../lib/logs.mjs'; import { pickDevice } from '../lib/simctl.mjs'; @@ -149,6 +149,8 @@ test('daemon serves the protocol over its socket', async () => { await assert.rejects(call(sock, 'scroll', { direction: 'down', until: 'far', max: 2 }), { code: 'not_found' }); await call(sock, 'tapXY', { x: 1, y: 2 }); + await assert.rejects(call(sock, 'tapXY', { x: 230, y: 1450 }), (err) => err.code === 'off_screen_point' && /400x800 pt screen/.test(err.message) && /not pixels/.test(err.message)); + await assert.rejects(call(sock, 'swipe', { x1: 10, y1: 10, x2: 10, y2: 900 }), { code: 'off_screen_point' }); await call(sock, 'key', { key: 'return' }); await call(sock, 'openurl', { url: 'app://x' }); assert.deepEqual(simctl.calls.at(-1), ['openUrl', 'FAKE-UDID', 'app://x']); @@ -177,6 +179,9 @@ test('daemon serves the protocol over its socket', async () => { assert.notEqual(fresh.path, current.path); assert.equal(formatResult('click', { target: '2 button "Go"', x: 1, y: 2 }), '✓ clicked 2 button "Go" at (1, 2)'); + assert.equal(formatResult('restart', { bundleId: 'dev.x', pid: 7, installed: '/b/X.app', ready: true }), '✓ installed /b/X.app and relaunched dev.x (pid 7) in the running session'); + assert.equal(formatResult('restart', { bundleId: 'dev.x', pid: 7, installed: null, ready: true }), '✓ relaunched dev.x (pid 7) in the running session'); + assert.equal(formatResult('relaunch', { bundleId: 'dev.x', pid: 7 }), '✓ relaunched dev.x (pid 7)'); await call(sock, 'stop', { shutdown: true }); for (let i = 0; i < 50 && !stopped; i++) await new Promise((r) => setTimeout(r, 20)); @@ -184,3 +189,42 @@ test('daemon serves the protocol over its socket', async () => { assert.ok(!existsSync(join(sightmapDir, '.sim-session'))); assert.deepEqual(simctl.calls.slice(-2).map((c) => c[0]), ['terminate', 'shutdown']); }); + +function stubSession({ describe, accessibilityEnabled }) { + const helper = { request: async () => describe(), close() {}, alive: true, restarts: 0 }; + const simctl = accessibilityEnabled === undefined ? {} : { accessibilityEnabled: async () => accessibilityEnabled }; + return new SimSession({ udid: 'U-1', bundleId: 'dev.x', processName: 'X', sightmapDir: join(dir, 'stub', '.sightmap'), helper, simctl, timing: { overlaySettleMs: 0 } }); +} + +const axDisabled = () => { throw Object.assign(new Error('no elements'), { code: 'ax_disabled' }); }; + +test('an empty tree lists the likely causes, dropping app accessibility when it is on', async () => { + const off = stubSession({ describe: axDisabled, accessibilityEnabled: false }); + await assert.rejects(off.handle('snapshot', {}), (err) => { + assert.equal(err.code, 'ax_disabled'); + const lines = err.message.split('\n'); + assert.match(lines[1], /^ {2}1\. .*accessibilityViewIsModal.*no-hide-descendants/); + assert.match(lines[2], /^ {2}2\. App accessibility is off.*xcrun simctl spawn U-1 defaults write/); + assert.match(lines[3], /^ {2}3\. A system sheet/); + return true; + }); + const on = stubSession({ describe: axDisabled, accessibilityEnabled: true }); + await assert.rejects(on.handle('snapshot', {}), (err) => { + assert.match(err.message, /App accessibility is enabled on this simulator, so that is not the cause/); + assert.doesNotMatch(err.message, /defaults write/); + assert.equal(err.message.split('\n').length, 3); + return true; + }); +}); + +test('a snapshot showing only a small floating element says the app window is hidden', async () => { + const el = (type, frame, extra = {}) => ({ role: `AX${type}`, type, label: null, identifier: null, value: null, frame, enabled: true, children: [], ...extra }); + const describe = () => ({ + screen: { w: 390, h: 844 }, + root: el('Application', { x: 0, y: 0, w: 390, h: 844 }, { label: 'App', children: [el('Button', { x: 330, y: 700, w: 48, h: 48 }, { label: 'Dev menu' })] }), + }); + const snap = await stubSession({ describe, accessibilityEnabled: true }).handle('snapshot', {}); + assert.match(snap.text, /\[Mostly empty\]\nOnly a small floating element is exposed \(button "Dev menu"\)/); + assert.match(snap.text, /no-hide-descendants/); + assert.doesNotMatch(snap.text, /defaults write/); +}); diff --git a/skills/subtext-ios-sim/test/e2e.sh b/skills/subtext-ios-sim/test/e2e.sh index 7aabe29..a178ece 100755 --- a/skills/subtext-ios-sim/test/e2e.sh +++ b/skills/subtext-ios-sim/test/e2e.sh @@ -50,6 +50,12 @@ took # The sample app's .sightmap/ lives next to it, so run from there. cd "$skill/fixtures/$fixture" + +step "sightmap validate" +sightmap_bin="$(node --input-type=module -e "const { resolveSightmap } = await import('$skill/lib/sightmap-cli.mjs'); const s = resolveSightmap(); console.error(' sightmap ' + s.version + ' (' + s.source + ')'); console.log(s.bin)")" +"$sightmap_bin" validate --sightmap-dir .sightmap || fail "sightmap validate rejected fixtures/$fixture/.sightmap" +took + trap cleanup EXIT step "start --detach" @@ -151,6 +157,16 @@ grep -q '"text":""' "$(ls -t .sightmap/.sim-runs/*.jsonl | head -1)" | step "status" "${sim[@]}" status +step "start on the running session: same app relaunches, another app is refused" +again="$("${sim[@]}" start --app "$app" --detach)" +echo "$again" +expect "installed $app and relaunched" "$again" +if "${sim[@]}" start --bundle-id com.apple.Preferences --detach 2>"$out/switch.txt"; then fail "start with another bundle id was not refused"; fi +cat "$out/switch.txt" +expect 'subtext-sim stop' "$(cat "$out/switch.txt")" +expect 'not pixels' "$("${sim[@]}" tap-xy 230 5000 2>&1 || true)" +took + step "stop" trap - EXIT if [[ "${E2E_KEEP_BOOTED:-}" == 1 ]]; then "${sim[@]}" stop; else "${sim[@]}" stop --shutdown; fi diff --git a/skills/subtext-ios-sim/test/rn.test.mjs b/skills/subtext-ios-sim/test/rn.test.mjs index 85c4fbb..39979ce 100644 --- a/skills/subtext-ios-sim/test/rn.test.mjs +++ b/skills/subtext-ios-sim/test/rn.test.mjs @@ -7,6 +7,7 @@ import { normalize } from '../lib/normalize.mjs'; import { annotate, resolve } from '../lib/sightmap-bridge.mjs'; import { parseCorpus } from '../lib/corpus.mjs'; import { assertActionable } from '../lib/daemon.mjs'; +import { ConsoleBuffer, parseLogLine, sessionUrlIn } from '../lib/logs.mjs'; const here = dirname(fileURLToPath(import.meta.url)); const describeRN = () => JSON.parse(readFileSync(join(here, 'fixtures', 'describe-rn.json'), 'utf8')); @@ -71,3 +72,37 @@ test('a LogBox toast is reported with its ✕ and refuses taps underneath', () = assert.throws(() => assertActionable(snap, byIdentifier(snap, 'product-row-15'), 'product-row-15'), (err) => err.code === 'covered' && /LogBox/.test(err.message) && /tap-xy 368 812/.test(err.hint)); }); + +const URL_NA = 'https://app.fullstory.com/ui/o-TEST01-na1/session/5d3f1a2b-0c4e-4f6a-9b8d-7e6f5a4b3c2d%3A1234567890123456789'; +const URL_EU = 'https://app.eu1.fullstory.com/ui/o-TEST02-eu1/client-session/6105678901234567890%3A5432109876543210987'; + +test('session URLs are found anywhere in React Native console lines', () => { + const cases = [ + [`Fullstory session URL: ${URL_NA}`, URL_NA], + [`'[fullstory]', 'session ready', '${URL_NA}'`, URL_NA], + [`{ sessionUrl: '${URL_EU}', isNew: true }`, URL_EU], + [`{"sessionUrl":"${URL_EU}"}`, URL_EU], + [`onReady (${URL_NA}).`, URL_NA], + [`FULLSTORY_SESSION_URL ${URL_EU}`, URL_EU], + ['FULLSTORY_SESSION_URL https://app.example/s/1', 'https://app.example/s/1'], + ['Loaded https://app.fullstory.com/ui/o-TEST01-na1/home', null], + ['nothing here', null], + ]; + for (const [message, expected] of cases) assert.equal(sessionUrlIn(message), expected, message); +}); + +test('the newest session URL wins, from any console source or level', () => { + const buf = new ConsoleBuffer(); + const seen = []; + buf.onSessionUrl = (url) => seen.push(url); + const rnLine = JSON.stringify({ + eventType: 'logEvent', messageType: 'Info', subsystem: 'com.facebook.react.log', category: 'javascript', + eventMessage: `'FullStory.onReady', { replayStartUrl: '${URL_NA}', replayNowUrl: '${URL_NA}?playerState=x' }`, timestamp: 't', + }); + buf.push(parseLogLine(rnLine)); + assert.equal(buf.sessionUrl, URL_NA); + buf.push({ level: 'stderr', source: 'stderr', message: `relaunch: ${URL_EU}` }); + buf.push({ level: 'debug', source: 'os_log', message: 'unrelated' }); + assert.equal(buf.sessionUrl, URL_EU); + assert.deepEqual(seen, [URL_NA, URL_EU]); +}); diff --git a/skills/subtext-ios-sim/test/sightmap.test.mjs b/skills/subtext-ios-sim/test/sightmap.test.mjs new file mode 100644 index 0000000..2d424f7 --- /dev/null +++ b/skills/subtext-ios-sim/test/sightmap.test.mjs @@ -0,0 +1,52 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { compareVersions, SIGHTMAP_EXTRACT_OBJECT, sightmapVersion, sightmapVersionWarning } from '../lib/sightmap-cli.mjs'; + +const skill = join(dirname(fileURLToPath(import.meta.url)), '..'); +const bin = process.env.SIGHTMAP_BIN || 'sightmap'; +const version = sightmapVersion(bin); +const canValidate = version && version !== 'unknown' && compareVersions(version, SIGHTMAP_EXTRACT_OBJECT) >= 0 + ? {} + : { skip: `needs a sightmap ${SIGHTMAP_EXTRACT_OBJECT}+ CLI (set SIGHTMAP_BIN); found ${version ?? 'none'}` }; + +function validate(dir) { + const r = spawnSync(bin, ['validate', '--sightmap-dir', dir], { encoding: 'utf8' }); + return { ok: r.status === 0, out: `${r.stdout}${r.stderr}`.trim() }; +} + +test('sightmap versions outside the tested range warn', () => { + assert.equal(sightmapVersionWarning('0.36.1'), null); + assert.equal(sightmapVersionWarning('0.34.0'), null); + assert.match(sightmapVersionWarning('0.33.0'), /extract: \{ from/); + assert.match(sightmapVersionWarning('0.31.2'), /outside the range/); + assert.match(sightmapVersionWarning('0.37.0'), /outside the range/); + assert.match(sightmapVersionWarning('unknown'), /could not read/); +}); + +for (const app of ['SampleApp', 'RNSampleApp']) { + test(`fixtures/${app}/.sightmap passes sightmap validate`, canValidate, () => { + const { ok, out } = validate(join(skill, 'fixtures', app, '.sightmap')); + assert.ok(ok, out); + assert.doesNotMatch(out, /deprecated/); + }); +} + +test('every YAML example in SKILL.md and README.md passes sightmap validate', canValidate, () => { + const blocks = ['SKILL.md', 'README.md'].flatMap((doc) => + [...readFileSync(join(skill, doc), 'utf8').matchAll(/```ya?ml\n([\s\S]*?)```/g)].map((m, i) => ({ doc, i, yaml: m[1] }))); + assert.ok(blocks.length > 0); + assert.ok(blocks.some((b) => /properties:/.test(b.yaml)), 'at least one example declares a property'); + for (const { doc, i, yaml } of blocks) { + const dir = mkdtempSync(join(tmpdir(), 'subtext-sim-yaml-')); + mkdirSync(join(dir, 'views')); + writeFileSync(join(dir, 'views', 'example.yaml'), yaml); + const { ok, out } = validate(dir); + assert.ok(ok, `${doc} YAML example #${i + 1}:\n${out}`); + assert.doesNotMatch(out, /deprecated/, `${doc} YAML example #${i + 1}`); + } +}); From 7fa7aefef1c3dd79deeac0ca3cc6c49c22286fb9 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Wed, 28 Oct 2026 17:08:15 -0400 Subject: [PATCH 09/12] docs: add subtext-ios-sim field notes from dev and field testing Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- docs/ios-sim-loop/FIELD-NOTES.md | 56 ++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 docs/ios-sim-loop/FIELD-NOTES.md diff --git a/docs/ios-sim-loop/FIELD-NOTES.md b/docs/ios-sim-loop/FIELD-NOTES.md new file mode 100644 index 0000000..a93f8c1 --- /dev/null +++ b/docs/ios-sim-loop/FIELD-NOTES.md @@ -0,0 +1,56 @@ +# subtext-ios-sim field notes + +A running record of what came up while building and testing the iOS Simulator loop: on the development Mac ("dev Mac") and in field tests on a second Mac against a real React Native app ("field Mac"). Newest entries at the top of each section. Add to it from either machine; keep it free of org IDs, credentials and internal-only details. + +Status values: **fixed** (commit noted), **in progress**, **open**, **upstream** (outside this skill), **won't fix**. + +## Skill, CLI and helper + +| Found | Where | Issue | Status | +|---|---|---|---| +| 2026-10-28 | field Mac | Empty accessibility tree while app accessibility was already on. The cause was an always-mounted view with `accessibilityViewIsModal = YES` hiding its siblings, but the error said to enable accessibility. | fixed 61a37a8: errors list modal/hidden-root views first and skip the accessibility advice when it's on; a `doctor --app` probe is still open | +| 2026-10-28 | field Mac | `snapshot` crashes with `extract.startsWith is not a function`: sightmap 0.36 emits `extract` as an object, and the bridge only handled strings. | fixed 61a37a8: both forms accepted; npx pin is now sightmap 0.36.1; `doctor` reports the resolved version | +| 2026-10-28 | field Mac | `session-url` missed the URL when React Native logged it as a multi-argument line. | fixed 61a37a8: any Fullstory session URL in any console entry, newest wins | +| 2026-10-28 | field Mac | An existing corpus matched 0%: session corpora use `[testid=…]` while the simulator exposes `identifier`, and wrapper views aren't accessibility elements, so parent chains never match. A draft simulator corpus reached 56–100% per screen. | fixed 61a37a8: `testid` aliases `identifier`; descendant chains skip ancestors whose identifier nothing carries. sightmap's own `explain`/`coverage` don't apply that rule, so they can report fewer matches | +| 2026-10-28 | field Mac | Skill YAML examples fail `sightmap validate` on 0.36 (`extract: text` is deprecated; native views have no route). | fixed 61a37a8: examples and fixtures pass 0.36 `validate`, which now runs in tests and the e2e. Views need a `route`, so native views use placeholder routes like `/login` | +| 2026-10-28 | field Mac | `start --bundle-id` on a running session relaunched the previous app while printing "relaunched". | fixed 61a37a8: errors on a different app; reports what it installed and relaunched | +| 2026-10-28 | field Mac | `tap-xy` accepted points far off the screen; units (points vs pixels) weren't stated. | fixed 61a37a8: off-screen points refused; units are points | +| 2026-10-28 | field Mac | `--sightmap-dir` ties the corpus to the session, so a draft corpus can't be tried against a running session. | open: `--corpus DIR` idea | +| 2026-10-28 | field Mac | The dev-menu floating button is its own window, and it's the only visible element when the tree is empty. | fixed 61a37a8: snapshot adds a `[Mostly empty]` note (size heuristic, untested against a real dev-menu tree) | +| 2026-10-28 | field Mac | Producer and consumer ran one after the other: the test prompt told the consumer to wait for the producer, and the skill allowed either. | fixed e9fb6b3: the skill requires concurrent work and explains SDK delivery | +| 2026-10-28 | both | macOS keychain prompt on the first helper build: SwiftPM offered a saved `github.com` credential while downloading idb's public frameworks. | fixed 4410044: `--disable-keychain --disable-netrc` | +| 2026-10-28 | dev Mac | The SwiftUI fixture was assembled by hand with `swiftc`. It lacked the simulated entitlements Xcode embeds, so the Fullstory SDK's background uploads all failed and no session reached Fullstory. Looked like a server or org problem until device logs showed every `/rec/bundle` upload failing inside the simulator. | fixed c38b875: committed `SimSample.xcodeproj`, built with `xcodebuild` | +| 2026-10-28 | dev Mac | iOS shows a "Save Password?" sheet after sign-in once the app has a real identity (Xcode build). It briefly swallowed taps. | documented; a fresh `snapshot` clears it | +| 2026-10-28 | dev Mac | Mobile network capture is off by default and can't be set by API or the Subtext MCP; it's the main onboarding roadblock for network data. | fixed 0227741: `doctor` prints it as a manual check | +| 2026-10-28 | dev Mac (clean-machine run) | The `owner/repo#ref` marketplace shorthand cloned over SSH, which fails without GitHub SSH keys. | fixed 19e7fb5: README leads with the HTTPS URL | +| 2026-10-28 | dev Mac (clean-machine run) | The fixture build ignored `SUBTEXT_SIM_CACHE`; the helper build used the global SwiftPM cache. | fixed 19e7fb5 | +| 2026-10-28 | dev Mac (clean-machine run) | `console list` was swamped by Apple framework errors. | fixed 19e7fb5: hidden by default, `--all` shows them | +| 2026-10-28 | dev Mac | Enabling app accessibility only takes effect at boot, so the first `start` on a fresh simulator reboots it once. | documented | +| 2026-10-28 | dev Mac | iOS 26 SwiftUI tab bars appear as one "Tab Bar" element with no tabs; switching tabs needs `tap-xy`. | open | +| 2026-10-28 | dev Mac | Scrolling a long list to a far row takes ~3–4 s per screenful. | open | + +## Fullstory data and the Subtext MCP + +| Found | Where | Issue | Status | +|---|---|---|---| +| 2026-10-28 | field Mac | Every network request appears twice at the same millisecond in the session; the server's own logs show one. | upstream: capture or ingest | +| 2026-10-28 | field Mac | `review-zoom` at `detail` returned response headers only: no request headers, bodies or query strings. Check against the org's header allowlists and body rules before calling it a bug. | upstream / open | +| 2026-10-28 | field Mac | `review-zoom` at `standard` collapses network traffic into counts; endpoint names only show at `machine`. | fixed 61a37a8: skill recommends `machine` for naming endpoints | +| 2026-10-28 | field Mac | Network data lagged 35 s to 2 min behind the app, longer than the ~20 s upload interval. | upstream / open | +| 2026-10-28 | dev Mac | `review-open` reports "invalid session_url" when a session's IDs don't exist server-side, which reads like a bad link rather than "this session never arrived". | upstream: clearer error | + +## Setup friction + +| Found | Where | Issue | Status | +|---|---|---|---| +| 2026-10-28 | field Mac | `pod install` crashed without a UTF-8 locale. | fixed 61a37a8: README prerequisite (`export LANG=en_US.UTF-8`) | +| 2026-10-28 | field Mac | `subtext-verify` installs disabled from the marketplace. | open | +| 2026-10-28 | field Mac | Stale Metro modules in git worktrees made edits appear not to apply; `--clear` fixed it. | fixed 61a37a8: skill note | +| 2026-10-28 | field Mac | Upgrading an existing install from the branch: `plugin update` says "already at latest", because the version doesn't change. | documented: `marketplace update`, then uninstall and install | + +## What worked (field Mac, React Native) + +- Every `testID` surfaced as `identifier`; `snapshot`, `click`, `fill` and `scroll --until` each took under a second. +- A dev client with Metro worked as well as Release for the live loop, and was cheaper. +- Recorded taps lined up with producer steps to within about 30 ms once the clock offset was known. +- Network capture had zero errors across about 900 events, and every `/api/*` route got a `requests:` name. From 4bfa88eb76c6e0195d71e78b2a44096c6ef59a64 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Thu, 29 Oct 2026 08:46:20 -0400 Subject: [PATCH 10/12] docs: field notes from the subtext-ios-sim retest Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- docs/ios-sim-loop/FIELD-NOTES.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/ios-sim-loop/FIELD-NOTES.md b/docs/ios-sim-loop/FIELD-NOTES.md index a93f8c1..16f8d96 100644 --- a/docs/ios-sim-loop/FIELD-NOTES.md +++ b/docs/ios-sim-loop/FIELD-NOTES.md @@ -8,6 +8,14 @@ Status values: **fixed** (commit noted), **in progress**, **open**, **upstream** | Found | Where | Issue | Status | |---|---|---|---| +| 2026-10-28 | field Mac (retest, 7fa7aef) | The Fullstory session ended mid-run (SDK: `Received HTTP Status Code: 206, terminating FullStory session`), and nothing in subtext-sim noticed: `session-url` and `status` kept returning the dead URL. | open: detect session end, log `session-ended`, report it from `session-url`/`status`, make relaunch the producer's standard response | +| 2026-10-28 | field Mac (retest) | The selector skip rule merged three components onto the same `[testid*="-option-"]` match, and the wrong one won. | open: don't skip into substring-only selectors, or flag skip-induced conflicts | +| 2026-10-28 | field Mac (retest) | `click MobileTopBarAvatar` reported success but opened the Expo dev menu: the dev-menu window's touch area is larger than its accessibility frame. | open: treat dev-menu windows as covering; document hiding the floating button | +| 2026-10-28 | field Mac (retest) | The empty-tree diagnosis was only fixed in `snapshot`; `click`, `fill` and `scroll` still give the "long lists" advice. | open | +| 2026-10-28 | field Mac (retest) | Coverage misses labelled, tappable `other` elements (e.g. tournament cards) and counts keyboard keys while the keyboard is up. | open | +| 2026-10-28 | field Mac (retest) | `console list --grep -i 206` took `-i` as the pattern; `fill "" --clear` reported the placeholder as the value; clicks right after a sheet closes or a back navigation fail until retried. | open | +| 2026-10-28 | field Mac (retest) | `doctor` used sightmap 0.36.0 from PATH rather than the pinned 0.36.1. | open: decide whether PATH should win | +| 2026-10-28 | field Mac (retest) | Metro in a nested git worktree ignores edits: Watchman roots at the parent checkout, and `--clear` doesn't help; restarting Metro does. | open: skill note | | 2026-10-28 | field Mac | Empty accessibility tree while app accessibility was already on. The cause was an always-mounted view with `accessibilityViewIsModal = YES` hiding its siblings, but the error said to enable accessibility. | fixed 61a37a8: errors list modal/hidden-root views first and skip the accessibility advice when it's on; a `doctor --app` probe is still open | | 2026-10-28 | field Mac | `snapshot` crashes with `extract.startsWith is not a function`: sightmap 0.36 emits `extract` as an object, and the bridge only handled strings. | fixed 61a37a8: both forms accepted; npx pin is now sightmap 0.36.1; `doctor` reports the resolved version | | 2026-10-28 | field Mac | `session-url` missed the URL when React Native logged it as a multi-argument line. | fixed 61a37a8: any Fullstory session URL in any console entry, newest wins | @@ -33,6 +41,8 @@ Status values: **fixed** (commit noted), **in progress**, **open**, **upstream** | Found | Where | Issue | Status | |---|---|---|---| +| 2026-10-28 | field Mac (retest) | Sessions ended with HTTP 206, which the ingest server returns for a bad request it won't accept, mainly a bundle that's too large; the SDK then terminates the session. The SDK also logged "Maximum number of views reached". Only the first 26 s of one session arrived, and none of the run's own requests were ingested. Sessions lived 51 s to 5.5 min. | upstream / app: bundle size on large React Native trees | +| 2026-10-28 | field Mac (retest) | Duplicate network events are unchanged (65 unique requests appeared as 130 events), and `detail` still shows response headers only. | upstream | | 2026-10-28 | field Mac | Every network request appears twice at the same millisecond in the session; the server's own logs show one. | upstream: capture or ingest | | 2026-10-28 | field Mac | `review-zoom` at `detail` returned response headers only: no request headers, bodies or query strings. Check against the org's header allowlists and body rules before calling it a bug. | upstream / open | | 2026-10-28 | field Mac | `review-zoom` at `standard` collapses network traffic into counts; endpoint names only show at `machine`. | fixed 61a37a8: skill recommends `machine` for naming endpoints | @@ -54,3 +64,9 @@ Status values: **fixed** (commit noted), **in progress**, **open**, **upstream** - A dev client with Metro worked as well as Release for the live loop, and was cheaper. - Recorded taps lined up with producer steps to within about 30 ms once the clock offset was known. - Network capture had zero errors across about 900 events, and every `/api/*` route got a `requests:` name. + +## What worked (field Mac retest, 7fa7aef) + +- 8 of 9 re-checks pass: properties no longer crash, `session-url` works, the existing corpus matches every screen (it matched 0% before), `validate` passes, bundle-id switching, off-screen taps and `machine` grain all work. The empty-tree fix is partial. +- The producer and consumer ran concurrently. The consumer started 25 s after the producer and sent four coverage requests; the producer acted on two (45 s and 2m42s later). One wasn't acted on because the producer's brief limited relaunches. +- Timings: `doctor` 9.2 s, `start` 23.8 s, `snapshot` 0.26 s average, `click` 0.32 s average, `relaunch` to a new session URL 10 s. From 701aa25fad9f35cd961f9b9d0f2d919d4819cd20 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Thu, 29 Oct 2026 09:04:11 -0400 Subject: [PATCH 11/12] subtext-ios-sim: fixes from React Native retest - Detect Fullstory session end (SDK 206/202 termination lines): log a session-ended run-log entry, make session-url/status report it, and have the producer relaunch for a new session - Selector skip rule needs an exact leaf and prefers full-chain matches; remaining skip-induced conflicts are labelled - click/fill refuse taps inside a floating dev-menu button's cover zone - snapshot, click, fill and scroll share one empty-tree diagnosis - Coverage counts labelled tappable `other` elements and ignores keyboard keys - CLI: -i/--ignore-case for console grep, no flag-looking values, placeholder vs value after fill --clear, bounded settle-and-retry in click/fill, clearer doctor sightmap wording - Docs: Metro/Watchman in nested worktrees, what "invalid session_url" usually means, --corpus not yet available Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- skills/subtext-ios-sim/README.md | 4 +- skills/subtext-ios-sim/SKILL.md | 36 +- skills/subtext-ios-sim/lib/BRIDGE.md | 12 +- skills/subtext-ios-sim/lib/cli.mjs | 49 +- skills/subtext-ios-sim/lib/daemon.mjs | 143 +++++- skills/subtext-ios-sim/lib/doctor.mjs | 11 +- skills/subtext-ios-sim/lib/logs.mjs | 48 +- skills/subtext-ios-sim/lib/normalize.mjs | 39 +- skills/subtext-ios-sim/lib/selector.mjs | 57 ++- .../subtext-ios-sim/lib/sightmap-bridge.mjs | 76 +++- skills/subtext-ios-sim/test/bridge.test.mjs | 28 ++ skills/subtext-ios-sim/test/cli.test.mjs | 114 ++++- .../test/fakes/fake-helper.mjs | 11 +- .../test/fixtures/describe-options.json | 118 +++++ .../test/fixtures/describe-rn-keyboard.json | 422 ++++++++++++++++++ .../test/fixtures/skip-conflict-corpus.json | 48 ++ skills/subtext-ios-sim/test/rn.test.mjs | 68 +++ skills/subtext-ios-sim/test/sightmap.test.mjs | 10 +- 18 files changed, 1182 insertions(+), 112 deletions(-) create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-options.json create mode 100644 skills/subtext-ios-sim/test/fixtures/describe-rn-keyboard.json create mode 100644 skills/subtext-ios-sim/test/fixtures/skip-conflict-corpus.json diff --git a/skills/subtext-ios-sim/README.md b/skills/subtext-ios-sim/README.md index 610d35f..d5f0f50 100644 --- a/skills/subtext-ios-sim/README.md +++ b/skills/subtext-ios-sim/README.md @@ -8,7 +8,7 @@ Drive and inspect an iOS app in the local iOS Simulator from a coding agent. The - Xcode 26 or later, opened once, with an iOS Simulator runtime (Xcode > Settings > Components) - `xcode-select -p` pointing at Xcode, not the Command Line Tools - Node 18 or later -- The sightmap CLI: nothing to install, `subtext-sim` runs the pinned `@sightmap/sightmap@0.36.1` through `npx`. A `sightmap` on your PATH (or `SIGHTMAP_BIN`) wins over the pin; `doctor` reports which one resolves and warns outside the tested range (0.32–0.36; the sample apps' `.sightmap/` need 0.34+). +- The sightmap CLI: nothing to install, `subtext-sim` runs the pinned `@sightmap/sightmap@0.36.1` through `npx`. A `sightmap` on your PATH (or `SIGHTMAP_BIN`) wins over the pin, so you can pin your own; `doctor` says which one is used ("using PATH sightmap 0.36.0 (…; pinned fallback is 0.36.1)") and warns only outside the tested range (0.32–0.36; the sample apps' `.sightmap/` need 0.34+). ## Install from the branch (Claude Code) @@ -104,7 +104,7 @@ This downloads the public Fullstory iOS SDK (the binary from [`fullstorydev/full Requirements on top of Xcode: Node 22.11+, npm, CocoaPods, and network access the first time. `pod install` needs a UTF-8 locale; `build.sh` sets one, but for your own app run `export LANG=en_US.UTF-8` first. Only the JS sources are checked in. `build.sh` generates the native iOS project from the React Native template for the pinned version, then runs `npm install`, `pod install` and `xcodebuild` in `~/Library/Caches/subtext-sim/RNSampleApp/` (set `OUT_DIR` to change that). It prints the `.app` path on stdout. The first build takes several minutes; after a JS change, a rebuild takes about 15 s. - `./build.sh` builds Release with the JS bundle embedded, so `subtext-sim start --app` works without Metro. `ios/SceneDelegate.swift` replaces the template's to keep `console.log` in Release builds (`RCTSetLogThreshold`). -- `CONFIGURATION=Debug ./build.sh` builds for Metro. Start Metro from the generated project (the build prints the command) before launching. Debug with Metro works for the whole loop and is the cheaper choice while iterating on JS; if edits don't show up, restart Metro with `--clear`. +- `CONFIGURATION=Debug ./build.sh` builds for Metro. Start Metro from the generated project (the build prints the command) before launching. Debug with Metro works for the whole loop and is the cheaper choice while iterating on JS; if edits don't show up, restart Metro with `--clear`. In a git worktree nested inside another checkout, Watchman roots at the parent checkout and Metro ignores the worktree's edits, which `--clear` doesn't fix: stop Metro, run `watchman watch-del-all`, add an empty `.watchmanconfig` at the worktree root, and start Metro from the worktree. - `FS_ORG_ID= ./build.sh` adds `@fullstory/react-native` (its Babel plugins, the `FullStory` pod and a `FullStory` `Info.plist` dictionary) and logs `FULLSTORY_SESSION_URL ` from `FullStory.onReady()`. `FS_RN_VERSION` and `FS_SDK_VERSION` pick the plugin and iOS SDK versions. `build.sh` uses target builds rather than `xcodebuild -scheme`, and drops the template's launch storyboard and asset catalog. Both need Xcode's iOS platform component for the SDK version, which a machine with only a Simulator runtime installed doesn't have. diff --git a/skills/subtext-ios-sim/SKILL.md b/skills/subtext-ios-sim/SKILL.md index 2f7cf77..08b433a 100644 --- a/skills/subtext-ios-sim/SKILL.md +++ b/skills/subtext-ios-sim/SKILL.md @@ -44,30 +44,30 @@ subtext-sim stop # --shutdown also shuts the simul **Targets** are a snapshot id (`12`), an exact accessibility identifier (`login-submit`), or a component query: `SignInButton`, `ProductRow[title^="Product 15"]`, `LoginForm SignInButton`, or a role with attributes for unmapped apps, `button[identifier="login-submit"]`. A query matching several elements errors and lists them; add a predicate or append `#N`. -**Off-screen elements:** `click` refuses targets whose center is off the screen or under the tab bar, toolbar or keyboard, and says what to do. Long SwiftUI/UIKit lists only expose the rows near the screen, so a row further down is "not found" until you scroll. `scroll down --until ` scrolls (slowly enough not to fling past it) until the target is tappable, then refreshes the snapshot ids. +**Off-screen elements:** `click` refuses targets whose center is off the screen, under the tab bar, toolbar or keyboard, or within 24 pt of a floating dev-menu button (see React Native apps), and says what to do. Long SwiftUI/UIKit lists only expose the rows near the screen, so a row further down is "not found" until you scroll. `scroll down --until ` scrolls (slowly enough not to fling past it) until the target is tappable, then refreshes the snapshot ids. -**Waiting:** actions return as soon as the tap lands; navigation animates for a moment after. If a snapshot still shows the old screen, snapshot again. +**Waiting:** actions return as soon as the tap lands; navigation animates for a moment after. If a snapshot still shows the old screen, snapshot again. `click` and `fill` wait out short animations themselves: when the target is missing, or the screen moves between two reads ~150 ms apart (a sheet closing, a back navigation), they re-read for up to ~1.5 s before tapping or failing. `--json` reports how many re-reads it took as `retries`. ## Commands | Command | What it does | |---|---| -| `doctor [--fix]` | Checks Xcode, simulators, Swift, Node, the sightmap CLI (the version that resolves: `SIGHTMAP_BIN`, then PATH, then the pinned npx package; warns outside the tested 0.32–0.36 range); builds the helper. | +| `doctor [--fix]` | Checks Xcode, simulators, Swift, Node, the sightmap CLI, and builds the helper. For sightmap it says which one is used, e.g. "using PATH sightmap 0.36.0 (…; pinned fallback is 0.36.1)": `SIGHTMAP_BIN`, then PATH (so you can pin your own), then the pinned npx package. It warns only outside the tested 0.32–0.36 range (or below 0.34 for `extract` objects). | | `start [--device NAME\|UDID] [--app PATH] [--bundle-id ID] [--label L] [--detach]` | Boots, enables app accessibility, installs, launches, starts the daemon and a new run log. On a running session: reinstalls and relaunches the same app; errors for a different one. | | `status` / `stop [--shutdown]` | Session health / stop the daemon and the app (and the simulator). | | `launch` / `relaunch` | Foreground the app / restart it (re-enables accessibility first). | | `snapshot [--view V] [--include-offscreen] [--out F] [--tree-out F] [--json]` | Annotated tree. `--out` writes the text plus a `.tree.json` that `sightmap coverage` / `explain --snap` read. | -| `click ` | Re-resolves, checks it's on screen, uncovered and enabled, taps its center. | -| `fill [--clear]` | Taps, types (appends unless `--clear`), and errors if the field doesn't show the text. | +| `click ` | Re-resolves, checks it's on screen, uncovered and enabled, taps its center. Re-reads for up to ~1.5 s while the target is missing or the screen is moving. | +| `fill [--clear]` | Taps, types (appends unless `--clear`), and errors if the field doesn't show the text. `fill "" --clear` empties a field; a field that then shows only its placeholder reports value `""` (and the placeholder separately). | | `scroll [--in ] [--until ] [--max N]` | One page, or until a target is tappable. `--in` scrolls inside that element. | | `swipe x1 y1 x2 y2 [--duration S]`, `tap-xy x y`, `type `, `key `, `button `, `openurl ` | Raw input in screen points (not pixels; off-screen points are refused). Keys: `return delete tab escape space` and arrows. Buttons: `home lock siri side`. | | `screenshot --out F.png` | PNG of the screen. | -| `console list [--limit N] [--grep S] [--level L] [--all]` / `console get ` | The app's own os_log/Logger output and its stdout/stderr (levels `stdout`, `stderr`). `--all` adds Apple framework errors and faults, which are hidden by default as noise. | -| `session-url` | The Fullstory session URL from the app's console: any `https://app…fullstory.com/ui//session/…` (or `client-session`) URL in any line, source or level; the newest wins. | +| `console list [--limit N] [--grep S] [-i] [--level L] [--all]` / `console get ` | The app's own os_log/Logger output and its stdout/stderr (levels `stdout`, `stderr`). `--grep` is case-insensitive (`-i` / `--ignore-case` is accepted). `--all` adds Apple framework errors and faults, which are hidden by default as noise. | +| `session-url` | The Fullstory session URL from the app's console: any `https://app…fullstory.com/ui//session/…` (or `client-session`) URL in any line, source or level; the newest wins. Errors once the SDK has ended that session (with `--json`: `ended: true` and the `reason`, exit code 1); `relaunch` for a new one. | | `run new [--label L]` / `run current` | Start a new run log / print the current run's id and path. | | `run tail [--follow] [--since SEQ] [--run ID]` | Print `.sightmap/.sim-runs/.jsonl` entries; `--follow` keeps streaming. | -Every command takes `--json`. Errors are one line with a `Next:` step. +Every command takes `--json`. Errors are one line with a `Next:` step. A flag value that starts with `-` goes after `=` or `--`: `--grep=-i`, `--grep -- -i`. ## Network, and the producer / consumer collaboration @@ -84,6 +84,10 @@ Network and Fullstory-captured console data come from the app's own Fullstory SD - the Subtext MCP authenticated to the same org. - **Console:** only `FS.log` reaches Fullstory. `print` / `os_log` output is local only; read it live with `subtext-sim console list`. - **Session URL:** `subtext-sim session-url` returns it, and the run log records it as a `session` entry. It's found in any console line that contains one (a React Native `console.log('session', url)` works, as does a `FULLSTORY_SESSION_URL ` line). A relaunch can start a new session; when the run log gets a new `session` entry, follow the newest one. +- **Session end:** the SDK can end a session for good. The daemon watches the console for the SDK's termination lines (`Received HTTP Status Code: 206, terminating FullStory session`, the 202 variant, `capture is terminated; not resuming session`), appends `{"action": "session-ended", "url", "reason"}` to the run log, and from then on `session-url` errors and `status` shows the session as ended. Nothing after that point is recorded until a `relaunch` starts a new session, which gets its own `session` entry. + - **206** means the server rejected a bundle and told the SDK not to retry, mostly because it was too large (e.g. a very large view tree; a preceding `Maximum number of views reached` warning is included in the reason). It's an app / SDK issue to report, not something to retry around: relaunching gets a new session, but the same screens will likely end it again. + - **202** means recording is blocked for the app or org. +- **`review-open` says "invalid session_url"?** The link is usually fine: it means the session never reached Fullstory, because it ended before its first bundle arrived or never uploaded (see the build requirements above). Check `status` / the run log for `session-ended`, then `relaunch`. - **Recording isn't reading.** Fullstory records views from inside the app; `subtext-sim` reads the accessibility tree from outside it. A healthy replay doesn't mean `snapshot` can see the screen, and an empty snapshot doesn't mean recording broke. ### Two agents, working at the same time @@ -98,12 +102,13 @@ Because data lags the app by up to ~30 s, one agent that taps, then waits, then **Producer** drives the app and never waits on network data: - It works through flows with `snapshot` / `click` / `fill` / `scroll` / `openurl`. Every action lands in the run log `.sightmap/.sim-runs/.jsonl` with its timestamp, target component and view. +- As soon as a `session-ended` entry appears in the run log (or `session-url` errors), it runs `subtext-sim relaunch` and carries on from the app's start screen, re-doing the step that was in flight. It doesn't wait for a crash. If sessions keep ending with a 206 on the same screen, it stops relaunching there and reports the screen. - It owns `components:` and `views:` in `.sightmap/`: elements it can't target cleanly become new components. - Between flows, it reads `.sightmap/.sim-runs/.consumer.md` for the consumer's requests (e.g. "re-run checkout; a POST /orders fired with no step that explains it") and works them into its next flows. - When finished, it reports to the orchestrator: the run id, the flows it covered, and any consumer requests it couldn't do. It leaves the app running, so the last bundle uploads and the consumer can still re-check a screen. **Consumer** reads captured data while the producer is still working: -1. Follow the producer with `subtext-sim run tail --follow` (or poll with `--since SEQ`). +1. Follow the producer with `subtext-sim run tail --follow` (or poll with `--since SEQ`). After a `session-ended` entry, switch to the newest `session` entry's URL for further reads, and note the gap: producer steps between `session-ended` and the next `session` were never recorded, so don't expect their requests. 2. About every 30 s while new producer steps are appearing, read the newest data: `review-open` the session URL, then `review-zoom` with `t0_ms` set past what it has already processed, and `resolution={"network": "machine", "console": "standard"}`. Close each client with `review-close` before opening the next. - **Network grain:** endpoint names (method + URL) show at `machine`. `standard` collapses traffic into counts, and `detail` adds response headers only, unless the org's capture rules record more. Use `{"network": "machine"}` to name endpoints, and `detail` only on the specific requests whose headers matter. 3. Match each request and `FS.log` message to the producer step just before it (by timestamp), then author: @@ -150,7 +155,8 @@ Write `extract` as an object (`{ from: dom.text }`, `{ from: dom.attr, path: NAM **Reusing a web or React Native session corpus.** Two rules make selectors written against a recorded session match in the simulator: - `[testid="x"]` matches the element whose accessibility identifier is `x`. -- In a descendant selector (`A B`), an ancestor step that names an exact identifier (`[testid="x"]`, `[identifier="x"]`, `#x`) is skipped when **no element in the tree** carries it, as long as the last step has an attribute of its own. A `testID` on a plain wrapper view never reaches the iOS tree, so `[testid="login-form"] [testid="login-submit"]` still matches the button. An identifier that is on screen is never skipped, `>` steps are never skipped, and `[testid="login-form"] button` doesn't match (it would match every button). +- In a descendant selector (`A B`), an ancestor step that names an exact identifier (`[testid="x"]`, `[identifier="x"]`, `#x`) is skipped when **no element in the tree** carries it, as long as the last step matches an attribute exactly (`=`, or `#id`). A `testID` on a plain wrapper view never reaches the iOS tree, so `[testid="login-form"] [testid="login-submit"]` still matches the button. An identifier that is on screen is never skipped, `>` steps are never skipped, and neither `[testid="login-form"] button` nor a prefix/substring leaf like `[testid="menu"] [testid*="-option-"]` is reached by skipping (they would match every button or option on screen). +- When several components match one element, one whose whole chain matched beats one that needed an ancestor skipped. Conflicts that remain between skipped matches are flagged `⚠ … (ancestor skipped)` in the snapshot: give the ancestor an accessible `testID`, or make the leaf selector exact. Prefer `[identifier=…]`; fall back to `[text=…]` only for copy that won't change or be localized. Snapshot after editing to confirm the components resolve. @@ -162,18 +168,21 @@ The loop is the same; RN maps onto the iOS accessibility tree like this: - **`testID` is the identifier.** Target it directly (`click login-submit`) and select on it (`[identifier="login-submit"]` or `[testid="login-submit"]`). - **Put `testID`s on accessible elements.** Only accessible elements reach the tree: a `Pressable`, a `TextInput`, a `View` with `accessible`. A `testID` on a plain wrapper `View` is invisible to the simulator (selectors that use it as an ancestor still work, see above). -- **Role-less `Pressable`s show up as `other`**, not `button`; only `accessibilityRole="button"` makes a `button`. Write role-free selectors (`'[identifier^="product-row-"]'`), not `button[identifier=…]`. Accessible elements with a `testID` count as interactive. +- **Role-less `Pressable`s show up as `other`**, not `button`; only `accessibilityRole="button"` makes a `button`. Write role-free selectors (`'[identifier^="product-row-"]'`), not `button[identifier=…]`. Coverage counts an `other` as interactive when it has a `testID`, or when it has a label or `testID` **and** a tappable trait (`Button`, `Link`, `Adjustable`, e.g. from an `accessibilityRole`). A Pressable with neither a `testID` nor a role has nothing that says it's tappable, so it isn't counted. +- **The system keyboard isn't part of the app:** its keys (and anything else in the keyboard) are left out of coverage and never match components. - **`` doesn't reach the tree** on the New Architecture: the text shows up, its identifier doesn't. Put the `testID` on an `accessible` `View` around the `Text`, or select on `[text=…]`. - **React Navigation tabs** become `tab "Products"` (the "tab, 1 of 3" suffix is stripped), so `tab` selectors and `TabBarItem[title="Network"]` work. Rows behind its JS tab bar are refused as covered, like a native one. - **Labels join child texts with ", "**: a row with "Product 1" and "$6.37" reads `"Product 1, $6.37"`. -**Debug with Metro is fine.** A Debug build (or an Expo dev client) loading JS from Metro works for the whole loop and is cheaper than Release, since JS edits reload without a native rebuild; Release isn't required. Start Metro (`npx react-native start`, or `npx expo start --dev-client`) before `subtext-sim start`; without it the snapshot shows a red "No script URL provided" box. If edits don't show up, restart Metro with `--clear`: it can serve stale modules, especially from a git worktree. If port 8081 is taken, run Metro with `--port N` and point the app at it: `xcrun simctl spawn defaults write RCT_jsLocation localhost:N`, then `relaunch`. A Release build embeds `main.jsbundle` and runs without Metro; it drops `console.log` below error unless the app calls `RCTSetLogThreshold(.info)` natively. +**Debug with Metro is fine.** A Debug build (or an Expo dev client) loading JS from Metro works for the whole loop and is cheaper than Release, since JS edits reload without a native rebuild; Release isn't required. Start Metro (`npx react-native start`, or `npx expo start --dev-client`) before `subtext-sim start`; without it the snapshot shows a red "No script URL provided" box. If edits don't show up, restart Metro with `--clear`: it can serve stale modules. **Git worktrees:** in a worktree nested inside another checkout, Watchman roots its watch at the parent checkout, so Metro never sees edits in the worktree and `--clear` doesn't help. Stop Metro, run `watchman watch-del-all` (an existing watch on the parent checkout otherwise keeps winning), add an empty `.watchmanconfig` at the worktree root so Watchman roots there, and start Metro again from the worktree. If port 8081 is taken, run Metro with `--port N` and point the app at it: `xcrun simctl spawn defaults write RCT_jsLocation localhost:N`, then `relaunch`. A Release build embeds `main.jsbundle` and runs without Metro; it drops `console.log` below error unless the app calls `RCTSetLogThreshold(.info)` natively. -**Empty or nearly empty tree.** When `snapshot` says the app exposes no accessibility elements, or shows only a small floating element (e.g. the Expo dev-menu button, which lives in its own window), the likely causes in order are: +**Empty or nearly empty tree.** When `snapshot` says the app exposes no accessibility elements, or shows only a small floating element (e.g. the Expo dev-menu button, which lives in its own window), `click`, `fill` and `scroll --until` give the same diagnosis instead of the long-list hint. The likely causes in order are: 1. A view that stays mounted hides the rest: `accessibilityViewIsModal = YES`, or `accessibilityElementsHidden` / `importantForAccessibility="no-hide-descendants"` on a root view. In RN this is often a full-window overlay (a toast host, a modal backdrop) that is invisible but still mounted. Unmount it, or set those props only while it's shown. 2. App accessibility is off: `subtext-sim relaunch` turns it on. The error leaves this out when it's already on. 3. A system sheet is animating in: snapshot again in a moment. +**The dev-menu floating button.** An Expo dev client (and some RN debug setups) draws a floating dev-menu button in its own small window, and it catches taps beyond its visible frame: a tap near it opens the dev menu instead. `click` refuses a target whose center is within 24 pt of it and suggests a `tap-xy` on another part of the target. Better, hide it: open the dev menu, Settings, turn off "Show floating action button"; open the menu later with Device → Shake (⌃⌘Z) in Simulator. `describe` doesn't report windows, so the button is recognized by heuristics: a dev-menu label, or a small square element without an identifier drawn last at the root of the tree. + **Overlays.** - In Debug, a `console.error` or `console.warn` raises a LogBox toast over the bottom of the screen. The snapshot gets a `[LogBox]` section with the `tap-xy` that hits the toast's ✕ (the ✕ has no element of its own), and `click` refuses targets under it. Tapping the toast itself opens the full-screen LogBox inspector. To keep toasts away entirely, call `LogBox.ignoreAllLogs()` in the app. - A system sheet from another process (e.g. iOS's "Save Password?" a moment after a login form goes away) replaces the snapshot with an `[Overlay]` section listing the sheet's buttons; `click 'button[text="Not Now"]'` dismisses it. @@ -195,4 +204,5 @@ The loop is the same; RN maps onto the iOS accessibility tree like this: - Network isn't live, and needs the Fullstory SDK in the app plus network capture enabled for the org. - The helper uses Apple private frameworks; a new Xcode release may need a plugin update. `doctor` says so when the helper fails to build. - On iOS 26 a SwiftUI tab bar shows up as one `other "Tab Bar"` element without its tabs; switch tabs with `tap-xy` from a screenshot. +- The corpus is always the session's `.sightmap/`; there is no `--corpus DIR` to try a draft corpus against a running session yet. - Scrolling a long list to a far row takes about 3 s per screenful (4–5 s for a React Native `FlatList`, whose tree is larger). diff --git a/skills/subtext-ios-sim/lib/BRIDGE.md b/skills/subtext-ios-sim/lib/BRIDGE.md index ab572d8..ea02b0f 100644 --- a/skills/subtext-ios-sim/lib/BRIDGE.md +++ b/skills/subtext-ios-sim/lib/BRIDGE.md @@ -28,7 +28,9 @@ with an optional `i`, descendant and `>` combinators, selector lists, `:not`, `:is`, `:where`, `:has`). We extract properties and apply the live snapshot's rules: view components before globals, first match wins, and conflicts are reported. Live `sightmap snapshot` behaves the same way, which we verified with -a probe page. +a probe page. One addition of ours: a component whose whole chain matched beats +one that matched only by skipping an absent ancestor (`matchKind`, see below), +and conflicts left between skipped matches render as `(ancestor skipped)`. ## Export shape across sightmap versions @@ -67,12 +69,15 @@ match the simulator's tree: step *k* (followed by whitespace, not `>`) may be skipped when its compound names an exact identifier (`#x`, `[identifier="x"]`, `[testid="x"]`, `[id="x"]`) and **no element in the whole tree** matches that compound, provided the - subject step has an attribute or `#id` of its own. A testID on a plain + subject step matches an attribute exactly (`=`, `~=`, `|=`, or `#id`). A testID on a plain wrapper view (not an accessibility element) never reaches the iOS tree, so `[testid="login-form"] [testid="login-submit"]` would otherwise never match. An identifier present anywhere on screen is never skipped (so a `[testid="card"] [testid="close"]` selector doesn't leak to another card's - close button outside it), prefix and substring matches are never skipped, and + close button outside it), prefix and substring ancestors are never skipped, a + subject that matches only by prefix or substring (`^=`, `$=`, `*=`) is never + reached by skipping (sibling components like `[testid="menu"] [testid*="-option-"]` + would otherwise all collapse onto every option on screen), and `[testid="list"] button` doesn't match (it would match every button). The rule doesn't apply to relative selectors inside `:has()`. `sightmap explain` / `coverage` don't apply this rule, so they can report @@ -98,6 +103,7 @@ differ. | `text` | `label`, or the children's text joined (innerText analog) | | `bounds` | `frame`, rounded to integers (sightmap requires ints) | | `isVisible` / `inViewport` | frame intersects the screen / its center is on screen | +| `isInteractive` | an interactive role; or an `other` with an identifier (RN `GenericElement`), or with a label or identifier and a `Button` / `Link` / `Adjustable` trait. Keyboard elements (`keyboard`, `key`, the `KeyboardKey` trait, and their descendants) are marked `system`: left out of coverage and component matching | | `isIgnored` | label-, identifier- and value-less `other` groups with 2+ children | Unlabeled single-child and empty `other` wrappers are removed during diff --git a/skills/subtext-ios-sim/lib/cli.mjs b/skills/subtext-ios-sim/lib/cli.mjs index 735560e..7593e7d 100644 --- a/skills/subtext-ios-sim/lib/cli.mjs +++ b/skills/subtext-ios-sim/lib/cli.mjs @@ -12,7 +12,10 @@ import { latestRunPath, readRun, RUNS_DIR } from './runlog.mjs'; const BIN = fileURLToPath(new URL('../bin/subtext-sim', import.meta.url)); -const BOOLEAN_FLAGS = new Set(['detach', 'json', 'include-offscreen', 'clear', 'help', 'shutdown', 'fix', 'follow', 'all']); +const BOOLEAN_FLAGS = new Set(['detach', 'json', 'include-offscreen', 'clear', 'help', 'shutdown', 'fix', 'follow', 'all', 'ignore-case']); +const SHORT_FLAGS = { '-h': 'help', '-i': 'ignore-case' }; +const NEGATIVE_NUMBER = /^-\d+(\.\d+)?$/; +const looksLikeFlag = (s) => s.startsWith('-') && s.length > 1 && !NEGATIVE_NUMBER.test(s); const VALUE_FLAGS = new Set(['device', 'app', 'bundle-id', 'view', 'out', 'tree-out', 'in', 'until', 'max', 'limit', 'grep', 'level', 'duration', 'sightmap-dir', 'label', 'since', 'run']); export const USAGE = `subtext-sim: drive an app in the local iOS Simulator (sightmap-browser style) @@ -29,7 +32,9 @@ Session Observe snapshot [--view V] [--include-offscreen] [--out F] [--tree-out F] [--json] screenshot --out F.png - console list [--limit N] [--grep S] [--level L[,L]] [--all] levels: debug info default error fault stdout stderr; --all includes Apple frameworks + console list [--limit N] [--grep S] [-i] [--level L[,L]] [--all] + --grep is case-insensitive (-i/--ignore-case accepted); levels: debug info + default error fault stdout stderr; --all includes Apple frameworks console get session-url Fullstory session URL from the app's logs @@ -49,7 +54,8 @@ Act (targets: snapshot id, accessibility identifier, or component query) button home lock siri side apple_pay volume_up volume_down openurl -Global: --json (machine-readable output), --sightmap-dir DIR (default: nearest .sightmap/)`; +Global: --json (machine-readable output), --sightmap-dir DIR (default: nearest .sightmap/) +A flag value that starts with "-" goes after \`--\` or \`=\`: \`--grep -- -i\`, \`--grep=-i\`.`; export function parseArgs(argv) { const flags = {}; @@ -68,14 +74,13 @@ export function parseArgs(argv) { if (eq !== -1) throw new SimError('usage', `--${name} takes no value`); flags[name] = true; } else if (VALUE_FLAGS.has(name)) { - const value = eq === -1 ? argv[++i] : arg.slice(eq + 1); - if (value === undefined) throw new SimError('usage', `--${name} needs a value`); - flags[name] = value; + flags[name] = eq === -1 ? valueAfter(argv, i, name) : arg.slice(eq + 1); + if (eq === -1) i += argv[i + 1] === '--' ? 2 : 1; } else { throw new SimError('usage', `unknown flag --${name}`); } - } else if (arg === '-h') { - flags.help = true; + } else if (SHORT_FLAGS[arg]) { + flags[SHORT_FLAGS[arg]] = true; } else { positionals.push(arg); } @@ -84,6 +89,20 @@ export function parseArgs(argv) { return { command, positionals: rest, flags }; } +/** The value of a space-separated flag; one that looks like a flag must come after `--` (or use `--name=value`). */ +function valueAfter(argv, i, name) { + const next = argv[i + 1]; + if (next === '--') { + if (argv[i + 2] === undefined) throw new SimError('usage', `--${name} needs a value`); + return argv[i + 2]; + } + if (next === undefined) throw new SimError('usage', `--${name} needs a value`); + if (looksLikeFlag(next)) { + throw new SimError('usage', `--${name} needs a value, but got the flag ${next}`, `to use ${JSON.stringify(next)} as the value, write --${name}=${next} or --${name} -- ${next}`); + } + return next; +} + const num = (value, name) => { const n = Number(value); if (value === undefined || value === '' || !Number.isFinite(n)) throw new SimError('usage', `${name} must be a number, got ${JSON.stringify(value)}`); @@ -139,7 +158,7 @@ export function toRequest({ command, positionals: p, flags }, cwd = process.cwd( case 'relaunch': return { cmd: 'relaunch', args: {} }; case 'session-url': - return { cmd: 'sessionUrl', args: {} }; + return { cmd: 'sessionUrl', args: { json: !!flags.json } }; case 'console': { const [sub, index] = p; if (sub === 'list') return { cmd: 'consoleList', args: { limit: flags.limit ? num(flags.limit, '--limit') : 50, grep: flags.grep, level: flags.level, all: !!flags.all } }; @@ -168,12 +187,14 @@ const consoleLine = (e) => { return `#${e.index} [--] ${e.level.padEnd(7)}${where} ${e.message.split('\n')[0]}`; }; +const waited = (r) => (r.retries ? ` after ${r.retries} re-read(s) while the screen settled` : ''); + /** Human output for each daemon command, in the shape of `sightmap browser`. */ export function formatResult(cmd, r) { switch (cmd) { case 'snapshot': return r.text; - case 'click': return `✓ clicked ${r.target} at (${r.x}, ${r.y})`; - case 'fill': return `✓ filled ${r.target} → value ${JSON.stringify(r.value)}`; + case 'click': return `✓ clicked ${r.target} at (${r.x}, ${r.y})${waited(r)}`; + case 'fill': return `✓ filled ${r.target} → value ${JSON.stringify(r.value)}${r.placeholder ? ` (empty; the field shows its placeholder ${JSON.stringify(r.placeholder)})` : ''}${waited(r)}`; case 'scroll': if (r.found) return `✓ ${r.found} is on screen after ${r.scrolls} scroll(s) ${r.direction}; snapshot ids refreshed, act on it by id ${r.id} or by the same query`; return `✓ scrolled ${r.direction}; take a new snapshot`; @@ -210,7 +231,9 @@ function formatStatus(s, sessionFile) { ` app: ${s.bundleId}${s.appPid ? ` (pid ${s.appPid})` : ''}${s.appPath ? ` ${s.appPath}` : ''}`, ` helper: ${s.helper.alive ? 'running' : 'idle'}${s.helper.restarts ? `, restarted ${s.helper.restarts}x` : ''}`, ` console: ${s.logs.entries} entries${s.logs.streaming ? '' : ' (log stream not running)'}`, - ...(s.sessionUrl ? [` fullstory: ${s.sessionUrl}`] : []), + ...(s.sessionEnded + ? [` fullstory: ✗ session ${s.sessionEnded.url ?? '(no URL)'} ended at ${s.sessionEnded.time}: ${s.sessionEnded.reason}. Next: \`subtext-sim relaunch\` for a new session`] + : s.sessionUrl ? [` fullstory: ${s.sessionUrl}`] : []), ` session: ${sessionFile}`, ...(s.run ? [` run: ${s.run.run} (${s.run.seq} entries)`] : []), ].join('\n'); @@ -320,6 +343,7 @@ function formatStart(r, cwd) { export function formatRunEntry(e) { if (e.action === 'session') return `${e.seq} ${e.ts} session ${e.url}`; + if (e.action === 'session-ended') return `${e.seq} ${e.ts} session-ended ${e.url ?? '(no URL)'}: ${e.reason}`; const t = e.target; const target = t ? ` → ${t.id} ${t.component ? `[${t.component}]` : t.role}${t.identifier ? ` ${t.identifier}` : ''}${t.text ? ` ${JSON.stringify(t.text)}` : ''}` : ''; const args = Object.entries(e.args ?? {}).filter(([k]) => k !== 'target').map(([k, v]) => `${k}=${JSON.stringify(v)}`).join(' '); @@ -443,6 +467,7 @@ export async function main(argv = process.argv.slice(2), { cwd = process.cwd() } const session = await requireSession(sightmapDir); const result = await call(session.socket, request.cmd, request.args); print(request.cmd, result); + if (request.cmd === 'sessionUrl' && result.ended) return 1; if (request.cmd === 'snapshot' && result.coverage?.interactive === 0) { console.error('error: 0 interactive elements; the app may still be loading. Next: wait a moment and snapshot again'); return 1; diff --git a/skills/subtext-ios-sim/lib/daemon.mjs b/skills/subtext-ios-sim/lib/daemon.mjs index 2a964c0..df14178 100644 --- a/skills/subtext-ios-sim/lib/daemon.mjs +++ b/skills/subtext-ios-sim/lib/daemon.mjs @@ -4,8 +4,8 @@ import { createServer } from 'node:net'; import { tmpdir } from 'node:os'; import { dirname, join, resolve as resolvePath } from 'node:path'; import { createInterface } from 'node:readline'; -import { BridgeError, annotate, centerOf, render, resolve as resolveQuery, toComponentTree, writeSnap } from './sightmap-bridge.mjs'; -import { floatingOnly, logBoxToast, normalize } from './normalize.mjs'; +import { BridgeError, annotate, centerOf, render, resolve as resolveQuery, sparseTreeNote, toComponentTree, writeSnap } from './sightmap-bridge.mjs'; +import { FLOATING_MARGIN, floatingControls, logBoxToast, normalize } from './normalize.mjs'; import { emptyTreeMessage, SimError, toWire } from './errors.mjs'; import { ConsoleBuffer, LogCollector } from './logs.mjs'; import { HelperClient } from './helper-client.mjs'; @@ -94,6 +94,16 @@ function hasAncestorTag(node, tag) { return false; } +/** A point inside the node's frame, on screen and outside `zone`, nearest its center; or null. */ +function clearSpot(node, zone, screen) { + const { x, y, w, h } = node.frame; + const c = centerOf(node); + const points = [0.15, 0.3, 0.5, 0.7, 0.85].flatMap((fx) => [0.25, 0.5, 0.75].map((fy) => ({ x: Math.round(x + w * fx), y: Math.round(y + h * fy) }))); + const clear = points.filter((p) => p.x >= 0 && p.y >= 0 && p.x <= screen.w && p.y <= screen.h + && !(p.x >= zone.x && p.x <= zone.x + zone.w && p.y >= zone.y && p.y <= zone.y + zone.h)); + return clear.sort((a, b) => Math.hypot(a.x - c.x, a.y - c.y) - Math.hypot(b.x - c.x, b.y - c.y))[0] ?? null; +} + /** Throws unless the node's center is on screen, uncovered by bars or the keyboard, and enabled. */ export function assertActionable(annotated, node, target) { const { screen } = annotated; @@ -112,6 +122,16 @@ export function assertActionable(annotated, node, target) { throw new SimError('covered', `${desc} is covered by a React Native LogBox toast (${describeNode(logBox.node)})`, `dismiss it with \`subtext-sim tap-xy ${logBox.dismiss.x} ${logBox.dismiss.y}\` (its ✕), then retry`); } + const unrelated = (o) => o !== node && !isAncestor(o, node) && !isAncestor(node, o); + const zoneOf = ({ frame: { x, y, w, h } }) => ({ x: x - FLOATING_MARGIN, y: y - FLOATING_MARGIN, w: w + 2 * FLOATING_MARGIN, h: h + 2 * FLOATING_MARGIN }); + const inside = (p, z) => p.x >= z.x && p.x <= z.x + z.w && p.y >= z.y && p.y <= z.y + z.h; + const floating = floatingControls(annotated).find((f) => unrelated(f) && inside(c, zoneOf(f))); + if (floating) { + const spot = clearSpot(node, zoneOf(floating), screen); + const away = spot ? `tap another part of the target with \`subtext-sim tap-xy ${spot.x} ${spot.y}\`` : 'scroll the target away from it'; + throw new SimError('covered', `${desc} is within ${FLOATING_MARGIN} pt of ${describeNode(floating)}, which looks like the React Native / Expo dev-menu button. It floats in its own window and catches taps beyond its frame, so tapping (${Math.round(c.x)}, ${Math.round(c.y)}) would likely open the dev menu`, + `${away}, or hide the floating button (Expo dev client: open the dev menu, Settings, turn off "Show floating action button"; reopen the menu later with Device → Shake in Simulator), then retry`); + } const cover = [...annotated.nodes, ...(band ? [band] : [])].find((o) => obstructs(o) && o.visible && o !== node && !isAncestor(o, node) && !isAncestor(node, o) && covers(o)); if (cover) { @@ -154,6 +174,12 @@ function clip(frame, screen) { const SECURE = new Set(['securetextfield']); +// Errors that can clear up once a sheet or navigation finishes animating. +const RETRYABLE = new Set(['not_found', 'stale_target', 'covered', 'offscreen', 'disabled']); + +/** Frames only: labels such as clocks and counters change without anything moving. */ +const layoutOf = (snap) => snap.nodes.map((n) => `${n.tag}|${n.identifier}|${Math.round(n.frame.x)},${Math.round(n.frame.y)},${Math.round(n.frame.w)},${Math.round(n.frame.h)}`).join('\n'); + // Points per second. Faster swipes fling the list, which can carry a target // past the screen between two reads; this speed stops where the finger stops. const SCROLL_SPEED = 300; @@ -167,13 +193,17 @@ const ACTION_BY_CMD = { export class SimSession { constructor({ udid, deviceName = null, bundleId, appPath = null, processName, sightmapDir, runDir, helper, simctl = realSimctl, logs = null, buffer = new ConsoleBuffer(), corpus = new CorpusCache(sightmapDir), log = () => {}, timing = {} }) { Object.assign(this, { udid, deviceName, bundleId, appPath, processName, sightmapDir, runDir, helper, simctl, logs, buffer, corpus, log }); - this.timing = { settleMs: 350, scrollSettleMs: 150, overlaySettleMs: 800, fillReadbackMs: 2000, readyTimeoutMs: 20_000, ...timing }; + this.timing = { settleMs: 350, scrollSettleMs: 150, overlaySettleMs: 800, fillReadbackMs: 2000, readyTimeoutMs: 20_000, retryIntervalMs: 150, retryTimeoutMs: 1500, ...timing }; this.last = null; this.lastView = null; this.lastTarget = null; this.appPid = null; this.runLog = new RunLog(sightmapDir); this.buffer.onSessionUrl = (url) => this.runLog.run && this.runLog.append({ action: 'session', url }); + this.buffer.onSessionEnded = ({ url, reason, status }) => { + this.log(`fullstory session ended: ${reason}`); + if (this.runLog.run) this.runLog.append({ action: 'session-ended', url, reason, status }); + }; this.startedAt = new Date().toISOString(); } @@ -227,6 +257,14 @@ export class SimSession { return this.axEnabled; } + /** The not-found error for a sparse tree, naming its likely causes; null when the tree looks healthy. */ + async sparseTreeError(snap, message) { + const note = sparseTreeNote(snap, { udid: this.udid, accessibilityEnabled: await this.accessibilityEnabled() }); + if (!note) return null; + return new SimError('not_found', `${message}; the screen exposes next to nothing to accessibility, so the target is not just further down a list:\n${note.join('\n')}`, + 'check the causes listed, in order, then snapshot again'); + } + async emptyTreeError() { return new SimError('ax_disabled', emptyTreeMessage({ udid: this.udid, accessibilityEnabled: await this.accessibilityEnabled() })); } @@ -321,6 +359,8 @@ export class SimSession { runNew({ label }) { const run = this.runLog.begin(label); if (this.buffer.sessionUrl) this.runLog.append({ action: 'session', url: this.buffer.sessionUrl }); + const ended = this.buffer.sessionEnded; + if (ended) this.runLog.append({ action: 'session-ended', url: ended.url, reason: ended.reason, status: ended.status }); return run; }, @@ -340,13 +380,14 @@ export class SimSession { helper: { alive: this.helper.alive, restarts: this.helper.restarts }, logs: { streaming: this.logs?.streaming ?? false, entries: this.buffer.nextIndex }, sessionUrl: this.buffer.sessionUrl, + sessionEnded: this.buffer.sessionEnded, run: { run: this.runLog.run, path: this.runLog.path, seq: this.runLog.seq }, }; }, async snapshot({ view, includeOffscreen = false, out, treeOut, json = false }) { const snap = await this.fresh({ view }); - if (floatingOnly(snap)) { + if (sparseTreeNote(snap)) { Object.assign(snap, { udid: this.udid, accessibilityEnabled: await this.accessibilityEnabled() }); snap.text = render(snap); } @@ -371,12 +412,15 @@ export class SimSession { if (typeof text !== 'string') throw new SimError('usage', 'fill needs the text to type'); const tapped = await this.tap(target); await sleep(this.timing.settleMs); + let placeholder = null; if (clear) { const field = relocate(await this.fresh(), tapped.node, target); const count = (field.value ?? '').length; for (let i = 0; i < count; i++) await this.input('key', { key: 'forward_delete' }); for (let i = 0; i < count; i++) await this.input('key', { key: 'delete' }); if (count) await sleep(this.timing.settleMs); + // Every character is deleted by now; a field that still shows text shows its placeholder. + placeholder = relocate(await this.fresh(), tapped.node, target).value; } if (text) await this.input('type', { text }); const stuck = (n) => SECURE.has(n.tag) || !text || (n.value ?? '').toLowerCase().includes(text.toLowerCase()); @@ -386,12 +430,17 @@ export class SimSession { await sleep(this.timing.settleMs); after = relocate(await this.fresh(), tapped.node, target); } while (!stuck(after) && Date.now() < deadline); - const value = after.value ?? ''; + const showsPlaceholder = !!placeholder && after.value === placeholder && !text; + const value = showsPlaceholder ? '' : after.value ?? ''; if (!stuck(after)) { throw new SimError('fill_mismatch', `typed ${quote(text)} but ${describeNode(after)} shows ${quote(value)}`, 'retry with --clear, or check with `subtext-sim snapshot` that the field took focus'); } - return { target: tapped.target, x: tapped.x, y: tapped.y, value: SECURE.has(after.tag) ? `(${value.length} secure characters)` : value }; + return { + target: tapped.target, x: tapped.x, y: tapped.y, retries: tapped.retries, + value: SECURE.has(after.tag) ? `(${value.length} secure characters)` : value, + ...(showsPlaceholder ? { placeholder } : {}), + }; }, async scroll({ direction, within, until, max = 20 }) { @@ -426,8 +475,14 @@ export class SimSession { } catch (err) { if (!['not_found', 'offscreen', 'covered'].includes(err.code)) throw err; } + if (i === 0) { + const sparse = await this.sparseTreeError(snap, `${quote(until)} not found`); + if (sparse) throw sparse; + } if (previous !== null && previous === snap.text) { this.last = snap; + const sparse = await this.sparseTreeError(snap, `${quote(until)} not found`); + if (sparse) throw sparse; throw new SimError('not_found', `${quote(until)} not found; reached the end after scrolling ${direction} ${i} time(s)`, `try \`subtext-sim scroll ${direction === 'down' ? 'up' : direction === 'up' ? 'down' : direction === 'left' ? 'right' : 'left'} --until ${quote(until)}\``); } @@ -437,6 +492,8 @@ export class SimSession { await sleep(this.timing.scrollSettleMs); } this.last = snap; + const sparse = await this.sparseTreeError(snap, `${quote(until)} not found`); + if (sparse) throw sparse; throw new SimError('not_found', `${quote(until)} not found after ${max} scroll(s) ${direction}`, 'raise --max, or check the query with `subtext-sim snapshot`'); }, @@ -506,12 +563,18 @@ export class SimSession { return entry; }, - sessionUrl() { + sessionUrl({ json = false } = {}) { + const ended = this.buffer.sessionEnded; + if (ended) { + if (json) return { url: ended.url, ended: true, reason: ended.reason, endedAt: ended.time }; + throw new SimError('session_ended', `the Fullstory session ${ended.url ?? '(no URL logged)'} ended at ${ended.time}: ${ended.reason}`, + 'run `subtext-sim relaunch` to start a new session, then `subtext-sim session-url` for its URL'); + } if (!this.buffer.sessionUrl) { throw new SimError('no_session_url', 'the app has not logged a Fullstory session URL', 'the app needs the Fullstory iOS SDK (for the sample app: FS_ORG_ID= fixtures/SampleApp/build.sh), then `subtext-sim relaunch`'); } - return { url: this.buffer.sessionUrl }; + return { url: this.buffer.sessionUrl, ended: false }; }, }; @@ -526,26 +589,58 @@ export class SimSession { } } - /** Re-describes, re-resolves the target, checks it is tappable, and taps its center. */ + /** + * Re-describes, re-resolves the target, checks it is tappable, and taps its + * center. Right after a sheet closes or a navigation pops, the target can be + * missing or still moving, so it re-reads every `retryIntervalMs` until two + * reads agree and the target is there, for up to `retryTimeoutMs`. + */ async tap(target) { const numeric = /^\d+$/.test(String(target).trim()); - let base = this.last; + const base = this.last; if (numeric && !base) throw new SimError('not_found', `no snapshot to resolve id ${target} against`, 'run `subtext-sim snapshot` first'); - const fresh = await this.fresh({ view: base?.viewInferred === false ? base.view ?? undefined : undefined }); - base ??= fresh; - let node; - try { - node = numeric ? relocate(fresh, resolveTarget(base, target), target) : resolveTarget(fresh, target); - } catch (err) { - if (err.code !== 'not_found' || numeric) throw err; - throw new SimError('not_found', `${err.message}; long lists expose only the rows near the screen`, - `if it is further down a list, run \`subtext-sim scroll down --until ${quote(target)}\`; otherwise \`subtext-sim snapshot\``); + const view = base?.viewInferred === false ? base.view ?? undefined : undefined; + const original = numeric ? resolveTarget(base, target) : null; + const locate = (snap) => (numeric ? relocate(snap, original, target) : resolveTarget(snap, target)); + const deadline = Date.now() + this.timing.retryTimeoutMs; + let retries = 0; + let stable = false; + let snap = await this.fresh({ view }); + for (;;) { + let node = null; + let failure = null; + try { + node = locate(snap); + assertActionable(snap, node, target); + } catch (err) { + if (!(err instanceof SimError) || !RETRYABLE.has(err.code)) throw err; + failure = err; + } + if (node) this.lastTarget = node; + const outOfTime = Date.now() >= deadline; + if (!failure && (stable || outOfTime)) { + const { x, y } = centerOf(node); + await this.input('tap', { x, y }); + return { target: describeNode(node), x: Math.round(x), y: Math.round(y), retries, node }; + } + const waitable = failure && (failure.code === 'not_found' || failure.code === 'stale_target' || !stable); + if (failure && (outOfTime || !waitable)) throw await this.notFound(snap, target, failure, numeric); + await sleep(this.timing.retryIntervalMs); + const next = await this.fresh({ view }); + stable = layoutOf(next) === layoutOf(snap); + if (failure || !stable) retries++; + snap = next; } - this.lastTarget = node; - assertActionable(fresh, node, target); - const { x, y } = centerOf(node); - await this.input('tap', { x, y }); - return { target: describeNode(node), x: Math.round(x), y: Math.round(y), node }; + } + + /** The error for a target that never became tappable: the sparse-tree diagnosis, the long-list hint, or the error itself. */ + async notFound(snap, target, err, numeric) { + if (err.code !== 'not_found') return err; + const sparse = await this.sparseTreeError(snap, err.message); + if (sparse) return sparse; + if (numeric) return err; + return new SimError('not_found', `${err.message}; long lists expose only the rows near the screen`, + `if it is further down a list, run \`subtext-sim scroll down --until ${quote(target)}\`; otherwise \`subtext-sim snapshot\``); } close() { diff --git a/skills/subtext-ios-sim/lib/doctor.mjs b/skills/subtext-ios-sim/lib/doctor.mjs index 218f91d..07a388c 100644 --- a/skills/subtext-ios-sim/lib/doctor.mjs +++ b/skills/subtext-ios-sim/lib/doctor.mjs @@ -2,7 +2,7 @@ import { execFile } from 'node:child_process'; import { existsSync } from 'node:fs'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; -import { resolveSightmap, SIGHTMAP_PACKAGE, sightmapVersionWarning } from './sightmap-cli.mjs'; +import { resolveSightmap, SIGHTMAP_PACKAGE, SIGHTMAP_VERSION, sightmapVersionWarning } from './sightmap-cli.mjs'; const SKILL_DIR = join(dirname(fileURLToPath(import.meta.url)), '..'); const HELPER_BUILD = join(SKILL_DIR, 'helper', 'build.sh'); @@ -94,6 +94,13 @@ async function checkNode() { return pass('node', `Node ${version}`); } +/** Which sightmap resolves and why: SIGHTMAP_BIN, then PATH (so users can pin their own), then the npx pin. */ +export function sightmapDetail({ version, source }) { + if (source.startsWith('npx')) return `using the pinned sightmap ${version} (${source})`; + const where = source.startsWith('SIGHTMAP_BIN') ? 'SIGHTMAP_BIN' : 'PATH'; + return `using ${where} sightmap ${version} (${source.replace(/^on PATH: |^SIGHTMAP_BIN=/, '')}; pinned fallback is ${SIGHTMAP_VERSION})`; +} + async function checkSightmap() { let resolved; try { @@ -101,7 +108,7 @@ async function checkSightmap() { } catch (err) { return fail('sightmap', err.message, `npm install -g ${SIGHTMAP_PACKAGE}`, ['npm', ['install', '-g', SIGHTMAP_PACKAGE]]); } - const detail = `sightmap ${resolved.version} (${resolved.source})`; + const detail = sightmapDetail(resolved); const warning = sightmapVersionWarning(resolved.version); if (!warning) return pass('sightmap', detail); const fix = resolved.source.startsWith('SIGHTMAP_BIN') diff --git a/skills/subtext-ios-sim/lib/logs.mjs b/skills/subtext-ios-sim/lib/logs.mjs index 9656404..cf9ff3f 100644 --- a/skills/subtext-ios-sim/lib/logs.mjs +++ b/skills/subtext-ios-sim/lib/logs.mjs @@ -43,6 +43,31 @@ export function sessionUrlIn(message) { return null; } +// The ingest server answers 206 ("bad request, don't retry", mostly a bundle +// that is too large) or 202 (recording blocked), and the SDK then terminates +// the session for good. +const HTTP_TERMINATION = /Received HTTP Status Code:?\s*(\d{3})\b[^\n]*terminating\b/i; +const CAPTURE_TERMINATED = /capture is terminated/i; +const MAX_VIEWS = /Maximum number of views reached/i; + +const TERMINATION_REASONS = { + 206: 'the Fullstory server rejected a bundle with HTTP 206 (usually a bundle that is too large, e.g. a very large view tree) and the SDK terminated the session', + 202: 'the Fullstory server answered HTTP 202 (recording is blocked for this app or org) and the SDK terminated the session', +}; + +/** What a console message says about the Fullstory session ending: `{status, reason}`, `{warning}`, or null. */ +export function sessionEndIn(message) { + const text = String(message); + const http = HTTP_TERMINATION.exec(text); + if (http) { + const status = Number(http[1]); + return { status, reason: TERMINATION_REASONS[status] ?? `the Fullstory server answered HTTP ${status} and the SDK terminated the session` }; + } + if (CAPTURE_TERMINATED.test(text)) return { status: null, reason: 'the Fullstory SDK reports that capture is terminated' }; + if (MAX_VIEWS.test(text)) return { warning: 'the Fullstory SDK logged "Maximum number of views reached"' }; + return null; +} + /** A bounded console buffer with stable, ever-increasing indexes. */ export class ConsoleBuffer { constructor(capacity = 5000) { @@ -50,6 +75,8 @@ export class ConsoleBuffer { this.entries = []; this.nextIndex = 0; this.sessionUrl = null; + this.sessionEnded = null; + this.sessionWarnings = []; } push(entry) { @@ -59,8 +86,18 @@ export class ConsoleBuffer { const url = sessionUrlIn(full.message); if (url && url !== this.sessionUrl) { this.sessionUrl = url; + this.sessionEnded = null; + this.sessionWarnings = []; this.onSessionUrl?.(url); } + const end = sessionEndIn(full.message); + if (end?.warning) { + if (!this.sessionWarnings.includes(end.warning)) this.sessionWarnings.push(end.warning); + } else if (end && !this.sessionEnded) { + const notes = this.sessionWarnings.length ? `; before that, ${this.sessionWarnings.join('; ')}` : ''; + this.sessionEnded = { url: this.sessionUrl, status: end.status, reason: `${end.reason}${notes}`, time: full.time, index: full.index }; + this.onSessionEnded?.(this.sessionEnded); + } return full; } @@ -146,10 +183,7 @@ export class LogCollector { stdio: ['ignore', 'pipe', 'pipe'], }); this.proc = proc; - createInterface({ input: proc.stdout }).on('line', (line) => { - const entry = parseLogLine(line); - if (entry) this.buffer.push(entry); - }); + createInterface({ input: proc.stdout }).on('line', (line) => this.ingest(line)); proc.on('exit', (code) => { if (this.proc !== proc) return; this.proc = null; @@ -160,6 +194,12 @@ export class LogCollector { }); } + /** One `log stream --style ndjson` line into the buffer. */ + ingest(line) { + const entry = parseLogLine(line); + if (entry) this.buffer.push(entry); + } + get streaming() { return !!this.proc; } diff --git a/skills/subtext-ios-sim/lib/normalize.mjs b/skills/subtext-ios-sim/lib/normalize.mjs index 0274d7d..696bc86 100644 --- a/skills/subtext-ios-sim/lib/normalize.mjs +++ b/skills/subtext-ios-sim/lib/normalize.mjs @@ -89,6 +89,20 @@ function isIdentifiedGeneric(raw) { return type === 'genericelement' && !!present(raw.identifier); } +// The system keyboard's window: its container, its keys, and anything carrying +// the KeyboardKey trait. Not part of the app, so not in coverage or components. +function isKeyboard(raw, tag) { + return tag === 'keyboard' || tag === 'key' || (raw.traits ?? []).includes('KeyboardKey'); +} + +const TAPPABLE_TRAITS = new Set(['Button', 'Link', 'Adjustable']); + +// A labelled or identified `other` whose traits say it can be tapped: React +// Native Pressables surface as `other`, and an accessibilityRole adds the trait. +function isTappableOther(raw, tag) { + return tag === 'other' && (raw.traits ?? []).some((t) => TAPPABLE_TRAITS.has(t)) && !!(present(raw.label) || present(raw.identifier)); +} + function collapse(raw, isRoot) { const tag = rnTabTitle(raw) !== null ? 'tab' : tagFor(raw.type, raw.role, raw.subrole); let children = (raw.children ?? []).flatMap((c) => collapse(c, false)); @@ -136,7 +150,8 @@ export function normalize(describeResult) { frame, visible: intersects(frame, screen), onScreen: centerOnScreen(frame, screen), - interactive: INTERACTIVE_TAGS.has(tag) || isIdentifiedGeneric(raw), + interactive: INTERACTIVE_TAGS.has(tag) || isIdentifiedGeneric(raw) || isTappableOther(raw, tag), + system: isKeyboard(raw, tag) || !!parent?.system, ignored: isWrapper(raw, tag), nthChild, text: '', @@ -182,6 +197,28 @@ export function floatingOnly({ nodes, root, screen, overlay }) { return (x1 - x0) * (y1 - y0) < 0.1 * screen.w * screen.h ? shown : null; } +const DEV_MENU_LABEL = /\b(dev(eloper)?[\s-]?menu|dev[\s-]?tools)\b/i; +/** How far beyond its frame a floating button still catches taps, in points. */ +export const FLOATING_MARGIN = 24; + +function fabShaped({ frame: { w, h }, children, identifier }) { + return !children.length && !identifier && w >= 32 && h >= 32 && w <= 96 && h <= 96 && w / h >= 0.6 && w / h <= 1.67; +} + +/** + * Controls that most likely float in a separate small window above the app, + * like the React Native / Expo dev-menu button. `describe` reports no windows, + * so this is a heuristic: a dev-menu label or identifier, or a small square + * element without an identifier drawn last at the root, as a window above the + * app's own would be. + */ +export function floatingControls({ root, nodes }) { + const top = root.children.filter((n) => n.visible); + const last = top.length > 1 ? top.at(-1) : null; + return nodes.filter((n) => n !== root && n.visible + && (DEV_MENU_LABEL.test(n.label ?? '') || DEV_MENU_LABEL.test(n.identifier ?? '') || (n === last && fabShaped(n)))); +} + const CSS_IDENT = /^-?[A-Za-z_][A-Za-z0-9_-]*$/; export function cssId(node) { diff --git a/skills/subtext-ios-sim/lib/selector.mjs b/skills/subtext-ios-sim/lib/selector.mjs index 665763a..afefe1b 100644 --- a/skills/subtext-ios-sim/lib/selector.mjs +++ b/skills/subtext-ios-sim/lib/selector.mjs @@ -216,7 +216,7 @@ function* descendants(node) { } } -function matchCompound(node, c) { +function matchCompound(node, c, skip = true) { if (c.tag && c.tag !== '*' && c.tag !== node.tag) return false; if (c.classes.length) return false; for (const id of c.ids) if (node.identifier !== id) return false; @@ -226,21 +226,22 @@ function matchCompound(node, c) { if (a.op === null ? actual === undefined : !compareValue(actual, a.op, a.value, a.ci)) return false; } for (const p of c.pseudos) { - if (p.name === 'not' && matchesList(node, p.list)) return false; - if ((p.name === 'is' || p.name === 'where') && !matchesList(node, p.list)) return false; - if (p.name === 'has' && !hasMatch(node, p.list)) return false; + if (p.name === 'not' && matchesList(node, p.list, null, skip)) return false; + if ((p.name === 'is' || p.name === 'where') && !matchesList(node, p.list, null, skip)) return false; + if (p.name === 'has' && !hasMatch(node, p.list, skip)) return false; } return true; } -function hasMatch(scope, list) { - for (const d of descendants(scope)) if (matchesList(d, list, scope)) return true; +function hasMatch(scope, list, skip) { + for (const d of descendants(scope)) if (matchesList(d, list, scope, skip)) return true; return false; } const IDENTIFIER_ATTRS = new Set(['identifier', 'testid', 'id']); +const EXACT_OPS = new Set(['=', '~=', '|=']); const namesIdentifier = (c) => c.ids.length > 0 || c.attrs.some((a) => IDENTIFIER_ATTRS.has(a.name) && a.op === '='); -const discriminates = (c) => c.ids.length > 0 || c.attrs.some((a) => a.op !== null); +const matchesExactly = (c) => c.ids.length > 0 || c.attrs.some((a) => EXACT_OPS.has(a.op)); function rootOf(node) { let r = node; @@ -265,28 +266,29 @@ function absentFromTree(node, compound) { /** * Ancestor step `k` of a descendant chain may be skipped when it names an exact - * identifier that no element of the tree carries, and the subject step has an - * attribute of its own: a testID on a plain wrapper view never reaches the iOS - * accessibility tree, so its descendants would otherwise never match, while a - * bare `[testid="list"] button` would match every button. + * identifier that no element of the tree carries, and the subject step matches + * an attribute exactly: a testID on a plain wrapper view never reaches the iOS + * accessibility tree, so its descendants would otherwise never match. A bare + * `[testid="list"] button`, or a substring-only `[testid="list"] [testid*="-option-"]`, + * would collapse onto every button or option on screen, so neither skips. */ -function skippable(node, parts, k, scope) { - return !scope && k < parts.length - 1 && parts[k + 1].combinator === ' ' - && discriminates(parts.at(-1).compound) +function skippable(node, parts, k, scope, skip) { + return skip && !scope && k < parts.length - 1 && parts[k + 1].combinator === ' ' + && matchesExactly(parts.at(-1).compound) && namesIdentifier(parts[k].compound) && absentFromTree(node, parts[k].compound); } -function matchAncestors(node, parts, k, scope) { +function matchAncestors(node, parts, k, scope, skip) { for (let a = node.parent; a; a = a.parent) { if (scope && a === scope) return false; - if (matchFrom(a, parts, k, scope)) return true; + if (matchFrom(a, parts, k, scope, skip)) return true; } - if (!skippable(node, parts, k, scope)) return false; - return k === 0 || matchAncestors(node, parts, k - 1, scope); + if (!skippable(node, parts, k, scope, skip)) return false; + return k === 0 || matchAncestors(node, parts, k - 1, scope, skip); } -function matchFrom(node, parts, k, scope) { - if (!matchCompound(node, parts[k].compound)) return false; +function matchFrom(node, parts, k, scope, skip) { + if (!matchCompound(node, parts[k].compound, skip)) return false; const { combinator } = parts[k]; if (k === 0) { if (!scope || combinator === null) return true; @@ -294,14 +296,21 @@ function matchFrom(node, parts, k, scope) { for (let a = node.parent; a; a = a.parent) if (a === scope) return true; return false; } - if (combinator === '>') return !!node.parent && matchFrom(node.parent, parts, k - 1, scope); - return matchAncestors(node, parts, k - 1, scope); + if (combinator === '>') return !!node.parent && matchFrom(node.parent, parts, k - 1, scope, skip); + return matchAncestors(node, parts, k - 1, scope, skip); } -function matchesList(node, list, scope = null) { - return list.some((complex) => matchFrom(node, complex.parts, complex.parts.length - 1, scope)); +function matchesList(node, list, scope = null, skip = true) { + return list.some((complex) => matchFrom(node, complex.parts, complex.parts.length - 1, scope, skip)); } export function matches(node, selector) { return matchesList(node, parseSelector(selector)); } + +/** `'full'` when the whole chain matched, `'skipped'` when it matched only by skipping absent ancestors, else null. */ +export function matchKind(node, selector) { + const list = parseSelector(selector); + if (!matchesList(node, list)) return null; + return matchesList(node, list, null, false) ? 'full' : 'skipped'; +} diff --git a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs index 6d47c87..b308195 100644 --- a/skills/subtext-ios-sim/lib/sightmap-bridge.mjs +++ b/skills/subtext-ios-sim/lib/sightmap-bridge.mjs @@ -2,7 +2,7 @@ import { mkdirSync, writeFileSync } from 'node:fs'; import { dirname } from 'node:path'; import { attrsOf, cssId, floatingOnly, logBoxToast, normalize } from './normalize.mjs'; import { emptyTreeMessage } from './errors.mjs'; -import { compareValue, matches } from './selector.mjs'; +import { compareValue, matches, matchKind } from './selector.mjs'; import { loadCorpus } from './corpus.mjs'; export class BridgeError extends Error { @@ -35,6 +35,21 @@ function matchedNodes(nodes, def) { return nodes.filter((n) => def.selectors.some((sel) => matches(n, sel))); } +/** Each node the definition matches, with whether its best match needed an absent ancestor skipped. */ +function matchedWithKind(nodes, def) { + const out = []; + for (const n of nodes) { + let kind = null; + for (const sel of def.selectors) { + const k = matchKind(n, sel); + if (k === 'full') { kind = k; break; } + kind ??= k; + } + if (kind) out.push({ node: n, skipped: kind === 'skipped' }); + } + return out; +} + function inferView(nodes, corpus) { const globalNames = new Set(corpus.globals.map((c) => c.name)); let best = null; @@ -117,7 +132,7 @@ function extractOne(node, extract, memo) { function computeCoverage(nodes) { const counts = { interactive: 0, t1: 0, t2: 0, t3: 0 }; for (const n of nodes) { - if (!n.interactive || !n.visible) continue; + if (!n.interactive || !n.visible || n.system) continue; counts.interactive++; if (n.component) counts.t1++; else { @@ -138,30 +153,38 @@ function computeCoverage(nodes) { export function annotate(normalized, sightmap, { view, sightmapBin, url } = {}) { const corpus = typeof sightmap === 'string' ? loadCorpus(sightmap, { bin: sightmapBin }) : sightmap; const { nodes } = normalized; + const matchable = nodes.filter((n) => !n.system); let activeView = null; if (view) { activeView = corpus.views.find((v) => v.name === view); if (!activeView) throw new BridgeError('unknown_view', `no view named ${quote(view)} in the corpus`); } else { - activeView = inferView(nodes, corpus); + activeView = inferView(matchable, corpus); } const defs = [...(activeView?.components ?? []), ...corpus.globals]; const candidates = new Map(); for (const def of defs) { - for (const node of matchedNodes(nodes, def)) { + for (const { node, skipped } of matchedWithKind(matchable, def)) { const list = candidates.get(node) ?? []; - if (!list.some((d) => d.name === def.name)) list.push(def); + const seen = list.find((c) => c.def.name === def.name); + if (!seen) list.push({ def, skipped }); + else if (seen.skipped && !skipped) seen.skipped = false; candidates.set(node, list); } } + // A component whose whole chain matched beats one that needed an absent + // ancestor skipped; among equals, view components and then corpus order win. const conflicts = []; for (const node of nodes) { - const list = candidates.get(node); - node.component = list ? list[0].name : null; - Object.defineProperty(node, 'componentDef', { value: list ? list[0] : null, enumerable: false, writable: true }); - if (list && list.length > 1) conflicts.push({ id: node.id, names: list.map((d) => d.name) }); + const list = candidates.get(node)?.sort((a, b) => a.skipped - b.skipped); + node.component = list ? list[0].def.name : null; + Object.defineProperty(node, 'componentDef', { value: list ? list[0].def : null, enumerable: false, writable: true }); + const full = list?.filter((c) => !c.skipped).length ?? 0; + if (list && list.length > 1 && full !== 1) { + conflicts.push({ id: node.id, names: list.map((c) => c.def.name), skipped: list.filter((c) => c.skipped).map((c) => c.def.name) }); + } } const memo = new Map(); @@ -197,6 +220,25 @@ export function snapshot(describeResult, sightmap, options) { return annotate(normalize(describeResult), sightmap, options); } +/** + * Why a tree offers (next to) nothing to act on, as lines naming the likely + * causes, or null when it looks healthy. Shared by `snapshot` and by the + * not-found errors of `click`, `fill` and `scroll`. + */ +export function sparseTreeNote(annotated, { udid = annotated.udid, accessibilityEnabled = annotated.accessibilityEnabled } = {}) { + if (annotated.overlay) return null; + const causes = emptyTreeMessage({ udid: udid ?? '', accessibilityEnabled: accessibilityEnabled ?? null }).split('\n').slice(1); + const floating = floatingOnly(annotated); + if (floating) { + const desc = floating.slice(0, 3).map((n) => `${n.tag}${n.label ? ` ${quote(n.label)}` : ''}`).join(', '); + return [`Only a small floating element is exposed (${desc}). It is probably a separate window, such as the React Native / Expo dev-menu button, and the app's own window shows nothing to accessibility.`, ...causes]; + } + if (!annotated.coverage.interactive) { + return ['The app exposes no interactive elements. If it is still loading, snapshot again in a moment; otherwise, likely causes:', ...causes]; + } + return null; +} + function nodeLabel(node, { showIdentifiers = true } = {}) { let body; if (node.component) { @@ -235,12 +277,8 @@ export function render(annotated, { includeOffscreen = false, showIdentifiers = 'Dismiss it with one of its buttons, e.g. `click \'button[text="Not Now"]\'`, then snapshot again.', ''); } - const floating = floatingOnly(annotated); - if (floating) { - const desc = floating.slice(0, 3).map((n) => `${n.tag}${n.label ? ` ${quote(n.label)}` : ''}`).join(', '); - lines.push('[Mostly empty]', `Only a small floating element is exposed (${desc}). It is probably a separate window, such as the React Native / Expo dev-menu button, and the app's own window shows nothing to accessibility.`, - emptyTreeMessage({ udid: annotated.udid ?? '', accessibilityEnabled: annotated.accessibilityEnabled ?? null }).split('\n').slice(1).join('\n'), ''); - } + const sparse = sparseTreeNote(annotated); + if (sparse) lines.push('[Mostly empty]', ...sparse, ''); const logBox = logBoxToast(annotated); if (logBox) { @@ -278,10 +316,14 @@ export function render(annotated, { includeOffscreen = false, showIdentifiers = if (annotated.conflicts.length) { lines.push('', '[Conflicts]'); - for (const { id, names } of annotated.conflicts) { + for (const { id, names, skipped = [] } of annotated.conflicts) { const node = annotated.byId.get(id); const desc = `${node.tag}${node.label ? ` ${quote(node.label)}` : ''}`; - lines.push(`⚠ #${id} ${desc} matched by ${names.length} components (${names.join(', ')}) — only ${names[0]} applied`); + const why = !skipped.length ? '' : skipped.length === names.length ? ' (ancestor skipped)' : ` (ancestor skipped for ${skipped.join(', ')})`; + lines.push(`⚠ #${id} ${desc} matched by ${names.length} components (${names.join(', ')}) — only ${names[0]} applied${why}`); + } + if (annotated.conflicts.some((c) => c.skipped?.length)) { + lines.push(' (ancestor skipped: the selector matched only because an identifier in its chain is on no element here; give the ancestor an accessible testID or make the leaf selector more specific)'); } } if (hidden.nodes) { diff --git a/skills/subtext-ios-sim/test/bridge.test.mjs b/skills/subtext-ios-sim/test/bridge.test.mjs index bf8b993..90a3151 100644 --- a/skills/subtext-ios-sim/test/bridge.test.mjs +++ b/skills/subtext-ios-sim/test/bridge.test.mjs @@ -111,6 +111,31 @@ test('first match wins and conflicts are reported, view components before global assert.match(snap.text, /matched by 3 components \(ViewPrimary, AnyButton, Primary\) — only ViewPrimary applied/); }); +test('skipping absent ancestors never collapses sibling components onto a substring-only leaf', () => { + const skipCorpus = loadCorpusFile(join(here, 'fixtures', 'skip-conflict-corpus.json')); + const snap = snapshot(fixture('options'), skipCorpus); + for (const id of ['nav-option-events', 'nav-option-leagues']) { + assert.equal(byIdentifier(snap, id).component, 'NavOption', `${id}: the component whose whole chain matched`); + } + assert.ok(!snap.conflicts.some((c) => c.names.includes('SportOption') || c.names.includes('FilterOption'))); + + const menuAbsent = fixture('options'); + menuAbsent.root.children[0].identifier = null; + const orphaned = snapshot(menuAbsent, skipCorpus); + assert.equal(byIdentifier(orphaned, 'nav-option-events').component, null, 'a substring-only leaf needs its ancestor on screen'); +}); + +test('a full-chain match beats a skipped one, and remaining skip conflicts are flagged', () => { + const snap = snapshot(fixture('options'), loadCorpusFile(join(here, 'fixtures', 'skip-conflict-corpus.json'))); + const done = byIdentifier(snap, 'done'); + assert.equal(done.component, 'MenuDone', 'listed after OuterDone, but its whole chain matched'); + assert.ok(!snap.conflicts.some((c) => c.id === done.id), 'ranking settled it'); + const close = byIdentifier(snap, 'close'); + assert.equal(close.component, 'SheetClose'); + assert.deepEqual(snap.conflicts.find((c) => c.id === close.id), { id: close.id, names: ['SheetClose', 'DialogClose'], skipped: ['SheetClose', 'DialogClose'] }); + assert.match(snap.text, /⚠ #\d+ button "Close" matched by 2 components \(SheetClose, DialogClose\) — only SheetClose applied \(ancestor skipped\)/); +}); + test('render hides off-screen nodes and says so', () => { const snap = snapshot(fixture('eager'), corpus); assert.equal(snap.view, 'Eager'); @@ -349,4 +374,7 @@ test('descendant selectors skip ancestors whose exact identifier is absent from assert.equal(count('[testid="login-form"] button'), 0, 'a subject without its own attribute is not widened'); assert.equal(count('[testid^="login-form"] [testid="login-submit"]'), 0, 'only exact identifiers are skippable'); assert.equal(count('[testid="login-form"] > [testid="login-submit"]'), 0, 'the child combinator is never skipped'); + assert.equal(count('[testid="login-form"] [testid^="login-sub"]'), 0, 'a prefix-only subject is not reached by skipping'); + assert.equal(count('[testid="login-form"] [testid*="submit"]'), 0, 'a substring-only subject is not reached by skipping'); + assert.equal(count('[testid="login-form"] button[text="Sign In"]'), 1, 'any exact attribute on the subject allows the skip'); }); diff --git a/skills/subtext-ios-sim/test/cli.test.mjs b/skills/subtext-ios-sim/test/cli.test.mjs index f61672d..635620e 100644 --- a/skills/subtext-ios-sim/test/cli.test.mjs +++ b/skills/subtext-ios-sim/test/cli.test.mjs @@ -4,11 +4,11 @@ import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'no import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { formatResult, parseArgs, toRequest } from '../lib/cli.mjs'; +import { formatResult, formatRunEntry, parseArgs, toRequest } from '../lib/cli.mjs'; import { call, readSession } from '../lib/client.mjs'; import { runDaemon, SimSession } from '../lib/daemon.mjs'; import { HelperClient } from '../lib/helper-client.mjs'; -import { ConsoleBuffer, parseLogLine } from '../lib/logs.mjs'; +import { ConsoleBuffer, LogCollector, parseLogLine, sessionEndIn } from '../lib/logs.mjs'; import { pickDevice } from '../lib/simctl.mjs'; import { readRun } from '../lib/runlog.mjs'; @@ -27,6 +27,18 @@ test('parseArgs splits command, positionals and flags', () => { assert.throws(() => parseArgs(['snapshot', '--json=1']), { code: 'usage' }); }); +test('parseArgs takes -i as a flag and refuses a flag-looking token as a value', () => { + assert.deepEqual(parseArgs(['console', 'list', '-i', '--grep', '206']).flags, { 'ignore-case': true, grep: '206' }); + assert.deepEqual(parseArgs(['console', 'list', '--grep', '206', '--ignore-case']).flags, { grep: '206', 'ignore-case': true }); + assert.throws(() => parseArgs(['console', 'list', '--grep', '-i', '206']), (err) => err.code === 'usage' && /--grep needs a value, but got the flag -i/.test(err.message) && /--grep=-i/.test(err.hint)); + assert.throws(() => parseArgs(['console', 'list', '--grep']), { code: 'usage' }); + assert.throws(() => parseArgs(['console', 'list', '--grep', '--']), { code: 'usage' }); + assert.deepEqual(parseArgs(['console', 'list', '--grep', '--', '-i', '--json']).flags, { grep: '-i', json: true }); + assert.equal(parseArgs(['console', 'list', '--grep=-i']).flags.grep, '-i'); + assert.equal(parseArgs(['scroll', 'down', '--max', '-1']).flags.max, '-1'); + assert.deepEqual(parseArgs(['fill', 'login-email', '', '--clear']).positionals, ['login-email', '']); +}); + test('toRequest maps commands onto daemon requests', () => { const req = (argv) => toRequest(parseArgs(argv), '/proj'); assert.deepEqual(req(['click', 'SignInButton']), { cmd: 'click', args: { target: 'SignInButton' } }); @@ -110,9 +122,10 @@ test('daemon serves the protocol over its socket', async () => { process.env.FAKE_HELPER_LOG = inputLog; const simctl = fakeSimctl(); let stopped = false; - const { record, stop } = await runDaemon({ + const { record, stop, session: daemonSession } = await runDaemon({ udid: 'FAKE-UDID', deviceName: 'Fake Phone', bundleId: 'dev.example.app', processName: 'App', sightmapDir, - helperPath: process.execPath, onStopped: () => { stopped = true; }, timing: { settleMs: 0, scrollSettleMs: 0, readyTimeoutMs: 1000 }, + helperPath: process.execPath, onStopped: () => { stopped = true; }, + timing: { settleMs: 0, scrollSettleMs: 0, readyTimeoutMs: 1000, retryIntervalMs: 0, retryTimeoutMs: 100 }, }, { simctl, helperArgs: [FAKE_HELPER], withLogs: false, log: () => {} }); stopDaemon = stop; const inputs = () => readFileSync(inputLog, 'utf8').split('\n').filter(Boolean).map((l) => JSON.parse(l)); @@ -131,18 +144,30 @@ test('daemon serves the protocol over its socket', async () => { assert.equal(snap.tree, undefined); assert.ok((await call(sock, 'snapshot', { json: true })).tree.children.length); - assert.deepEqual(await call(sock, 'click', { target: 'go' }), { target: '2 button[identifier="go"] "Go"', x: 60, y: 30 }); + assert.deepEqual(await call(sock, 'click', { target: 'go' }), { target: '2 button[identifier="go"] "Go"', x: 60, y: 30, retries: 0 }); assert.deepEqual((await call(sock, 'click', { target: '2' })).y, 30); assert.deepEqual((await call(sock, 'click', { target: 'button[text="Go"]' })).x, 60); await assert.rejects(call(sock, 'click', { target: 'far' }), (err) => err.code === 'offscreen' && /scroll down --until/.test(err.hint)); await assert.rejects(call(sock, 'click', { target: 'under' }), { code: 'covered' }); await assert.rejects(call(sock, 'click', { target: 'nope' }), { code: 'disabled' }); - await assert.rejects(call(sock, 'click', { target: 'Missing' }), { code: 'not_found' }); + await assert.rejects(call(sock, 'click', { target: 'Missing' }), (err) => err.code === 'not_found' && /long lists/.test(err.message)); assert.equal(inputs().filter((i) => i.cmd === 'tap').length, 3); + await daemonSession.helper.request('arm'); + const late = await call(sock, 'click', { target: 'late' }); + assert.ok(late.retries >= 2, `a button that appears a moment later is tapped after re-reads (${late.retries})`); + await daemonSession.helper.request('arm'); + const mover = await call(sock, 'click', { target: 'mover' }); + assert.equal(mover.y, 320, 'a sliding button is tapped where it stops'); + assert.ok(mover.retries >= 1); + assert.match(formatResult('click', mover), /after \d+ re-read\(s\) while the screen settled/); + const filled = await call(sock, 'fill', { target: 'email', text: 'a@b.co' }); assert.equal(filled.value, 'a@b.co'); assert.equal((await call(sock, 'fill', { target: 'email', text: 'x@y.z', clear: true })).value, 'x@y.z'); + const cleared = await call(sock, 'fill', { target: 'email', text: '', clear: true }); + assert.deepEqual([cleared.value, cleared.placeholder], ['', 'Email'], 'a cleared field showing its placeholder is empty'); + assert.match(formatResult('fill', cleared), /value "" \(empty; the field shows its placeholder "Email"\)/); await call(sock, 'scroll', { direction: 'down' }); assert.deepEqual(inputs().at(-1), { cmd: "swipe", args: { udid: "FAKE-UDID", x1: 200, y1: 640, x2: 200, y2: 160, duration: 1.6 } }); @@ -155,6 +180,7 @@ test('daemon serves the protocol over its socket', async () => { await call(sock, 'openurl', { url: 'app://x' }); assert.deepEqual(simctl.calls.at(-1), ['openUrl', 'FAKE-UDID', 'app://x']); await assert.rejects(call(sock, 'sessionUrl'), { code: 'no_session_url' }); + await checkSessionEnd(sock, daemonSession); await assert.rejects(call(sock, 'bogus'), { code: 'unknown_cmd' }); const status = await call(sock, 'status'); @@ -190,10 +216,58 @@ test('daemon serves the protocol over its socket', async () => { assert.deepEqual(simctl.calls.slice(-2).map((c) => c[0]), ['terminate', 'shutdown']); }); +const URL_1 = 'https://app.fullstory.com/ui/o-TEST01-na1/session/1111111111111111111%3A2222222222222222222'; +const URL_2 = 'https://app.fullstory.com/ui/o-TEST01-na1/session/1111111111111111111%3A3333333333333333333'; +const sdkLine = (eventMessage, messageType = 'Default') => JSON.stringify({ + eventType: 'logEvent', messageType, subsystem: 'com.fullstory.FullStory', category: 'FullStory', processID: 4242, timestamp: '2026-10-28 14:02:11.123456-0700', eventMessage, +}); + +test('Fullstory SDK termination lines are recognized', () => { + assert.equal(sessionEndIn('Received HTTP Status Code: 206, terminating FullStory session').status, 206); + assert.match(sessionEndIn('[FullStory] Received HTTP Status Code: 206, terminating FullStory session').reason, /too large/); + assert.match(sessionEndIn('FullStory: Received HTTP Status Code: 202, terminating FullStory session').reason, /recording is blocked/); + assert.equal(sessionEndIn('capture is terminated; not resuming session').status, null); + assert.deepEqual(sessionEndIn('WARNING: Maximum number of views reached'), { warning: 'the Fullstory SDK logged "Maximum number of views reached"' }); + assert.equal(sessionEndIn('Received HTTP Status Code: 200'), null); + assert.equal(sessionEndIn('POST /orders 206'), null); +}); + +/** Feeds `log stream` lines through the daemon's log ingestion and checks the run log, `session-url` and `status`. */ +async function checkSessionEnd(sock, session) { + const logs = new LogCollector({ udid: 'FAKE-UDID', processName: 'App', buffer: session.buffer }); + const runEntries = async () => readRun((await call(sock, 'runCurrent')).path); + logs.ingest(sdkLine(`FULLSTORY_SESSION_URL ${URL_1}`, 'Info')); + assert.deepEqual(await call(sock, 'sessionUrl', {}), { url: URL_1, ended: false }); + logs.ingest(sdkLine('Maximum number of views reached; some views will not be captured', 'Error')); + logs.ingest(sdkLine('Received HTTP Status Code: 206, terminating FullStory session', 'Error')); + logs.ingest(sdkLine('capture is terminated; not resuming session')); + + const ended = (await runEntries()).filter((e) => e.action === 'session-ended'); + assert.equal(ended.length, 1, 'one session-ended entry per session'); + assert.equal(ended[0].url, URL_1); + assert.equal(ended[0].status, 206); + assert.match(ended[0].reason, /HTTP 206.*too large.*Maximum number of views reached/); + assert.ok(ended[0].ts); + assert.match(formatRunEntry(ended[0]), /session-ended https:\/\/app\.fullstory\.com\/.* HTTP 206/); + + await assert.rejects(call(sock, 'sessionUrl', {}), (err) => err.code === 'session_ended' && err.message.includes(URL_1) && /relaunch/.test(err.hint)); + const asJson = await call(sock, 'sessionUrl', { json: true }); + assert.deepEqual([asJson.url, asJson.ended], [URL_1, true]); + assert.match(asJson.reason, /206/); + assert.match((await call(sock, 'status')).sessionEnded.reason, /206/); + + logs.ingest(sdkLine(`FULLSTORY_SESSION_URL ${URL_2}`, 'Info')); + assert.deepEqual(await call(sock, 'sessionUrl', {}), { url: URL_2, ended: false }); + assert.equal((await call(sock, 'status')).sessionEnded, null); + const sessions = (await runEntries()).filter((e) => e.action === 'session' || e.action === 'session-ended').map((e) => `${e.action} ${e.url}`); + assert.deepEqual(sessions, [`session ${URL_1}`, `session-ended ${URL_1}`, `session ${URL_2}`]); +} + function stubSession({ describe, accessibilityEnabled }) { const helper = { request: async () => describe(), close() {}, alive: true, restarts: 0 }; const simctl = accessibilityEnabled === undefined ? {} : { accessibilityEnabled: async () => accessibilityEnabled }; - return new SimSession({ udid: 'U-1', bundleId: 'dev.x', processName: 'X', sightmapDir: join(dir, 'stub', '.sightmap'), helper, simctl, timing: { overlaySettleMs: 0 } }); + return new SimSession({ udid: 'U-1', bundleId: 'dev.x', processName: 'X', sightmapDir: join(dir, 'stub', '.sightmap'), helper, simctl, + timing: { overlaySettleMs: 0, settleMs: 0, scrollSettleMs: 0, retryIntervalMs: 0, retryTimeoutMs: 20 } }); } const axDisabled = () => { throw Object.assign(new Error('no elements'), { code: 'ax_disabled' }); }; @@ -228,3 +302,29 @@ test('a snapshot showing only a small floating element says the app window is hi assert.match(snap.text, /no-hide-descendants/); assert.doesNotMatch(snap.text, /defaults write/); }); + +test('click, fill and scroll on a mostly empty tree give the empty-tree diagnosis, not the long-list hint', async () => { + const el = (type, frame, extra = {}) => ({ role: `AX${type}`, type, label: null, identifier: null, value: null, frame, enabled: true, children: [], ...extra }); + const floating = () => ({ + screen: { w: 390, h: 844 }, + root: el('Application', { x: 0, y: 0, w: 390, h: 844 }, { label: 'App', children: [el('Button', { x: 330, y: 700, w: 48, h: 48 }, { label: 'Dev menu' })] }), + }); + const sparse = (err) => { + assert.equal(err.code, 'not_found'); + assert.match(err.message, /exposes next to nothing to accessibility/); + assert.match(err.message, /Only a small floating element is exposed \(button "Dev menu"\)/); + assert.match(err.message, /no-hide-descendants/); + assert.doesNotMatch(err.message, /defaults write/); + assert.doesNotMatch(`${err.message} ${err.hint}`, /long lists|further down a list, run/); + return true; + }; + const session = stubSession({ describe: floating, accessibilityEnabled: true }); + await assert.rejects(session.handle('click', { target: 'profile-avatar' }), sparse); + await assert.rejects(session.handle('fill', { target: 'login-email', text: 'a' }), sparse); + await assert.rejects(session.handle('scroll', { direction: 'down', until: 'login-email', max: 3 }), sparse); + + const loading = () => ({ screen: { w: 390, h: 844 }, root: el('Application', { x: 0, y: 0, w: 390, h: 844 }, { label: 'App', children: [ + el('StaticText', { x: 0, y: 100, w: 390, h: 600 }, { label: 'Loading…' }), el('StaticText', { x: 0, y: 700, w: 390, h: 40 }, { label: 'v1' })] }) }); + await assert.rejects(stubSession({ describe: loading, accessibilityEnabled: false }).handle('click', { target: 'go' }), + (err) => /no interactive elements/.test(err.message) && /defaults write/.test(err.message)); +}); diff --git a/skills/subtext-ios-sim/test/fakes/fake-helper.mjs b/skills/subtext-ios-sim/test/fakes/fake-helper.mjs index b66a564..eff554c 100644 --- a/skills/subtext-ios-sim/test/fakes/fake-helper.mjs +++ b/skills/subtext-ios-sim/test/fakes/fake-helper.mjs @@ -4,6 +4,10 @@ import { appendFileSync } from 'node:fs'; import { createInterface } from 'node:readline'; let email = ''; +let describes = 0; +let armedAt = null; +// After `arm`, a `late` button appears on the third describe and a `mover` button slides for three describes, like a sheet closing. +const settling = () => (armedAt === null ? 0 : Math.max(0, armedAt + 3 - describes)); const node = (type, frame, extra = {}) => ({ role: `AX${type}`, type, subrole: null, traits: [], label: null, identifier: null, value: null, frame, enabled: true, children: [], ...extra }); const tree = () => ({ screen: { w: 400, h: 800 }, @@ -16,6 +20,8 @@ const tree = () => ({ node('Button', { x: 10, y: 1200, w: 100, h: 40 }, { label: 'Far', identifier: 'far' }), node('Group', { x: 0, y: 740, w: 400, h: 60 }, { label: 'Tab Bar' }), node('Button', { x: 10, y: 750, w: 100, h: 40 }, { label: 'Under', identifier: 'under' }), + ...(armedAt !== null && !settling() ? [node('Button', { x: 200, y: 10, w: 100, h: 40 }, { label: 'Late', identifier: 'late' })] : []), + ...(armedAt !== null ? [node('Button', { x: 200, y: 300 + settling() * 40, w: 100, h: 40 }, { label: 'Mover', identifier: 'mover' })] : []), ], }), }); @@ -24,10 +30,11 @@ createInterface({ input: process.stdin }).on('line', (line) => { const { id, cmd, args } = JSON.parse(line); const reply = (msg) => process.stdout.write(`${JSON.stringify({ id, ...msg })}\n`); if (cmd === 'crash') process.exit(3); - if (process.env.FAKE_HELPER_LOG && cmd !== 'hello' && cmd !== 'describe') appendFileSync(process.env.FAKE_HELPER_LOG, `${JSON.stringify({ cmd, args })}\n`); + if (process.env.FAKE_HELPER_LOG && !['hello', 'describe', 'arm'].includes(cmd)) appendFileSync(process.env.FAKE_HELPER_LOG, `${JSON.stringify({ cmd, args })}\n`); switch (cmd) { case 'hello': return reply({ ok: true, result: { version: 'fake' } }); - case 'describe': return reply({ ok: true, result: tree() }); + case 'describe': describes++; return reply({ ok: true, result: tree() }); + case 'arm': armedAt = describes; return reply({ ok: true, result: {} }); case 'type': email += args.text; return reply({ ok: true, result: {} }); case 'key': if (args.key === 'delete' || args.key === 'forward_delete') email = email.slice(0, -1); return reply({ ok: true, result: {} }); case 'tap': case 'swipe': case 'button': return reply({ ok: true, result: {} }); diff --git a/skills/subtext-ios-sim/test/fixtures/describe-options.json b/skills/subtext-ios-sim/test/fixtures/describe-options.json new file mode 100644 index 0000000..e326501 --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-options.json @@ -0,0 +1,118 @@ +{ + "screen": { + "w": 402, + "h": 874 + }, + "root": { + "role": "AXApplication", + "subrole": null, + "type": "Application", + "label": "Sample", + "identifier": null, + "value": null, + "traits": [], + "frame": { + "x": 0, + "y": 0, + "w": 402, + "h": 874 + }, + "enabled": true, + "children": [ + { + "role": "AXScrollArea", + "subrole": null, + "type": "ScrollArea", + "label": null, + "identifier": "nav-menu", + "value": null, + "traits": [], + "frame": { + "x": 0, + "y": 100, + "w": 402, + "h": 300 + }, + "enabled": true, + "children": [ + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Events", + "identifier": "nav-option-events", + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 110, + "w": 370, + "h": 44 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Leagues", + "identifier": "nav-option-leagues", + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 160, + "w": 370, + "h": 44 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Done", + "identifier": "done", + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 210, + "w": 370, + "h": 44 + }, + "enabled": true, + "children": [] + } + ] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Close", + "identifier": "close", + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 500, + "w": 100, + "h": 44 + }, + "enabled": true, + "children": [] + } + ] + } +} diff --git a/skills/subtext-ios-sim/test/fixtures/describe-rn-keyboard.json b/skills/subtext-ios-sim/test/fixtures/describe-rn-keyboard.json new file mode 100644 index 0000000..155b8ef --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/describe-rn-keyboard.json @@ -0,0 +1,422 @@ +{ + "screen": { + "w": 402, + "h": 874 + }, + "root": { + "role": "AXApplication", + "subrole": null, + "type": "Application", + "label": "Subtext RN Sample", + "identifier": null, + "value": null, + "traits": [ + "None" + ], + "frame": { + "x": 0, + "y": 0, + "w": 402, + "h": 874 + }, + "enabled": true, + "children": [ + { + "role": "AXHeading", + "subrole": null, + "type": "Heading", + "label": "Events", + "identifier": null, + "value": null, + "traits": [ + "Header" + ], + "frame": { + "x": 150, + "y": 74, + "w": 102, + "h": 21 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXGenericElement", + "subrole": null, + "type": "GenericElement", + "label": "Spring Meetup, 64 members", + "identifier": null, + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 116, + "w": 370, + "h": 120 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXGenericElement", + "subrole": null, + "type": "GenericElement", + "label": "Summer Meetup, 32 members", + "identifier": null, + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 248, + "w": 370, + "h": 120 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXGenericElement", + "subrole": null, + "type": "GenericElement", + "label": "Season ends in 3 days", + "identifier": null, + "value": null, + "traits": [], + "frame": { + "x": 16, + "y": 380, + "w": 370, + "h": 40 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXGenericElement", + "subrole": null, + "type": "GenericElement", + "label": null, + "identifier": null, + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 16, + "y": 430, + "w": 370, + "h": 40 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXTextField", + "subrole": null, + "type": "TextField", + "label": null, + "identifier": "search", + "value": "Search events", + "traits": [ + "TextEntry" + ], + "frame": { + "x": 16, + "y": 480, + "w": 370, + "h": 44 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "Join", + "identifier": "join-button", + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 300, + "y": 760, + "w": 90, + "h": 44 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKeyboard", + "subrole": null, + "type": "Keyboard", + "label": null, + "identifier": null, + "value": null, + "traits": [], + "frame": { + "x": 0, + "y": 590, + "w": 402, + "h": 284 + }, + "enabled": true, + "children": [ + { + "role": "AXGroup", + "subrole": null, + "type": "Group", + "label": null, + "identifier": null, + "value": null, + "traits": [], + "frame": { + "x": 0, + "y": 590, + "w": 402, + "h": 230 + }, + "enabled": true, + "children": [ + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "q", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 3, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "w", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 42.6, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "e", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 82.2, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "r", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 121.80000000000001, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "t", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 161.4, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "y", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 201, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "u", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 240.60000000000002, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "i", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 280.2, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "o", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 319.8, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + }, + { + "role": "AXKey", + "subrole": null, + "type": "Key", + "label": "p", + "identifier": null, + "value": null, + "traits": [ + "KeyboardKey" + ], + "frame": { + "x": 359.40000000000003, + "y": 600, + "w": 36, + "h": 46 + }, + "enabled": true, + "children": [] + } + ] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": "return", + "identifier": null, + "value": null, + "traits": [ + "Button", + "KeyboardKey" + ], + "frame": { + "x": 300, + "y": 820, + "w": 98, + "h": 46 + }, + "enabled": true, + "children": [] + } + ] + }, + { + "role": "AXButton", + "subrole": null, + "type": "Button", + "label": null, + "identifier": null, + "value": null, + "traits": [ + "Button" + ], + "frame": { + "x": 324, + "y": 784, + "w": 60, + "h": 60 + }, + "enabled": true, + "children": [] + } + ] + } +} diff --git a/skills/subtext-ios-sim/test/fixtures/skip-conflict-corpus.json b/skills/subtext-ios-sim/test/fixtures/skip-conflict-corpus.json new file mode 100644 index 0000000..c65568b --- /dev/null +++ b/skills/subtext-ios-sim/test/fixtures/skip-conflict-corpus.json @@ -0,0 +1,48 @@ +{ + "memory": [], + "views": [], + "globals": [ + { + "name": "SportOption", + "selectors": [ + "[testid=\"sport-picker\"] [testid*=\"-option-\"]" + ] + }, + { + "name": "FilterOption", + "selectors": [ + "[testid=\"filter-sheet\"] [testid*=\"-option-\"]" + ] + }, + { + "name": "NavOption", + "selectors": [ + "[testid=\"nav-menu\"] [testid*=\"-option-\"]" + ] + }, + { + "name": "SheetClose", + "selectors": [ + "[testid=\"sheet-a\"] [testid=\"close\"]" + ] + }, + { + "name": "DialogClose", + "selectors": [ + "[testid=\"dialog-b\"] [testid=\"close\"]" + ] + }, + { + "name": "OuterDone", + "selectors": [ + "[testid=\"outer-wrapper\"] [testid=\"done\"]" + ] + }, + { + "name": "MenuDone", + "selectors": [ + "[testid=\"nav-menu\"] [testid=\"done\"]" + ] + } + ] +} diff --git a/skills/subtext-ios-sim/test/rn.test.mjs b/skills/subtext-ios-sim/test/rn.test.mjs index 39979ce..f6a5d56 100644 --- a/skills/subtext-ios-sim/test/rn.test.mjs +++ b/skills/subtext-ios-sim/test/rn.test.mjs @@ -11,6 +11,7 @@ import { ConsoleBuffer, parseLogLine, sessionUrlIn } from '../lib/logs.mjs'; const here = dirname(fileURLToPath(import.meta.url)); const describeRN = () => JSON.parse(readFileSync(join(here, 'fixtures', 'describe-rn.json'), 'utf8')); +const describeKeyboard = () => JSON.parse(readFileSync(join(here, 'fixtures', 'describe-rn-keyboard.json'), 'utf8')); const corpus = parseCorpus({ globals: [{ name: 'TabBarItem', selectors: ['tab'], properties: [{ name: 'title', extract: 'text' }] }], @@ -106,3 +107,70 @@ test('the newest session URL wins, from any console source or level', () => { assert.equal(buf.sessionUrl, URL_EU); assert.deepEqual(seen, [URL_NA, URL_EU]); }); + +test('labelled or identified `other` elements with a tappable trait count as interactive', () => { + const { nodes } = normalize(describeKeyboard()); + const byLabel = (label) => nodes.find((n) => n.label === label); + assert.equal(byLabel('Spring Meetup, 64 members').tag, 'other'); + assert.equal(byLabel('Spring Meetup, 64 members').interactive, true, 'Button trait and a label'); + assert.equal(byLabel('Season ends in 3 days').interactive, false, 'no tappable trait'); + assert.ok(!nodes.some((n) => n.tag === 'other' && !n.label && !n.identifier && n.interactive), 'no label or identifier'); +}); + +test('system keyboard keys are left out of coverage and component matching', () => { + const keyCorpus = parseCorpus({ + globals: [{ name: 'AnyButton', selectors: ['button'] }, { name: 'Key', selectors: ['key'] }, { name: 'Card', selectors: ['other[text*="members"]'] }], + }); + const snap = annotate(normalize(describeKeyboard()), keyCorpus); + const keys = snap.nodes.filter((n) => n.tag === 'key'); + assert.equal(keys.length, 10); + assert.ok(keys.every((k) => k.system && k.component === null)); + const ret = snap.nodes.find((n) => n.label === 'return'); + assert.equal(ret.system, true); + assert.equal(ret.component, null, 'a button inside the keyboard is not an app component'); + assert.equal(snap.nodes.find((n) => n.identifier === 'join-button').component, 'AnyButton'); + assert.deepEqual(snap.coverage, { interactive: 5, t1: 4, t2: 0, t3: 1 }); + assert.equal(snap.guide.get('Card'), 2); +}); + +function withoutKeyboard() { + const d = describeKeyboard(); + d.root.children = d.root.children.filter((c) => c.type !== 'Keyboard'); + const join = d.root.children.find((c) => c.identifier === 'join-button'); + join.frame = { x: 240, y: 760, w: 150, h: 44 }; + return d; +} + +test('a tap within 24 pt of a floating dev-menu button is refused with a spot to tap instead', () => { + const snap = annotate(normalize(withoutKeyboard()), corpus); + const join = byIdentifier(snap, 'join-button'); + assert.throws(() => assertActionable(snap, join, 'join-button'), (err) => { + assert.equal(err.code, 'covered'); + assert.match(err.message, /within 24 pt of \d+ button, which looks like the React Native \/ Expo dev-menu button/); + const [, x, y] = /tap-xy (\d+) (\d+)/.exec(err.hint); + assert.ok(Number(x) < 300 && Number(x) >= 240, `suggested x ${x} is on the target and clear of the button`); + assert.ok(Number(y) >= 760 && Number(y) <= 804); + assert.match(err.hint, /Show floating action button/); + return true; + }); + const fab = snap.nodes.find((n) => n.tag === 'button' && !n.label); + assert.doesNotThrow(() => assertActionable(snap, fab, String(fab.id)), 'the floating button itself can be tapped'); + assert.doesNotThrow(() => assertActionable(snap, snap.nodes.find((n) => n.label === 'Spring Meetup, 64 members'), 'card')); +}); + +test('an element labelled like the dev menu covers wherever it is in the tree', () => { + const d = withoutKeyboard(); + d.root.children.pop(); + d.root.children.unshift({ type: 'Button', role: 'AXButton', label: 'Open dev menu', frame: { x: 330, y: 70, w: 120, h: 30 }, traits: ['Button'], children: [] }); + d.root.children.push({ type: 'Image', role: 'AXImage', label: 'Avatar', identifier: 'profile-avatar', frame: { x: 340, y: 100, w: 40, h: 40 }, traits: ['Button'], children: [] }); + const snap = annotate(normalize(d), corpus); + assert.throws(() => assertActionable(snap, byIdentifier(snap, 'profile-avatar'), 'profile-avatar'), { code: 'covered' }); + assert.doesNotThrow(() => assertActionable(snap, byIdentifier(snap, 'join-button'), 'join-button'), 'far from it'); +}); + +test('a small last element with an identifier is the app\'s own, not a floating window', () => { + const d = withoutKeyboard(); + d.root.children.at(-1).identifier = 'chat-fab'; + const snap = annotate(normalize(d), corpus); + assert.doesNotThrow(() => assertActionable(snap, byIdentifier(snap, 'join-button'), 'join-button')); +}); diff --git a/skills/subtext-ios-sim/test/sightmap.test.mjs b/skills/subtext-ios-sim/test/sightmap.test.mjs index 2d424f7..71a64f5 100644 --- a/skills/subtext-ios-sim/test/sightmap.test.mjs +++ b/skills/subtext-ios-sim/test/sightmap.test.mjs @@ -5,7 +5,8 @@ import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { compareVersions, SIGHTMAP_EXTRACT_OBJECT, sightmapVersion, sightmapVersionWarning } from '../lib/sightmap-cli.mjs'; +import { compareVersions, SIGHTMAP_EXTRACT_OBJECT, SIGHTMAP_VERSION, sightmapVersion, sightmapVersionWarning } from '../lib/sightmap-cli.mjs'; +import { sightmapDetail } from '../lib/doctor.mjs'; const skill = join(dirname(fileURLToPath(import.meta.url)), '..'); const bin = process.env.SIGHTMAP_BIN || 'sightmap'; @@ -28,6 +29,13 @@ test('sightmap versions outside the tested range warn', () => { assert.match(sightmapVersionWarning('unknown'), /could not read/); }); +test('doctor says which sightmap is used and what the pinned fallback is', () => { + assert.equal(sightmapDetail({ version: '0.36.0', source: 'on PATH: /opt/bin/sightmap' }), `using PATH sightmap 0.36.0 (/opt/bin/sightmap; pinned fallback is ${SIGHTMAP_VERSION})`); + assert.equal(sightmapDetail({ version: '0.35.2', source: 'SIGHTMAP_BIN=/x/sightmap' }), `using SIGHTMAP_BIN sightmap 0.35.2 (/x/sightmap; pinned fallback is ${SIGHTMAP_VERSION})`); + assert.match(sightmapDetail({ version: SIGHTMAP_VERSION, source: `npx @sightmap/sightmap@${SIGHTMAP_VERSION}` }), /^using the pinned sightmap/); + assert.equal(sightmapVersionWarning('0.36.0'), null, 'a PATH sightmap inside the range is not a warning'); +}); + for (const app of ['SampleApp', 'RNSampleApp']) { test(`fixtures/${app}/.sightmap passes sightmap validate`, canValidate, () => { const { ok, out } = validate(join(skill, 'fixtures', app, '.sightmap')); From a3e5164f410b454ea2a80299a4d19c9c87c925d0 Mon Sep 17 00:00:00 2001 From: Chandler Van De Water Date: Thu, 29 Oct 2026 09:04:11 -0400 Subject: [PATCH 12/12] docs: mark retest fixes in subtext-ios-sim field notes Co-Authored-By: Claude Opus 5.5 Signed-off-by: Chandler Van De Water --- docs/ios-sim-loop/FIELD-NOTES.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/ios-sim-loop/FIELD-NOTES.md b/docs/ios-sim-loop/FIELD-NOTES.md index 16f8d96..db29bc8 100644 --- a/docs/ios-sim-loop/FIELD-NOTES.md +++ b/docs/ios-sim-loop/FIELD-NOTES.md @@ -8,14 +8,14 @@ Status values: **fixed** (commit noted), **in progress**, **open**, **upstream** | Found | Where | Issue | Status | |---|---|---|---| -| 2026-10-28 | field Mac (retest, 7fa7aef) | The Fullstory session ended mid-run (SDK: `Received HTTP Status Code: 206, terminating FullStory session`), and nothing in subtext-sim noticed: `session-url` and `status` kept returning the dead URL. | open: detect session end, log `session-ended`, report it from `session-url`/`status`, make relaunch the producer's standard response | -| 2026-10-28 | field Mac (retest) | The selector skip rule merged three components onto the same `[testid*="-option-"]` match, and the wrong one won. | open: don't skip into substring-only selectors, or flag skip-induced conflicts | -| 2026-10-28 | field Mac (retest) | `click MobileTopBarAvatar` reported success but opened the Expo dev menu: the dev-menu window's touch area is larger than its accessibility frame. | open: treat dev-menu windows as covering; document hiding the floating button | -| 2026-10-28 | field Mac (retest) | The empty-tree diagnosis was only fixed in `snapshot`; `click`, `fill` and `scroll` still give the "long lists" advice. | open | -| 2026-10-28 | field Mac (retest) | Coverage misses labelled, tappable `other` elements (e.g. tournament cards) and counts keyboard keys while the keyboard is up. | open | -| 2026-10-28 | field Mac (retest) | `console list --grep -i 206` took `-i` as the pattern; `fill "" --clear` reported the placeholder as the value; clicks right after a sheet closes or a back navigation fail until retried. | open | -| 2026-10-28 | field Mac (retest) | `doctor` used sightmap 0.36.0 from PATH rather than the pinned 0.36.1. | open: decide whether PATH should win | -| 2026-10-28 | field Mac (retest) | Metro in a nested git worktree ignores edits: Watchman roots at the parent checkout, and `--clear` doesn't help; restarting Metro does. | open: skill note | +| 2026-10-28 | field Mac (retest, 7fa7aef) | The Fullstory session ended mid-run (SDK: `Received HTTP Status Code: 206, terminating FullStory session`), and nothing in subtext-sim noticed: `session-url` and `status` kept returning the dead URL. | fixed 701aa25: `session-ended` run-log entry; `session-url`/`status` report it; the skill has the producer relaunch. Matched on the SDK's log text, which hasn't been checked against a real 206 run yet | +| 2026-10-28 | field Mac (retest) | The selector skip rule merged three components onto the same `[testid*="-option-"]` match, and the wrong one won. | fixed 701aa25: skipping needs an exact leaf; full-chain matches win; leftover conflicts labelled `(ancestor skipped)`. Cost: `[testid="list"] [testid^="row-"]` no longer matches when the list isn't on screen | +| 2026-10-28 | field Mac (retest) | `click MobileTopBarAvatar` reported success but opened the Expo dev menu: the dev-menu window's touch area is larger than its accessibility frame. | fixed 701aa25: 24 pt cover zone around a dev-menu-like floating element (heuristic; the helper doesn't report windows; untested on a real Expo tree) | +| 2026-10-28 | field Mac (retest) | The empty-tree diagnosis was only fixed in `snapshot`; `click`, `fill` and `scroll` still give the "long lists" advice. | fixed 701aa25: one shared diagnosis | +| 2026-10-28 | field Mac (retest) | Coverage misses labelled, tappable `other` elements (e.g. cards) and counts keyboard keys while the keyboard is up. | fixed 701aa25 | +| 2026-10-28 | field Mac (retest) | `console list --grep -i 206` took `-i` as the pattern; `fill "" --clear` reported the placeholder as the value; clicks right after a sheet closes or a back navigation fail until retried. | fixed 701aa25: `-i` flag; placeholder vs value; click/fill retry for up to 1.5 s | +| 2026-10-28 | field Mac (retest) | `doctor` used sightmap 0.36.0 from PATH rather than the pinned 0.36.1. | fixed 701aa25: PATH still wins; doctor says so and names the pinned fallback | +| 2026-10-28 | field Mac (retest) | Metro in a nested git worktree ignores edits: Watchman roots at the parent checkout, and `--clear` doesn't help; restarting Metro does. | fixed 701aa25: skill note | | 2026-10-28 | field Mac | Empty accessibility tree while app accessibility was already on. The cause was an always-mounted view with `accessibilityViewIsModal = YES` hiding its siblings, but the error said to enable accessibility. | fixed 61a37a8: errors list modal/hidden-root views first and skip the accessibility advice when it's on; a `doctor --app` probe is still open | | 2026-10-28 | field Mac | `snapshot` crashes with `extract.startsWith is not a function`: sightmap 0.36 emits `extract` as an object, and the bridge only handled strings. | fixed 61a37a8: both forms accepted; npx pin is now sightmap 0.36.1; `doctor` reports the resolved version | | 2026-10-28 | field Mac | `session-url` missed the URL when React Native logged it as a multi-argument line. | fixed 61a37a8: any Fullstory session URL in any console entry, newest wins | @@ -47,7 +47,7 @@ Status values: **fixed** (commit noted), **in progress**, **open**, **upstream** | 2026-10-28 | field Mac | `review-zoom` at `detail` returned response headers only: no request headers, bodies or query strings. Check against the org's header allowlists and body rules before calling it a bug. | upstream / open | | 2026-10-28 | field Mac | `review-zoom` at `standard` collapses network traffic into counts; endpoint names only show at `machine`. | fixed 61a37a8: skill recommends `machine` for naming endpoints | | 2026-10-28 | field Mac | Network data lagged 35 s to 2 min behind the app, longer than the ~20 s upload interval. | upstream / open | -| 2026-10-28 | dev Mac | `review-open` reports "invalid session_url" when a session's IDs don't exist server-side, which reads like a bad link rather than "this session never arrived". | upstream: clearer error | +| 2026-10-28 | dev Mac | `review-open` reports "invalid session_url" when a session's IDs don't exist server-side, which reads like a bad link rather than "this session never arrived". | upstream: clearer error; skill explains it (701aa25) | ## Setup friction