Skip to content

Add subtext-device skill: Subtext layer for sightmap device - #40

Open
chandlervdw wants to merge 9 commits into
mainfrom
chandler/subtext-device
Open

chandlervdw wants to merge 9 commits into
mainfrom
chandler/subtext-device

Conversation

@chandlervdw

@chandlervdw chandlervdw commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

What

Adds skills/subtext-device/, the Subtext layer for driving a native iOS or React Native app in the local iOS Simulator. The Simulator tool itself lives upstream and stays vendor-neutral: sightmap device (sightmap/sightmap#522) and sightmap device network (sightmap/sightmap#523). This skill covers only what Subtext adds on top.

  • Ask before recording (default: local only). Before a run, the agent asks whether it should record a session to Fullstory. Unless the user says yes, it installs a copy of the built app with RecordOnStart = false and ServerUrl pointed at an unreachable local address, re-signed with its entitlements kept. It confirms no session started, then works from local capture alone. The user's project is never edited, and the question is asked every run.
  • Two network sources, used together.
    • The Fullstory session (review-*) gives attribution: which tap and screen caused each request.
    • sightmap device network gives detail: full URL, headers and bodies, live and free.
    • The consumer joins them by time, method and URL.
    • Fallback order:
      1. Session plus local HAR capture.
      2. The session alone, e.g. a Release build.
      3. Local capture alone: a local-only run, no SDK, a 206 cut-off, or the credit budget spent.
      4. The summary source only, where requests: can't be authored and the report says so.
  • How Fullstory data reaches agents:
    • in-app capture and ~20 s bundle uploads;
    • point-in-time review-open reads at 10 credits each;
    • review-zoom grains: machine for endpoint names, detail for headers.
  • Requirements:
    • an app built by Xcode, and a Debug build (get-task-allow) for full local capture;
    • mobile network capture on, and the MCP on the same org;
    • a separate dev org for debug builds.
  • Sessions: finding the session URL with sightmap device console list --grep fullstory.com/ui/ (newest wins), and detecting session end from the SDK's termination lines (206 / 202 / capture is terminated). Both are recorded in the run log with run note --kind session / --kind session-ended.
  • The concurrent producer/consumer workflow (in references/producer-consumer.md):
    • the consumer follows network list --since/--follow continuously and spends review-open credits only on attribution;
    • review-open → review-zoom → review-close run in sequence;
    • feedback goes through .sightmap/.device-runs/<run>.consumer.md;
    • ownership of .sightmap/ sections is split, with a credit budget and ~30 s pacing.
  • Privacy: local capture output stays redacted. Bodies are used only to find field paths for properties:, and captured values never go into .sightmap/, the run log or reports.
  • Corpus guidance: wrapper testIDs don't reach iOS accessibility, so device corpora use leaf selectors.

Markdown only. Also included:

  • a routing row in subtext-using-subtext;
  • mentions in the README and subtext-setup-plugin;
  • docs/subtext-device/: a short PLAN.md describing the split, and FIELD-NOTES.md with the prototype history, the acceptance run, the network-source ranking and the local-only verification;
  • a minor changeset.

Supersedes #39, which shipped its own prototype tool (subtext-sim) inside the skill.

Merge blocker

This PR does not bump the @sightmap/sightmap pin, run npm run vendor-skills, or hand-write skills/sightmap-device/. Those are generated and need a sightmap release that includes sightmap device and sightmap device network. Until then subtext-device names a prerequisite skill that isn't bundled.

Dependencies

Upstream gaps

Resolved in #522 since this PR opened:

  • console list --since / --follow, and reporting of entries dropped from the 5,000-entry buffer;
  • run note for session markers;
  • relaunches logged as relaunch.

By design, sightmap device has no session-url command: finding the Fullstory session is Subtext's job, done with a console grep.

Testing and field history

  • Prototype (closed Add subtext-ios-sim: local iOS Simulator loop (preview) #39): built and tested on a development Mac against a SwiftUI sample, then field-tested twice on a second Mac against a real Expo / React Native app.

    • Fixes included testid aliasing, session-URL parsing, concurrent producer/consumer, and session-end detection after HTTP 206s on large view trees.
  • sightmap device acceptance run on the same app: install, doctor, start, corpus (100% coverage with a leaf-selector corpus), concurrent producer/consumer and network all passed. Act and snapshot blockers were fixed upstream, and a final re-check is pending.

  • Network sources, ranked by spikes on iOS 26.2:

    1. Fullstory session.
    2. CFNetwork HAR log.
    3. React Native over Metro.
    4. CFNetwork summary.

    xctrace hangs against the simulator. Full table in FIELD-NOTES.md.

  • Local-only default, verified on the SwiftUI sample built with the SDK: no session, no upload tasks, and no contact with Fullstory over a full upload interval. The ServerUrl backstop and the React Native / Expo path are untested.

  • Open Fullstory-side issues: duplicated network events, detail returning response headers only, and network data lagging 35 s to 2 min.

Field validation (Oct 30, 2026)

The current skill was run end to end on a real Expo / React Native app (iOS Simulator, iOS 26.2) on another Mac.

Recording run (passed):

  • producer and consumer ran concurrently; local requests joined to the session by time, method and URL within 1 s;
  • session notes recorded; at least 3 request names attributed;
  • about 30 properties: found from local bodies;
  • 16 authored entries, validate clean;
  • a 206 ended the session, and the run fell back to local-only capture.

Local-only run (passed): the re-signed copy launched, no session URL appeared after 2 min, nothing was sent to Fullstory, and har capture was active.

The run found ten skill-text gaps, fixed in this PR: which re-sign form fails, stalled sessions with no termination line, first-open failures before the first bundle, duplicate requests in both sources, checking get-task-allow, apps whose Debug builds record to production, adding properties to a named route, --since on the termination check, React Native screen attribution, and host-less summary requests.

Still unexercised: the ServerUrl backstop, and the exact wording of the ask-first question.

🤖 Generated with Claude Code

chandlervdw and others added 6 commits October 29, 2026 10:14
Driving the iOS Simulator moved upstream as `sightmap device`
(sightmap/sightmap#522). This keeps only the Fullstory side: how SDK data
reaches agents, capture requirements, finding the session URL and session
end in the console, and the concurrent producer/consumer workflow. Carries
the prototype's plan and field notes into docs/subtext-device/.

Supersedes #39. The vendored sightmap-device skill follows once a sightmap
release includes `sightmap device`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
Explain that wrapper testIDs don't reach iOS accessibility, so device
corpora use leaf selectors; run review-open/zoom/close in sequence; and
record the acceptance run and post-MVP backlog in the field notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
…ogether

Lead with the Fullstory session for attribution (which tap and screen
caused each request) and `sightmap device network` for exact routes,
headers and bodies, joined by time, method and URL; degrade to either
alone, then to the summary source. Redaction and field-paths-only rules
for local capture; record session URLs and ends with `run note`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
Before a simulator run, ask whether it should record a session to
Fullstory, defaulting to no. For a local-only run, install a copy of the
built app with RecordOnStart=false and ServerUrl pointed at an
unreachable local address, re-signed with its entitlements, verify no
session starts, and work from `sightmap device network` alone. Verified
on the SwiftUI sample: no session, no uploads, no contact with Fullstory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
CODE_SIGNING_ALLOWED=NO drops the __TEXT,__entitlements section, so the
SDK's background uploads fail with NSURLErrorDomain -1. Name the flag,
the otool check, and the rebuild fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
Co-authored-by: Indent <noreply@indent.com>
Signed-off-by: Chandler Van De Water <chandler@fullstory.com>
@indent
indent Bot marked this pull request as ready for review October 30, 2026 15:34
chandlervdw and others added 2 commits October 30, 2026 14:31
…ulator and in review

Review's lidar tree carries an accessibilityIdentifier as the view's id, so
[identifier=...] and [testid=...] selectors match in sightmap device but name
nothing in review-snapshot. Also note that SwiftUI content records as anonymous
swiftui layers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
View-scoped components don't apply in review for mobile sessions yet
(page URL is the view-controller name), so corpora that must serve review
use global components with per-screen memory; a UIBarButtonItem's id
repeats on its wrapper views; wait ~3 s before backgrounding so the last
taps upload. Recorded in the field notes, with the click-after-modal
issue routed to sightmap device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Chandler Van De Water <chandler.vdw@gmail.com>
@indent
indent Bot requested review from joelgwebber and jurassix November 4, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant