fix(engine): the filesystem/search module cycle — the #775 wedge root cause

[aarontrowbridge]

Closes #1709. Kills the wedge series behind the 2026-09-03/04 + 2026-10-03/04/05 hub incidents.

The root cause, end to end

FileSystem.node captured FileSystemSearch.node into its deps array at module init, while search.ts imported the FileSystem namespace back from ../filesystem for shapes it uses exclusively inside effect bodies. Under bun/esbuild module-cycle semantics that capture froze to undefined (TDZ in one entry order, silent undefined in the other — both reproduced minimally):

• every per-directory location build crashed with an anonymous TypeError … 'a.name' inside the layer walk,
• the builder caught and swallowed it (failed to load project references),
• the half-built fresh location layer leaked (watchers, snapshot trackers, LLM machinery) on every session prompt,
• at ~8+ parallel sessions the leak rate wedged the engine: RSS 2.5–3.6 GB in minutes, main thread spinning with GC threads idle, TCP-accepts/HTTP-silent — the watchdog-recovered outages all week.

The twice-per-boot ERROR pair (#1709) was the same crash at the boot-time location build.

The fix, three parts

1. filesystem/shared.ts (new) — the shared shapes (schema package's Entry/Match/Submatch/FindInput + GlobInput/GrepInput moved from filesystem.ts). Both sides import from it; neither imports the other. The cycle is gone at the source.
2. layer-node.ts (overlay twin) — the walk names the parent on an undefined dependency instead of crashing anonymously. The next module cycle cannot hide behind a swallowed TypeError. (The instrumented build that diagnosed this produced exactly this naming: undefined dependency under parent=@opencode/v2/FileSystem … deps=[…, UNDEFINED] — the line that located the culprit.)
3. Regression tests in BOTH module-evaluation orders — the freeze depends on which side of the cycle evaluates first, so each order pins the other (4/4 pass).

Verification

• bun test test/filesystem/cycle-order-* (materialized core) — 4 passed
• engine_typecheck_gate.sh — clean (only Base-pin drift: manifest upstream_base_sha stale vs absorbed overlay #1229-tracked base-drift)
• overlay_typecheck_gate.sh — OK across 7 packages
• drift_gate.mjs — PASS (manifest refreshed, 865 files)
• build:binary → a5ef305a, deployed live on the hub (via ops/hub-restart.sh stage/swap/restart): boot run logs ZERO ERROR/WARN where every prior boot logged the a.name pair; /config+/agent for the trigger directory 200 clean; harness/sessions verified through the tunnel.

Incident evidence (erlich)

~/.amico/server/fleet-watchdog/wedge-20261005-040014-wedge/ (smaps, gdb thread states, log tail with the named parent), the sourcemapped diagnostic build decoding #1709 to layer-node.ts:241, and the minimal both-orders bun repro.

Follow-ups

• The 8+ parallel sessions load that exposed the leak rate is now just load — but the per-prompt caught-and-swallowed error pattern deserves a look generally: silent catches that leak half-built fresh layers are the wedge shape to audit.
• live-test/harness-main (the in-fleet tree carrying feat(app): the harness switcher in the chat box (#1549) #1550 + this) will want this merged before its next redeploy.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 625461d1-69d2-4bd8-9be3-22658ff4a5c8

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[aarontrowbridge]

Addendum (same night): the cycle fix killed the location-build leak, but the wedge recurred on the FIXED engine at 04:20 EDT — same signature (silence at stream start, heap ~10 MB/s to 3.1 GB, main thread saturated), zero guard hits, clean location builds. The surviving driver: the provider stream itself. Every streamed delta was pushed into an unbounded fragment buffer (joined only at stream end) and the request carries max_tokens only when generation config sets it — a model turn that never stops generating accumulates without limit and takes the engine down. The DB corroborates: no part over 10 MB has ever completed — the growth never lands anywhere; it dies with the engine.

The commit at the branch tip (now also live-test/harness-main) adds the guard rails to the publisher's fragment accumulator: 32 MB per fragment / 64 MB per turn (4–8× the largest legitimate part ever recorded, 7.9 MB — a rail, not a limiter), crossing them dies with an honest named stream overflow error — the session sees a real failure, the engine survives. Env-overridable + per-turn streamCaps for tests. stream-overflow.test.ts pins both caps; the full core suite (43 tests), engine + overlay typecheck gates, drift gate, all green. Built 70ca1345, deployed live on the hub (the tree's provenance branch pushed).

Follow-up worth its own PR: a default max_tokens in the llm request builder (provider-dependent, changes model behavior — deliberately not bundled with a safety net).

[aarontrowbridge]

Third leg (same night, live-tested): the response-side cap. The publisher caps never fired across two more wedges — zero frames ever reached the publisher. The wedge lives at the transport: the SSE framer buffers by line, so an unterminated SSE line (or any never-terminating response body) accumulates at receive speed with no event ever completing — heap ~10 MB/s from stream start, no deltas, HTTP silent. A status-check turn on the same session completed clean, so the pathology is provider-side and stochastic — which is exactly why the bound must sit at the seam every byte crosses.

010c5838 (deployed live): 64 MB per provider response (8× the largest legitimate turn ever recorded) enforced in the http-json transport's frames(), dying with the named llm response exceeded … error — an endless body dies in ~6 s at observed rates instead of wedging the engine. Env: AMICODE_MAX_LLM_RESPONSE_BYTES. Tests: endless unterminated body trips the cap with the named error; bounded SSE body passes untouched. Full suite green (45 engine tests), both typecheck gates, drift gate.

The three layers now bound the full path: response bytes → fragment accumulation → per-turn total. A runaway anywhere in the pipeline now costs one failed turn with an honest error, never the hub.