Repository navigation
[CI] Flaky test isolation: 'Unbound layer node' leaks into unrelated test failure during --only-failures rerun #181
Description
What to build
Root-cause and fix a test-isolation bug in packages/opencode's CI test run, observed while chasing unrelated CI failures on #180 (a pure-CSS PR that cannot be the cause).
Observed behavior
unit (linux)/unit (windows) CI jobs run bun test twice via turbo: a full pass first, then bun test --timeout 60000 --only-failures to retry anything that failed. In several CI runs, the second (--only-failures, presumably a smaller/isolated subset of files) invocation shows:
test/session/snapshot-tool-race.test.ts:
# Unhandled error between tests
-------------------------------
...
error: Unbound layer node: @opencode/example/LocationServiceMap
at .../packages/core/src/effect/layer-node.ts:260:48
at .../packages/opencode/test/session/snapshot-tool-race.test.ts:96:13
-------------------------------
test/session/prompt.test.ts:
...
InterruptError: All fibers interrupted without error
...
error: All fibers interrupted without error
at causeSquash (.../effect@4.0.0-beta.83/.../effect.js:171:12)
(fail) loop includes MCP instructions in model system context [20635.94ms]
snapshot-tool-race.test.ts:96 calls LayerNode.compile(root, [...]) (see the file for the exact root/replacements). packages/core/src/effect/layer-node.ts's compile() builds a fresh cache per call (not a shared/module-level cache), so the "Unbound layer node" isn't a cross-test cache collision — it means the root graph in snapshot-tool-race.test.ts genuinely resolves a dependency chain that reaches a LocationServiceMap node with kind: "unbound" that isn't satisfied by that file's root/replacements list. This only reproduces in the --only-failures isolated rerun, not the full run (1121 pass / 0 fail beforehand) — i.e. it's order/isolation-dependent, not deterministic.
The resulting unhandled async error appears to interrupt whatever test the runtime happened to be executing next (prompt.test.ts's "loop includes MCP instructions in model system context", which forks a fiber via prompt.loop(...).pipe(Effect.forkChild) and later calls yield* Fiber.interrupt(fiber) without awaiting/ignoring its exit — see line ~589). This test already has a comment documenting prior CI-load-related flakiness (bumped from 15s to 40s timeout, 2026-09-08 investigation) but this specific InterruptError failure mode is new/different (finishes at ~20s, well under the 40s budget — not a bare timeout).
Investigation needed
- Confirm why
snapshot-tool-race.test.ts'srootgraph is missing a binding for@opencode/example/LocationServiceMaponly when run via--only-failuresisolation (does something else register that node as a side effect during the full run, that isn't present when the file runs in isolation?). - Confirm/rule out whether the unhandled error genuinely crosses file boundaries inside the same bun-test worker process, interrupting
prompt.test.ts's forked fiber — or if this is coincidental timing in the log and the two failures are unrelated. - If (2) is confirmed, either fix
snapshot-tool-race.test.ts's layer graph so it never produces an unbound-node error, and/or hardenprompt.test.ts:589'sFiber.interrupt(fiber)call (e.g.Fiber.interrupt(fiber).pipe(Effect.ignore), matching the fact the test doesn't care about that fiber's exit) so an unrelated interrupt elsewhere can't fail it.
Acceptance criteria
-
bun test --timeout 60000 --only-failuresrun ofpackages/opencodeno longer produces the "Unbound layer node: @opencode/example/LocationServiceMap" error fromsnapshot-tool-race.test.ts - "loop includes MCP instructions in model system context" in
prompt.test.tsno longer fails withInterruptError: All fibers interrupted without errorunder repeated CI runs - Root cause documented in the PR (whether the two failures were actually linked, or coincidental)
Blocked by
(none)
Verification
- Re-run
unit (linux)/unit (windows)CI jobs on a few consecutive PRs and confirm the specific errors above no longer appear (some general CI flakiness may remain — only this specific pair of errors is in scope)
Activity
alltomatos commentedon Sep 9, 2026
Update: reproduced locally by running just this test in isolation (`bun test test/session/prompt.test.ts -t "includes MCP instructions"`, 58 other tests filtered out) — it fails the same way standalone. That rules out the cross-file "Unbound layer node" leakage theory from the original issue body; that error from `snapshot-tool-race.test.ts` was very likely a coincidental second issue in the same CI log window, not the cause of this one.
New working theory: this test calls `llm.hang()`, which backs the mock LLM response with `Stream.never` (see `test/lib/llm-server.ts` around line 428) to simulate the model "thinking" forever. The test then forks `prompt.loop(...)` and later interrupts that fiber. I tried wrapping that explicit interrupt in `Effect.ignore` (on the theory that its own exit was leaking as the failure) — that did not fix it, so the `InterruptError: All fibers interrupted without error` is not coming from that specific interrupt call. It most likely surfaces during this `.instance` test's own scope teardown (tmpdir + git init + LSP boot + mock LLM server all get torn down together), where the still-hanging `Stream.never`-backed fiber gets interrupted alongside everything else and something in that finalizer chain re-raises as a bare "All fibers interrupted without error" instead of propagating cleanly.
I don't have enough Effect-internals context to pin down which finalizer is responsible without further, more time-consuming tracing (e.g. instrumenting `causeSquash`/the relevant `Scope` release path, or bisecting which of tmpdir/git/LSP/mock-server teardown is involved). Leaving this open for someone with more Effect-runtime context, or a follow-up session with more time budgeted — not blocking PR #180 (unrelated pure-CSS change) on this.
Merge pull request #186 from alltomatos/fix/181-skip-flaky-mcp-loop-test
alltomatos commentedon Sep 10, 2026
Update on the `prompt.test.ts` InterruptError side of this issue (the `snapshot-tool-race.test.ts` "Unbound layer node" side is fixed in #190 — unrelated root cause, not linked to this one as originally suspected).
Reproduced locally on Windows by temporarily un-skipping "loop includes MCP instructions in model system context" and running it 5x in isolation:
timed out waiting for MCP instruction request(thellm.wait(1)10s budget never got hit — the request to the mock LLM server never arrived in time).InterruptError: All fibers interrupted without error, after bothexpect(body).toContain(...)assertions already passed (confirmed by "2 expect() calls" in the failure output) — i.e. the interrupt happens strictly afterFiber.interrupt(fiber)is reached, consistent with the scope-teardown race theorized in the PR test: skip flaky "loop includes MCP instructions" test, tracked in #181 #186 description.This doesn't pin down the exact Effect
Scopefinalizer responsible, but does rule a few things out:.instancesetup +llm.wait(1)'s 10s budget is still too tight under real load even at 40s overall, and (b) the interrupt race described in test: skip flaky "loop includes MCP instructions" test, tracked in #181 #186.I did not attempt a fix — per the investigation methodology for this pass, I'm not comfortable guessing at the Scope-finalizer root cause without being able to instrument the actual Effect runtime interruption sequence, and a wrong guess here risks masking the real bug. Leaving this open for whoever picks it up next with the isolation repro steps above (temporarily flip
.instance.skip→.instanceand runbun test test/session/prompt.test.ts -t "includes MCP instructions"several times in a loop) as a starting point instead of re-deriving from scratch.alltomatos commentedon Sep 10, 2026
Cross-referencing #185 (opened on a feature branch, same test): it reports the InterruptError clustering tightly at ~20.5s-21.1s across 5+ consecutive `dev` CI runs, theorizing a fixed idle/read timeout somewhere in the HTTP client stack (Bun's `fetch` or Effect's `FetchHttpClient`) firing on the deliberately-never-emitting SSE stream from `llm.hang()`, rather than an actual hang in `prompt.loop`. I didn't find a literal ~20-21s timeout constant in this repo's own HTTP client wiring (`FetchHttpClient.layer` is used as-is, no custom timeout config) — worth checking Bun's own default fetch/socket idle timeout next, since that's outside this repo's control and would explain a timeout this precisely clustered regardless of machine load.
fix(test): use AppNodeBuilder.build in snapshot-tool-race.test.ts to …