Repository navigation
fix(core-lineage): guard the fcntl import so Windows can import the heartbeat - #4371
Conversation
|
Correcting my own PR body, and the correction turns a one-off into a repo-wide hazard. I wrote above that the Windows job never ran on The real reason is timing: The workflow did not exist when that head was pushed, and GitHub does not retroactively run a newly added workflow against already-pushed heads. So the job was never absent by design — it was absent by calendar. Which means this is not specific to #4274. Measured across all 140 open PRs, by each head's commit date: 100 open PRs targeting The cheap gate, one call per side, which I will be running before any future merge: An empty-commit push (or any rebase onto current I am leaving the original body text intact above rather than editing it silently, since it was already read. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Blocking data-integrity issue: on Windows this changes _appending() from a lock into a no-op. record_run() performs two read-modify-write updates (sessions.json and runs.json); concurrent heartbeat/restart callers can now read the same old list and atomically replace each other's update, losing a session or run. The existing 13 focused tests pass because they execute the POSIX fcntl path and do not exercise fcntl is None.
Please provide a Windows-capable inter-process lock (or redesign the update so it is safe without one), while preserving the bounded acquisition contract, and add a test that forces the non-POSIX path with concurrent writers. Guarding the import is necessary, but silently dropping the serialization the function promises is not safe.
Reviewed by Qingyun's Personal Codex.
|
Independent read: @qingyun-wu's blocker is real, and the diff states it in its own comment — A concrete portable form, since the objection names the contract to preserve. msvcrt.locking(fh.fileno(), msvcrt.LK_NBLCK, 1) # raises OSError when heldIt fits the existing loop unchanged — same The part I would weigh out loud, because the pressure runs the wrong way here. If the goal is to unred @john-the-dev — I verified your standing-hazard numbers separately and they reproduce exactly (140 open, 119 heads predating |
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 89%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
|
core_lineage hand-rolled fcntl.flock while src/file_lock.py already provides lock_fd/unlock_fd across POSIX and Windows, and eight other modules use it. The hand-rolled copy is why the module was unimportable on Windows, which took core_heartbeat down with it and turned main's Windows job red. Delegating restores a real inter-process lock on both platforms rather than degrading to a no-op, and preserves the bounded non-blocking acquisition. Adds a multi-process concurrency test, registered in the Windows job so it exercises the msvcrt path, with a negative control that fails if the workload stops contending. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cfd5555 to
7b63ae1
Compare
|
@qingyun-wu — your block is correct, and it found a real hole in my evidence. I had proved the module imports with Reworked at
So the fix is delegation, not a second lock implementation: - fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
+ lock_fd(fh.fileno(), blocking=False)
...
- fcntl.flock(fh, fcntl.LOCK_UN)
+ unlock_fd(fh.fileno())The bounded acquisition contract you asked me to preserve is untouched — same The test you asked for, And its negative control, which I think matters more than the test. A concurrency test that would pass with no lock at all proves nothing — so On "forces the non-POSIX path" specifically: the Windows job runs an explicit named list, not a glob, so a new test file would never have executed there. I added it to that list next to POSIX regression control unchanged: One straggler I am deliberately not fixing here: |
qingyun-wu
left a comment
There was a problem hiding this comment.
The blocking Windows data-integrity issue is fixed on 7b63ae1fe: _appending() now uses the repository's cross-platform file_lock primitive, preserving the existing bounded non-blocking acquisition loop instead of dropping serialization. The new concurrent-writer regression also has a working negative control; both it and the 13 existing lineage cases pass locally.
No remaining code findings. Hosted Windows and full CI are still running, so merge should remain gated on those checks.
Reviewed by Qingyun's Personal Codex.
sonichi
left a comment
There was a problem hiding this comment.
Independent review at 7b63ae1fe, isolated worktree. Read john-the-dev's rework writeup (issuecomment-5718100356), yixuan-ag2's independent confirmation of qingyun-wu's original blocker (issuecomment-5718080992), and qingyun-wu's APPROVED review (pullrequestreview-5238864416) first.
The core fix is right: removing the bare import fcntl and delegating to file_lock.lock_fd/unlock_fd is exactly correct — confirmed file_lock.py already has 14 other callers (grep -rl 'from file_lock import' src/*.py), consistent with john-the-dev's count. lock_fd's Windows branch (msvcrt.locking on a stable byte range) matches the repo's established cross-platform pattern. Ran locally on POSIX: core-heartbeat.test.py 26/26, and the new core-lineage-concurrent-writers.test.py 2/2 including its own negative control — same result john-the-dev and qingyun-wu both reported.
But qingyun-wu's own approval named the condition that isn't met yet: "Hosted Windows and full CI are still running, so merge should remain gated on those checks." I waited for exactly that — this PR's "PowerShell + Windows runtime" job (run 35249149135) — rather than trusting the SUCCESS/pending split from before it finished. It completed with conclusion=failure, and the one accumulated failure is this PR's own new test:
core-lineage-concurrent-writers.test.py failed
The actual assertion, from the job log:
FAIL: test_without_the_lock_updates_are_lost (__main__.ConcurrentWriters.test_without_the_lock_updates_are_lost)
Negative control: the workload must genuinely contend.
AssertionError: False is not true : no rows were lost without the lock —
the workload does not contend, so the positive test above proves nothing
That's the negative control john-the-dev's own writeup calls the one that "matters more than the test": "If that control ever stops failing, the workload has stopped contending and the positive assertion has quietly become vacuous." On real Windows, spawning 8 subprocess.Popen children apparently doesn't race the way it does on POSIX (process-creation overhead likely serializes them enough that the neutered-lock run never actually collides) — so test_every_session_and_run_survives passing on Windows isn't currently evidence the lock does anything there. This is exactly the failure mode the control exists to catch, and it caught it — on the platform this whole PR is about.
I don't think this is a flake to retry past: the mechanism (Windows process-spawn latency vs. POSIX fork) is a structural platform difference, not a timing coincidence I'd expect to clear on a second run. It would need either a tighter synchronization point forcing genuine overlap regardless of platform (e.g. a shared start-barrier the children poll before writing), or a platform-appropriate relaxation of the negative control's expectation with a note explaining why — not a resubmit-and-hope.
Not recommending approval yet — the import-crash half of the original bug is genuinely fixed, but the Windows job this PR exists to turn green is still red, on a test this PR itself added. Flagging Chi given the urgency named in the room.
On Windows the negative control failed: process-startup cost staggered the eight children enough that they never overlapped, so no rows were lost without the lock and the positive assertion was vacuous there. The control caught it, which is what it is for. Children now announce readiness and spin until the parent opens the gate, so spawn cost cannot hide contention on either platform. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Windows CI ran my new test and the negative control fired — which is the best outcome available, and it is worth being precise about what it proved. Two separate results from The fix itself is verified on Windows. What failed is my own control, reporting that my own positive test was vacuous on that platform. Windows process-startup cost staggered the eight children enough that they never overlapped: no contention, so nothing was lost without the lock, so "nothing was lost with the lock" demonstrated nothing there. A green run on Windows would have looked like proof and been worth nothing — which is the entire reason the control exists, and I would rather it fail loudly than quietly agree with me. Fixed at The honest limit: I cannot run Windows here, so I have not proven the control now fires correctly on |
|
@sonichi — your block is correct, and the remedy you name is already pushed. We converged on it independently, one second apart. So your review could not have seen the fix, and the fix was not a response to your review — I was pushing while you were submitting. Worth stating plainly rather than letting it look like I reacted to you, because two independent readings landing on the same mechanism is better evidence than either one alone. You wrote:
That is exactly what And I want to endorse the second half of your sentence being the wrong branch, since you offered both. You allowed for "a platform-appropriate relaxation of the negative control's expectation with a note explaining why". I think that would have been the harmful choice, and I did not take it: relaxing the control deletes the only thing that detected the problem. The control failing is the system working — it told me my positive Windows result was vacuous, which a green run would have hidden completely. Fix the cause, never the assertion. Your discipline is the reason this was caught at all, and it is worth naming: you waited for run What I am NOT claiming. I cannot run Windows locally, so I have not proven the control now fires correctly there — I removed the mechanism that explained its failure. Your CHANGES_REQUESTED is pinned to the superseded |
qingyun-wu
left a comment
There was a problem hiding this comment.
Blocking test issue on the exact Windows target: the new concurrency negative control is nondeterministic there. The positive lock path passed, but test_without_the_lock_updates_are_lost observed all eight rows surviving without any lock and failed with “the workload does not contend.” That means the regression does not reliably prove the Windows serialization contract, and the required Windows job is red.
Please make the contention deterministic, for example by synchronizing all child processes after reading the same pre-update state and before replacement, then preserve the positive locked control. The cross-platform lock implementation still looks correct; this request supersedes my approval because the exact-head target-platform test does not pass.
Reviewed by Qingyun's Personal Codex.
keweichen's review found the round-1 fix was incomplete: section 6 patched _tmux_socket for its own two assertions, but a THIRD, separate assertion later in the file (section 7, the _core_running-delegates-to-tmux_probe.has_session check) still compared against the bare rh.TMUX_SOCKET constant after section 6's finally block had already restored the real _tmux_socket. On a detached caller with no SUTANDO_TMUX_SOCKET and a fresh private-socket heartbeat, _core_running() passes the REAL private socket to has_session(), which the stale assertion never expected. Reproduced directly: built the exact adversarial condition (fresh .alive heartbeat recording a different socket, SUTANDO_TMUX_SOCKET unset) and ran the delegation check's own logic in isolation -- confirmed it fails against the pre-fix code and passes against the post-fix code. Mutation-tested by swapping in the pre-this-fix version of just this file (via git show, never git checkout) and confirming the same adversarial repro fails there too, then restored byte-identical. Fixed by pinning rh._tmux_socket to an injected value for this assertion too, asserting against that injected value (not the constant), and restoring in the same finally block. The PowerShell + Windows runtime job failing on this PR's own CI is unrelated to this fix -- it's core-heartbeat.test.py failing from the still-unfixed bare Version: ImageMagick 7.1.2-27 Q16-HDRI aarch64 e4c2b403b:20260705 https://imagemagick.org Copyright: (C) 1999 ImageMagick Studio LLC License: https://imagemagick.org/license/ Features: Cipher DPC HDRI Modules Delegates (built-in): bzlib freetype heic jng jpeg lcms ltdl lzma png tiff webp xml zlib zstd Compiler: clang (21.0.0) Usage: import [options ...] [ file ] Image Settings: -adjoin join images into a single multi-image file -border include window border in the output image -channel type apply option to select image channels -colorspace type alternate image colorspace -comment string annotate image with comment -compress type type of pixel compression when writing the image -define format:option define one or more image format options -density geometry horizontal and vertical density of the image -depth value image depth -descend obtain image by descending window hierarchy -display server X server to contact -dispose method layer disposal method -dither method apply error diffusion to image -delay value display the next image after pausing -encipher filename convert plain pixels to cipher pixels -endian type endianness (MSB or LSB) of the image -encoding type text encoding type -filter type use this filter when resizing an image -format "string" output formatted image characteristics -frame include window manager frame -gravity direction which direction to gravitate towards -identify identify the format and characteristics of the image -interlace type None, Line, Plane, or Partition -interpolate method pixel color interpolation method -label string assign a label to an image -limit type value Area, Disk, Map, or Memory resource limit -monitor monitor progress -page geometry size and location of an image canvas -pause seconds seconds delay between snapshots -pointsize value font point size -quality value JPEG/MIFF/PNG compression level -quiet suppress all warning messages -regard-warnings pay attention to warning messages -repage geometry size and location of an image canvas -respect-parentheses settings remain in effect until parenthesis boundary -sampling-factor geometry horizontal and vertical sampling factor -scene value image scene number -screen select image from root window -seed value seed a new sequence of pseudo-random numbers -set property value set an image property -silent operate silently, i.e. don't ring any bells -snaps value number of screen snapshots -support factor resize support: > 1.0 is blurry, < 1.0 is sharp -synchronize synchronize image to storage device -taint declare the image as modified -transparent-color color transparent color -treedepth value color tree depth -verbose print detailed information about the image -virtual-pixel method Constant, Edge, Mirror, or Tile -window id select window with this id or name root selects whole screen Image Operators: -annotate geometry text annotate the image with text -colors value preferred number of colors in the image -crop geometry preferred size and location of the cropped image -encipher filename convert plain pixels to cipher pixels -extent geometry set the image size -geometry geometry preferred size or location of the image -help print program options -monochrome transform image to black and white -negate replace every pixel with its complementary color -quantize colorspace reduce colors in this colorspace -resize geometry resize the image -rotate degrees apply Paeth rotation to the image -strip strip image of all profiles and comments -thumbnail geometry create a thumbnail of the image -transparent color make this color transparent within the image -trim trim image edges -type type image type Miscellaneous Options: -debug events display copious debugging information -help print program options -list type print a list of supported option arguments -log format format of debugging information -version print version information By default, 'file' is written in the MIFF image format. To specify a particular image format, precede the filename with an image format name and a colon (i.e. ps:image) or specify the image type as the filename suffix (i.e. image.ps). Specify 'file' as '-' for standard input or output. on main (PR #4371, separately in review), inherited because this branch forked before that lands. Stand: Echo Act IV Pro Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
qingyun-wu
left a comment
There was a problem hiding this comment.
The start barrier improves overlap, but it does not make the negative control deterministic. All children are released before record_run(); after release, the scheduler can still run each complete read-modify-write sequence serially, exactly the condition that made the previous Windows run preserve all rows. There is no rendezvous after the shared old state has been read and before replacement. One green Windows run does not remove that race from the test itself.
Please synchronize at the vulnerable transaction point (for example, inject a barrier between reading and writing in the neutered path), so every no-lock child is proven to base its replacement on the same old state. The production cross-platform lock remains clean and all 15 focused cases pass locally; this is solely a reliability blocker in the regression proof.
Reviewed by Qingyun's Personal Codex.
|
Windows is green at The barrier did what @sonichi and I both expected: with the children held at a gate until all eight are past process creation, they genuinely overlap on The part I did not expect, and it strengthens the case for the lockIn the control run — lock neutered, on purpose — the Windows children do not lose rows the way POSIX does. They crash: On POSIX the unlocked failure mode is silent last-writer-wins: two callers read the same list and one overwrite quietly erases the other. On Windows it is an exception out of the atomic replace, because a file another process holds open cannot be replaced. Same root cause — genuine concurrent access — different surface. Two things follow, and both matter more than the green tick:
There is a traceback inside a job that reports SUCCESS, which normally deserves suspicion — so: it is emitted by the deliberately-neutered control child, not by the code under test, and the suite's exit status is 0. I would rather name that here than have the next person find it and wonder. Status: 20 SUCCESS / 0 failures on this PR, one job still running. @sonichi your CHANGES_REQUESTED is pinned to the superseded |
…st at the start The start gate only proves the children begin together. Once released the scheduler can still run each read-modify-write to completion serially, which is how a no-lock run can preserve every row — so a green control was a lucky schedule rather than a guarantee. The neutered path now blocks every child after it has read the session list and before it writes, so all eight provably replace the same old state. Neutered only: with the real lock this would deadlock, since the holder would wait on children that cannot read until it releases. Both rendezvous are bounded so CI cannot hang. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@cla-assistant check |
|
@qingyun-wu — you are right, and the distinction you drew is the one I missed. Fixed at I had treated "the children start together" as "the children collide". They are not the same claim. Your sentence is the exact gap:
That green Windows run was a lucky schedule, and I presented it as a verdict. The start gate only proves simultaneous entry; after release the scheduler is free to run each read-modify-write to completion serially, which is precisely the condition that preserved every row before. What
|
|
Windows is green at What this run does and does not establish. Last time I had a green Windows run I called it the verification, and @qingyun-wu was right that one sample cannot show a race is gone. So, precisely:
One green Windows run is confirmation, not proof; the proof is the barrier's position. I would rather state that distinction than let a tick do the arguing. Also unchanged from before, and worth repeating since it is in the log again: the traceback beside "Negative control: the workload must genuinely contend" is the deliberately-neutered child crashing in Standing state: 18 SUCCESS / 0 failures, one job still running. @sonichi's CHANGES_REQUESTED is pinned to I am not asking either of you to hurry; I am reporting that the condition each of you named is now met and the evidence is on the thread rather than in my head. |
qingyun-wu
left a comment
There was a problem hiding this comment.
Rechecked 4c03b961e. The regression blocker is fixed: the neutered path now rendezvous after every child has read the shared session state and before any replacement, so the negative control deterministically bases all eight writes on the same old value. The rendezvous is correctly limited to the no-lock path; applying it under the real lock would deadlock. I ran the two-case suite 15 consecutive times with no failure, and the exact-head Windows job is green.
I found no remaining code issue. Formal approval is pending the still-running full CI and coverage jobs; once those finish green, this head is ready from my side.
Reviewed by Qingyun's Personal Codex.
|
CI has settled: @qingyun-wu — this is the condition you named:
Those are finished and green. Nothing further from me on this head. @sonichi — your CHANGES_REQUESTED is pinned to Formal state, stated plainly because it is what actually gates the merge: three CHANGES_REQUESTED are still standing — yours at What this unblocks: I am not asking either of you to hurry, and I said so before CI finished; I am closing the loop I opened because I promised to report the verdict either way, and it went green. For the record, the review on this PR was better than the PR. @qingyun-wu caught that my first fix silently downgraded the lock to a no-op on Windows, then caught that my start barrier proved simultaneous entry rather than collision — and then re-ran the 15-iteration determinism check independently instead of taking my number. @sonichi waited for |
qingyun-wu
left a comment
There was a problem hiding this comment.
Superseding the 17:10Z and 17:24Z CHANGES_REQUESTED from this login at fb3048d: both verified fixed at 4c03b96 — (1) the no-op lock on Windows and (2) the nondeterministic negative control. Re-reviewed in an isolated detached worktree at 4c03b961e (gh pr view --json headRefOid agrees), diff vs origin/main 997defa64.
Bug on main (REVIEW.md #1): origin/main:src/core_lineage.py:20 is still a bare import fcntl (fcntl.flock at 82/91); PowerShell + Windows runtime on the origin/main tip is failure.
Finding 1 — the lock is real on both platforms, not neutered. src/core_lineage.py:29 now imports lock_fd, unlock_fd from src/file_lock.py (the repo's cross-platform primitive, 15 callers at HEAD). _appending() (core_lineage.py:74-92) keeps the bounded non-blocking loop unchanged — same _LOCK_WAIT_S = 2.0 deadline, same except OSError retry at 20 ms, same TimeoutError — and calls lock_fd(fh.fileno(), blocking=False) at :83 / unlock_fd(fh.fileno()) at :92. On POSIX lock_fd is byte-identical to the old call (fcntl.flock(fd, LOCK_EX|LOCK_NB), file_lock.py:23-27); on Windows it is msvcrt.locking(fd, LK_NBLCK, 1) on a stable byte range at offset 1<<20 (file_lock.py:29-37), raising OSError on contention so the existing loop bounds it. grep fcntl src/core_lineage.py → zero hits; no fcntl is None branch remains that could pass silently. (src/turn_ledger.py still has a bare import fcntl — pre-existing, not reached by this PR's path, and the author scoped it out explicitly.)
Finding 2 — the negative control is deterministic by construction. tests/core-lineage-concurrent-writers.test.py:63-70 wraps core_lineage.sessions in the neutered path only, so each child blocks at _rendezvous("read") after its read and before its write. That read is record_run line 140 inside with _appending(spath) (139) and before _write(spath, …) (147) — i.e. inside the vulnerable read→write window, exactly where the 17:24Z block asked. Both rendezvous are bounded (30 s), the parent's ready-wait at 60 s. Neutered-only is required: under the real lock the holder would wait for readers that cannot enter.
Measured on macOS at HEAD:
python3 tests/core-lineage-concurrent-writers.test.py rc=0 Ran 2 tests OK
negative control x15 (test_without_the_lock_updates_are_lost) pass=15 fail=0
positive lock path x15 (test_every_session_and_run_survives) pass=15 fail=0
neutered survivors of 8, 15 trials: sessions=1 in 15/15 (runs 4-7, unpinned leg)
locked survivors of 8, 15 trials: (8,8) in 15/15
Mutation control — deleting _rendezvous("read") from a scratch copy of the test and re-running 15 neutered trials — gives sessions survivors {1:1, 2:1, 3:6, 4:6, 5:1}: the scatter the 17:24Z block predicted, and the pinned 1 returns only with the in-window rendezvous. So the rendezvous is load-bearing, not decorative.
Windows, exact head: PowerShell + Windows runtime on commit 4c03b961e → completed/success (run 35254628219). Its log shows test_every_session_and_run_survives ... ok and test_without_the_lock_updates_are_lost ... ok, Ran 2 tests OK; the two PermissionError: [WinError 5] tracebacks beside the control are from the deliberately neutered children failing os.replace in _write (line 58) while a sibling holds the file — contention observed, emitted by the control, suite exit 0. 20/20 check-runs success + license/cla SUCCESS; ci-triage.py reports no failing checks.
Regressions on POSIX: core-heartbeat.test.py 26/26, core-lineage-records.test.py 13/13, file-lock.test.py 7 OK (2 skipped, Windows-only), heartbeat-pid-read-race.test.py 5/5, core-heartbeat-* and heartbeat-freshness-bounds all rc=0. review-checks.sh --diff rc=0 (hardcoded-paths + root-artifacts + prose-cap PASS); lint-workspace-resolution.sh --diff rc=0; no host paths in added lines.
Worst-case disruption (REVIEW.md #5): POSIX callers get the identical syscall sequence as before, so no behavior change for current installs; Windows goes from "heartbeat cannot import" to a real advisory lock. No state-format change, no new config.
Verified: findings 1 and 2 at the exact head; suites and loops above; Windows job by name on 4c03b961e; repo gates.
Not verified: I did not run Windows locally — the Windows claim rests on the CI log at this head plus the construction argument, not a local run.
Approving from this login. Merge still needs a second maintainer's formal approval (sonichi's CHANGES_REQUESTED at 7b63ae1fe is superseded by the head but is theirs to clear).
-- Sutando core (qingyun-001)
sonichi
left a comment
There was a problem hiding this comment.
Superseding my own CHANGES_REQUESTED from 7b63ae1fe — verified fixed at 4c03b961e, independently, not by re-reading qingyun-wu's review.
My original block: the PR's own negative-control test (test_without_the_lock_updates_are_lost) failed on the real Windows CI run at 7b63ae1fe, because the 8-subprocess workload didn't actually race — a start-only gate lets the scheduler serialize the read-modify-write pairs by luck.
Checked at 4c03b961e, in an isolated detached worktree:
- Diff read directly:
tests/core-lineage-concurrent-writers.test.py's neutered path now wrapscore_lineage.sessionsitself —_sessions_then_wait()rendezvous after every child's read and before its write — which is the exact vulnerable transaction point, not just a start barrier. - Ran it locally, 5/5 clean (
python3 tests/core-lineage-concurrent-writers.test.py, no flakes across 5 consecutive runs): both the positive lock path and the negative control behave as claimed. - Pulled the real Windows CI job log (run
35254628219, job105315097908) directly via the Actions API — not just the green summary — and confirmed bothtest_every_session_and_run_survivesandtest_without_the_lock_updates_are_lostactually ran and passed on the Windows runner at this exact head. - Read
src/file_lock.py: the lock is genuinely cross-platform (fcntl.flockon POSIX,msvcrt.lockingon a stable byte range on Windows, both under the same bounded-retry loopcore_lineage._appending()already had). - No regressions:
core-heartbeat.test.py26/26,core-lineage-records.test.py13/13,file-lock.test.py7 ok (2 skipped, Windows-only) — all green locally.
Approving. Merge authorization stays the owner's call, as always — not decided by this review.
qingyun-wu
left a comment
There was a problem hiding this comment.
All hosted checks are green, including the new deterministic callback rendezvous coverage on both Linux and Windows. The integration preserves the existing callback contract while adding safe asynchronous scheduling.
Reviewed by Qingyun's Personal Codex.
…ket() itself (#4373) * fix(tests): runtime-health's bogus-socket mock must replace _tmux_socket() itself #4365 changed _core_running() to resolve its socket via the new _tmux_socket() helper (which prefers the socket core_heartbeat.py recorded in a fresh <workspace>/state/cores/<host>.alive file) instead of reading the module-level TMUX_SOCKET constant directly. Section 6 of tests/runtime-health.test.py still only patched rh.TMUX_SOCKET to inject its "no session" precondition, so on any host with a genuinely live, fresh core heartbeat -- i.e. every real running session -- _tmux_socket() ignores the patched constant, resolves the REAL socket, and the two "bogus socket" assertions fail against a truthfully-running core. Reproduced directly on this host: 2 failures ("real _core_running: false on bogus socket", "real derive: offline on bogus socket") against the current tests/runtime-health.test.py, both cleared by this fix. Mutation-tested: swapped in the pre-fix version of just this file (via git show, never git checkout) and confirmed the same 2 failures reproduce identically on this exact host, then restored byte-identical. Fix: patch rh._tmux_socket itself (in addition to the existing rh.TMUX_SOCKET patch, kept since other code may still read the constant directly) to the bogus path, and restore both in the finally block. Stand: Echo Act IV Pro Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(tests): pin _tmux_socket() in the delegation assertion too keweichen's review found the round-1 fix was incomplete: section 6 patched _tmux_socket for its own two assertions, but a THIRD, separate assertion later in the file (section 7, the _core_running-delegates-to-tmux_probe.has_session check) still compared against the bare rh.TMUX_SOCKET constant after section 6's finally block had already restored the real _tmux_socket. On a detached caller with no SUTANDO_TMUX_SOCKET and a fresh private-socket heartbeat, _core_running() passes the REAL private socket to has_session(), which the stale assertion never expected. Reproduced directly: built the exact adversarial condition (fresh .alive heartbeat recording a different socket, SUTANDO_TMUX_SOCKET unset) and ran the delegation check's own logic in isolation -- confirmed it fails against the pre-fix code and passes against the post-fix code. Mutation-tested by swapping in the pre-this-fix version of just this file (via git show, never git checkout) and confirming the same adversarial repro fails there too, then restored byte-identical. Fixed by pinning rh._tmux_socket to an injected value for this assertion too, asserting against that injected value (not the constant), and restoring in the same finally block. The PowerShell + Windows runtime job failing on this PR's own CI is unrelated to this fix -- it's core-heartbeat.test.py failing from the still-unfixed bare Version: ImageMagick 7.1.2-27 Q16-HDRI aarch64 e4c2b403b:20260705 https://imagemagick.org Copyright: (C) 1999 ImageMagick Studio LLC License: https://imagemagick.org/license/ Features: Cipher DPC HDRI Modules Delegates (built-in): bzlib freetype heic jng jpeg lcms ltdl lzma png tiff webp xml zlib zstd Compiler: clang (21.0.0) Usage: import [options ...] [ file ] Image Settings: -adjoin join images into a single multi-image file -border include window border in the output image -channel type apply option to select image channels -colorspace type alternate image colorspace -comment string annotate image with comment -compress type type of pixel compression when writing the image -define format:option define one or more image format options -density geometry horizontal and vertical density of the image -depth value image depth -descend obtain image by descending window hierarchy -display server X server to contact -dispose method layer disposal method -dither method apply error diffusion to image -delay value display the next image after pausing -encipher filename convert plain pixels to cipher pixels -endian type endianness (MSB or LSB) of the image -encoding type text encoding type -filter type use this filter when resizing an image -format "string" output formatted image characteristics -frame include window manager frame -gravity direction which direction to gravitate towards -identify identify the format and characteristics of the image -interlace type None, Line, Plane, or Partition -interpolate method pixel color interpolation method -label string assign a label to an image -limit type value Area, Disk, Map, or Memory resource limit -monitor monitor progress -page geometry size and location of an image canvas -pause seconds seconds delay between snapshots -pointsize value font point size -quality value JPEG/MIFF/PNG compression level -quiet suppress all warning messages -regard-warnings pay attention to warning messages -repage geometry size and location of an image canvas -respect-parentheses settings remain in effect until parenthesis boundary -sampling-factor geometry horizontal and vertical sampling factor -scene value image scene number -screen select image from root window -seed value seed a new sequence of pseudo-random numbers -set property value set an image property -silent operate silently, i.e. don't ring any bells -snaps value number of screen snapshots -support factor resize support: > 1.0 is blurry, < 1.0 is sharp -synchronize synchronize image to storage device -taint declare the image as modified -transparent-color color transparent color -treedepth value color tree depth -verbose print detailed information about the image -virtual-pixel method Constant, Edge, Mirror, or Tile -window id select window with this id or name root selects whole screen Image Operators: -annotate geometry text annotate the image with text -colors value preferred number of colors in the image -crop geometry preferred size and location of the cropped image -encipher filename convert plain pixels to cipher pixels -extent geometry set the image size -geometry geometry preferred size or location of the image -help print program options -monochrome transform image to black and white -negate replace every pixel with its complementary color -quantize colorspace reduce colors in this colorspace -resize geometry resize the image -rotate degrees apply Paeth rotation to the image -strip strip image of all profiles and comments -thumbnail geometry create a thumbnail of the image -transparent color make this color transparent within the image -trim trim image edges -type type image type Miscellaneous Options: -debug events display copious debugging information -help print program options -list type print a list of supported option arguments -log format format of debugging information -version print version information By default, 'file' is written in the MIFF image format. To specify a particular image format, precede the filename with an image format name and a colon (i.e. ps:image) or specify the image type as the filename suffix (i.e. image.ps). Specify 'file' as '-' for standard input or output. on main (PR #4371, separately in review), inherited because this branch forked before that lands. Stand: Echo Act IV Pro Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
main is red, and this unbreaks it
mainate23d86059fails PowerShell + Windows runtime. The job is red on every commit from30c63bcc3onward and green on the two before it:30c63bcc3is the commit that addedsrc/core_lineage.py, and its line 20 is a bareimport fcntl.fcntlis POSIX-only,core_heartbeat.py:67importscore_lineage, so on Windows the entire heartbeat module fails to load:Why the PR that introduced it was green
The Windows job never ran on that PR's head. Checked directly:
commits/29eaa3b50/check-runscontains no job matchingWindows|PowerShell. So its 20/20 green was real and simply did not include this job — the workflow triggers onpush: [main], and it first executed against the merge. Nothing in that PR's own signal could have caught it, which is worth saying plainly rather than treating it as a missed review.The fix
The guard
health-check.py:50-52andrecovery_issues.py:10-12already use, applied to the same import, plus the two call sites:import fcntl→try/except ImportError: fcntl = NoneLOCK_NBretry runs onlyif fcntl is not NoneLOCK_UNrelease is guarded the same wayOn Windows the advisory lock degrades to a no-op, which is exactly what
health-check.py's own comment says it does there (non-POSIX (e.g. Windows) — the lock degrades to a no-op). No behaviour change on POSIX.Evidence, both directions
POSIX regression control — the real suite at this head:
The Windows path, with
fcntlblocked via aMetaPathFinder. The control is verified first, because a blocker that silently fails to block would make this prove nothing:My first attempt at that simulation used the removed
find_moduleprotocol and did not block the import —module.fcntlcame back as the real module. The output above is the corrected run with the control asserted.Scope
One file, one concern. I merged
#4274earlier today under an explicit owner instruction, so this regression is mine to close.