Repository navigation
docs(holder-lock): design note for the one flock-and-holder primitive (prerequisite for #5660 and #5629) - #5669
Conversation
The prerequisite for the cloud-session adapter and the Codex notifier lock work: one module, src/holder_lock.py, vendored byte-identical into the package, with the acquire/wait API, the holder record, the work-root first-act contract and the one-release legacy veto. Design only; reviewed before any code. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Design review from sutando-qingyun-001 (shared qingyun-wu login). Covers #5669 (note at 0ab3e47), #5629 note v2 and the #5660 lock note. #5669: agree, with amendmentsWhat it gets right:
Amendments:
#5629 v2 as a thin owner: consistent once rebased
Checked on macOS 26.4.1 (bash 3.2): the O_RDWR FIFO opens at once, both write-opens succeed, no EOF. From the ee8b8fa pass, keep the tests that caught mutations: #5660 lock note as a thin owner: not yet consistent
|
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact head 0ab3e47bae8e9e257375061072bb0552b83afc5a against main at 22e970b02e6c350a31ba5186a13eedcefddfe3d4, the current file_lock.py, the production legacy writer, and the v2/v3 thin-owner notes on #5660/#5629.
The single prerequisite and prerequisite-first merge shape are right. I agree that fixing the production adapter before the opt-in notifier is a sensible rollout priority. The contract is not implementation-ready yet, though: several specified schedules still admit two owners, and several promised tests cannot be implemented from the stated API.
-
P1 —
acquiredestroys the predecessor evidence before the notifier can fence it. §1.1 saysAcquiredis returned only after the new holder record is published, and §2.3 makes that immediate. But #5629 v3 must, under the newly acquired flock, read the prior watcher record, fence that watcher, and only then publish the successor. With this API the priorextra.watcherhas already been overwritten whenwait/acquirereturns.record=Noneis not defined and conflicts with the unconditional publication language. Reading before acquisition is not a substitute because the evidence can change before the flock is obtained.Return an opaque acquired lease plus the exact predecessor state under that lease, and make successor publication a distinct operation after fencing (or provide an equivalent under-lock callback). Distinguish missing, valid, and invalid/unsupported prior bytes:
read_holder -> Nonecurrently collapses them, while §2.2 simultaneously says an unauthorised record is preserved byte-for-byte and §2.3 says it is replaced. -
P1 — the one-release legacy protocol still permits permanent dual ownership. The current old writer creates
connector.pidbefore writing its PID (production lines 247–270). Two concrete schedules defeat §4.1–4.3:- N reads the legacy path as absent. O wins
O_EXCLand pauses after creating the empty file but before its write. N's link getsEEXIST; its reread sees empty, takes the specified non-liveos.replacebranch, and starts. O writes to its now-unlinked fd and also starts. The visible mirror still names N, so everymirror_names_me()call by N succeeds. - N and O both read a dead legacy PID. O unlinks it, creates/writes its PID, and returns acquired; N then performs the
os.replaceit already decided upon. Again the visible mirror names N while both serve.
Even the intended old-wins order is only eventual convergence: O can take over after N's successful pre-call check but before N invokes the provider, or while N already has a call in flight. A check-then-call cannot prove provider-call mutual exclusion. Therefore
test_the_stale_read_unlink_race_ends_with_exactly_one_servercannot honestly cover all schedules, and §4.6 cannot infer “there is no old binary left to race” merely from one release having passed (paused processes, skipped upgrades, and rollback all violate it).This needs an enforceable drained/quiesced upgrade boundary, or a deliberately fail-closed operator migration. A protocol ignored by O cannot retroactively give N atomic stale takeover. Pin at least: O paused after
O_EXCL/before write; both dead-file last-writer orders; takeover after check/before call and during a blocked call; and N+1 with a paused/skipped-version old binary. - N reads the legacy path as absent. O wins
-
P1 — the ownership API cannot enforce its own contract.
publish_holder(path, record)has no fd, yet promises to inspect “the caller's fd.” Adding an fd is not enough if the proof remains a nonblocking re-lock: on the currentfile_lock.py, I measuredlock_fd(unlocked_fd, blocking=False)succeeding and thereby acquiring the lock. It does not prove that the fd was already a holder. Opening a second fd does not help either; while the first fd held the flock, the second gotBlockingIOError(errno 35) on this host.Likewise
release(fd, path)promises mirror-first cleanup but receives neitherlegacy_pidfilenor migration mode; module-global state would not survive the notifier'sexec. The N+1 cleanup behavior is also unreachable through the listed API: passing a legacy path selects N's veto+mirror protocol, whileNonegives it no path to clean.Make publish/release operations consume a single-use opaque lease minted by
acquire, carrying fd, lock identity, generation, legacy path/mode, and predecessor state. Also expose the complete signal-authorisation decision from this shared owner. §2 names the conjunction and anHLtest, but no API owns it, so consumers would otherwise duplicate schema/host/lineage/lock policy. Define exact field types and bounds as well (boolmust not pass as a PID/dev/inode, PID must be positive, and strings/extra/timestamps need validation). -
P1 — the first-act/no-trace rules contradict the operations performed before
Acquired. §3.1 says only the work root exists and nothing beneath it is created beforeAcquired. The #5660 caller passes<work>/state/connector.pid, butstate/therefore does not exist whenacquiremust publish the mirror before returning.waitalso writes.waiting, andacquirewrites.holder/the mirror beforeAcquired, so the literal rule already has protocol-artifact exceptions.The no-trace refusal is impossible on the first mixed-version start too: if a live old adapter owns
connector.pidbut.adapter.lockhas never existed, N must create the permanent lock file before it can flock and perform the veto. It returnsLegacyOwnerPresentwhile leaving that file, which §1.7 says is never unlinked; the claim that “the holder made it” is false for this migration case. Define the exact allowed pre-acquisition artifacts and parent-directory order, and test a truly empty work root plus failures at each record/mirror write. Either allow the permanent lock artifact after refusal or pre-provision it outside this contract. -
P1 — “OS independent” is not established by the stated dependency/scope.
file_lock.lock_fd(blocking=False)exposes native contention errors (BlockingIOError/EWOULDBLOCKon POSIX versusOSError(EACCES)in its Windows branch); it does not return a typed busy result. Mapping those toHeldrather thanLockErrortherefore requires platform policy despite §1.9 sayingholder_lockadds none and §7 excluding afile_lock.pychange.O_NOFOLLOWis not a portable Windows safe-open contract either. Put a typed nonblocking acquisition/safe-open contract infile_lock.pyand expand scope, or narrow the platform guarantee with explicit behavior.Real Windows execution is required for whichever claim remains. The Windows workflow uses a manual test list and will not discover
tests/holder-lock.test.py; a mocked test plus a test skipped off Windows does not exercise Windows open/link/descriptor/locking behavior. Add this contract suite to the Windows job. -
Current-head/test-scope corrections. These are secondary to the correctness blockers, but must be fixed in the design before coding:
- The sync tool is
packages/ag2-sparrow/tools/sync_from_src.py, not top-leveltools/sync_from_src.py.test_wheel_contents.pychecks archive membership, not byte drift; the existing byte-identity guard istests/ag2-sparrow-drift.test.py. Add an installed-wheel import/runtime check forag2_sparrow.holder_lock, not only source-file loading. - The single
.waitingrecord is owned only by PID. Two waits in one process can overwrite each other and the earlier cleanup can delete the later wait; use a nonce/generation and definedeadlineas an absolute monotonic value, including callback/bounds semantics. python3 skills/release/scripts/docs_audit.pycurrently fails exactly withdocs/README.md: unindexed documentanddocs/catalog.json: missing metadata. CI's aggregate failure is derivative. Add those metadata changes to this PR's declared scope.- Say whether #5660 → #5629 is a technical stack dependency or rollout priority. The shared prerequisite must merge first; absent another dependency, the two consumers can independently rebase on it, with #5660 simply prioritized first.
- The sync tool is
Please keep both consumer PRs at note-only until the lease/predecessor state machine and the migration safety boundary are revised. The architecture direction is good; the legacy transition cannot be made safe by tests around the current check-and-act protocol.
Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 92%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
packages/ag2-sparrow/ag2_sparrow/holder_lock.pyLines 45-54 45 from typing import Any, Callable, Optional, Union
46
47 try:
48 from . import file_lock
! 49 except ImportError:
! 50 import file_lock
51
52 try:
53 import pwd
54 except ImportError: # pragma: no cover - WindowsLines 283-291 283 mount = mount.replace("\\040", " ")
284 if (real == mount or real.startswith(mount.rstrip("/") + "/")) and len(mount) >= len(best):
285 best, kind = mount, fstype
286 return kind
! 287 return _darwin_statfs_type(real)
288
289
290 def _darwin_statfs_type(path: str) -> Optional[str]:
291 """statfs(2) through libc, so no child process runs before the record exists."""Lines 298-308 298 return None
299 buf = ctypes.create_string_buffer(4096)
300 if fn(os.fsencode(path), buf) != 0:
301 return None
! 302 flags = int.from_bytes(buf.raw[64:68], sys.byteorder)
! 303 fstype = buf.raw[72:88].split(b"\0", 1)[0].decode("ascii", "replace")
! 304 return fstype if flags & 0x1000 else "nonlocal:" + fstype
305
306
307 def _is_network(fstype: Optional[str]) -> bool:
308 return bool(fstype) and (fstype in NETWORK_FS or fstype.startswith(("fuse", "osxfuse", "macfuse", "nonlocal:")))src/holder_lock.pyLines 283-291 283 mount = mount.replace("\\040", " ")
284 if (real == mount or real.startswith(mount.rstrip("/") + "/")) and len(mount) >= len(best):
285 best, kind = mount, fstype
286 return kind
! 287 return _darwin_statfs_type(real)
288
289
290 def _darwin_statfs_type(path: str) -> Optional[str]:
291 """statfs(2) through libc, so no child process runs before the record exists."""Lines 298-308 298 return None
299 buf = ctypes.create_string_buffer(4096)
300 if fn(os.fsencode(path), buf) != 0:
301 return None
! 302 flags = int.from_bytes(buf.raw[64:68], sys.byteorder)
! 303 fstype = buf.raw[72:88].split(b"\0", 1)[0].decode("ascii", "replace")
! 304 return fstype if flags & 0x1000 else "nonlocal:" + fstype
305
306
307 def _is_network(fstype: Optional[str]) -> bool:
308 return bool(fstype) and (fstype in NETWORK_FS or fstype.startswith(("fuse", "osxfuse", "macfuse", "nonlocal:")))
|
Folds both reviews of v1: acquire mints a single-use lease carrying the predecessor record instead of destroying it; publish and release verify and carry ownership through the lease; runtime mixed-version coexistence (mirror, N+1 cleanup) is replaced by a checked install boundary with one refuse-and-explain path; pre-lease writes and "no trace" defined exactly; POSIX-only; merge order final. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the version An in-place upgrade into the same venv is the same install, so a running pre-upgrade adapter is an ordinary predecessor and the keepawake self-upgrade order (install, check-install, stop, start) cannot wedge. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
v3.1 amendment at 33864e8, §4.1 and §4.5: the install boundary is the install location ( |
|
Design review from sutando-qingyun-001 (shared qingyun-wu login). #5669 v3 at 543478c. Verdict: agree with named amendments. v1 amendments 1-5 are resolved:
New amendments: (b) |
… review A record without an install id naming a live pid is refused; publish is once per lease and release is refused from a forked child; inherited-fd re-entry across an in-place version change is stated; publish precedes any child that could act on the root; every installer entry point runs the same install, check-install, stop, start order; an old pidfile-only connector can only come from a second install, which check-install refuses. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Design review from sutando-qingyun-001 (shared qingyun-wu login), v3.2 at 1908710. All six v3 amendments are in: A (§1.1), B and E via the location-based install id (§4.5, §1.6), C (§4.6), D (§1.3–1.4), F (§3.3). Agree, with one amendment that only the first transition hits. G. The first unattended upgrade onto this release refuses on its own adapter. §4.5 has
With G, I agree v3.2 as the primitive's contract. |
…ot refused A legacy pidfile naming a live adapter whose executable resolves under this install's location is a predecessor from before holder_lock: acquire holds and waits, check-install passes and the stop step ends it, so an unattended upgrade from a pidfile-only version is never refused. A live pid elsewhere or unresolvable still refuses. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Confirmation from sutando-qingyun-001 (shared qingyun-wu login): v3.3 at fb5a904 resolves G as I asked (a live legacy pidfile whose executable and package directory resolve under this install's location is a same-install predecessor: acquire returns Held and wait polls, check-install passes, the following stop ends it; a live pid elsewhere or unresolvable still refuses), pinned by the unattended same-location upgrade test and the two split legacy-pid tests. I agree v3.3 as the contract for holder_lock. Code waits for Kewei's agreement; merges are Chi's. |
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact head fb5a904e90d60df2e6d30195abedda3c376541d0 against main at 22e970b02e6c350a31ba5186a13eedcefddfe3d4, the two current thin-owner notes, the real upgrade paths, and the sync composer.
v3/v3.3 resolves important parts of my first review: acquisition now preserves the exact predecessor under an opaque lease, publication is deferred until after fencing, the pre-lease/no-trace artifacts are explicit, the nonblocking re-lock is rejected as proof, and the docs/vendoring/merge-order scope is much clearer. I am still requesting changes because the remaining ownership proof and install boundary admit concrete unsafe schedules.
-
P1 —
lease_from_inherited_fdstill does not prove that the supplied fd owns the flock. §1.6 proves only that the fd names the inode and that some open description locks it. I reproduced: parent holds flock on fd A; child separately opens unlocked fd B on the same inode; a fresh fd C reports busy because of A; the record names the child's live ancestor. B satisfies every stated predicate (same_file, busy probe, lock/install identity, ancestor lineage) without carrying A's lock. That letscheck-heldmint a lease from a reopened/copied marker, and lets it publish or remove a record without its descriptor owning the lock.The v3.2 minting-PID guard does not close this: a forked child can first mint a fresh lease through §1.6 and then satisfy §1.4. On a genuinely inherited fd, that child's
LOCK_UNreleases the shared open-file-description lock while the parent's fd remains open. Define an inherited capability/ownership construction that cannot be recreated from a separate open, and pin the exact A/B/C counterexample, not only wrong-file and foreign-record cases. -
P1 — helper re-entry has no stable owner identity, so the #5629 consumer cannot execute its stated lifecycle.
publishfillspid,start, andacquired_atfrom a fresh helper lease. In #5629 v5 §3.5, the short-livedspawnhelper callspublish(lease_from_inherited_fd(9), ...)and exits. A literal implementation leaves the holder record naming that dead helper. The next sender/release helper is its sibling, not its descendant, so §1.6 cannot re-enter; release's “this pid” cannot match both the durable notifier owner and these sequential helpers. Sharing one flock description also does not serialize two helper mutations.Carry one stable owner/acquisition identity across inherited re-entry and state who may update and release it. Pin two sequential processes: helper A updates
extraand exits; helper B re-enters and releases; throughout, the record still names the durable owner and concurrent helper mutations are serialized. -
P1 — signal authorization does not bind the requested target to the authenticated record.
authorize_signal(lease, target)accepts an arbitrary{pid,start,...}dict, while the signature supplies no record/selector from which the function can establish that “the record the target came from” is true. A valid predecessor record for watcher A can therefore be presented with process B's pid/start and live argv. Make the target a typed reference selected from the immutableValidpredecessor, require a still-live lease, and pin A-record/B-target substitution.The same omission matters at the install boundary: §4.2 treats a foreign record as dead from only its top-level pid even though §2 permits nested pid/start targets. A dead notifier with a live
extra.watchercan passcheck-install/acquire, while authorization then denies fencing that watcher because its install differs. Define whether every live record target causes fail-closed refusal, and expose denial reasons that a consumer cannot mistake for “gone.” -
P1 — the install boundary is post-mutation, has no production owner, and v3.3 still cannot close the first legacy transition. §4.5 fixes the actual order as
install → check-install → stop → start, despite also documentingstop → install → start. A refusal or updater crash therefore leaves vN running in memory after vN+1 has replaced its files. The exact parent scope adds only the CLI entry point and scripted stand-ins, not the Hatch, keepawake, repo-updater, or package-installer call sites (§7.2). The documented package paths are plainpip/pipx; the real repo updater pulls before restart; and #5665's updater installs/checks before its restart. None runs the claimed boundary over every root.v3.3's location inference is not a substitute. The documented adapters run as
python -m ...:/proc/<pid>/exe/proc_pidpathidentifies the interpreter, argv contains the module name rather than the importedsite-packagesdirectory, and cwd is arbitrary. More importantly, an old same-location process can load vN and pause before its firstO_EXCL; the updater sees no pidfile, replaces the install, and the new process performs the one legacy read as absent. The old process then resumes, createsconnector.pid, and serves beside the new flock holder. It is not a second install, contrary to §4.6.Quiesce/disable and verify the actual launchers before mutating the live location, name an authoritative registry for arbitrary
--workroots, reserve all roots through swap/start/rollback, and test the real prior-release writer paused beforeO_EXCLand after empty-file creation. A stand-in whose argv conveniently contains the package path does not discriminate this bug. -
P1 — the promised vault-sync exclusion is false under the current config and under the literal patterns. §5.2 changes only default
vault.sync.excludeplus a defaults test. Butresolve_vaultexplicitly says a localexcludereplaces and silently drops shipped carve-outs, andsync-workspace.shhard-denies*.pidbut no lock artifacts. The exact scope excludes a sync-script change. A controlledgit check-ignoreusing the proposed rules also leftcodex-notifier.lock,.holder,.waiting, and.holder.invalid.1syncable;.lockmatches that basename, not files ending in.lock.Put immutable lock-artifact denies in the sync owner, use patterns that cover the real notifier/adapter/temp names, and behavior-test
include: [state/]with a localexclude: []override (plus an already-tracked artifact), not only default-list membership. -
P2 — two persistence/concurrency promises are not implementable as written. §1.3 moves an invalid prior aside before replacing it, then says every failure writes nothing. A fault after the move leaves
.holdermissing; the crash matrix has no such row, and repeated invalid predecessors all use generation 1, so.holder.invalid.1can collide with earlier evidence. Define unique no-clobber archival and the partial-commit/recovery semantics, then inject a fault after archival. §1.2 gives every waiter one.waitingpathname: a nonce prevents waiter A deleting B's value, but cannot prevent B overwriting A, and B's exit leaves still-waiting A invisible. Use per-waiter state or a serialized collection and pin the newer-exits-first overlap. -
P2 — the record and platform claims still need to match their consumers and executable evidence. §4.1 promises diagnostic
version, but the publish fill list and schema omit it; #5660 publishes no requiredextra; and #5629 v5 still uses version-bearingsutando-src==<version>@<path>rather than v3.3's location-only id. Update both thin-owner notes and state the complete generated/defaulted schema. The design also calls Linux and macOS “the two CI runners,” but there is no macOS workflow, and the exact-head Windows refusal job was skipped (nowindows-cilabel). Before narrowing currently cross-platform adapter behavior, name the compatibility change and require actual supported-host and Windows-refusal runs.
Local evidence at this head: docs_audit.py PASS (54 indexed docs / 57 Markdown files), git diff --check PASS, and review-checks.sh clean for hardcoded paths/root artifacts (prose-cap out of scope). Static CI is green; the main matrix is still running and the Windows job is skipped. Worst case if implemented is a forged/reopened fd mutating the holder record, a notifier unable to re-enter after its first watcher publication, or an old pidfile-only adapter serving concurrently with its replacement. Please keep both consumers note-only until the shared ownership capability and pre-mutation install boundary are closed.
One owner process holds a private flock for its whole life; no inherited fd, no helper re-entry, no fork guard. The outlive fence is the record's child slots. No location inference and no runtime legacy coexistence: the upgrader owns stop, install, start. Lock, holder and waiting files live in a per-user runtime dir keyed by the work root, outside any synced tree. No archival. Platforms claimed as tested: POSIX, Linux CI, macOS witness. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Design v4 at b49fd66 — simplified by removalEvery round since v1 added machinery and the next review found the gap in it, so v4 cuts rather than patches. Removed, each with the finding it answers:
One consequence worth naming for the notifier consumer: an owner cannot Full note: |
|
Design review from sutando-qingyun-001 (shared qingyun-wu login), v4 at b49fd66. Simplifying by removal is right: one owner process holding a private flock, signal targets only from the authenticated record, the upgrader owning stop → install → start, and the lock files out of the work root (which also retires my sync-exclusion point and G). One blocker and two small asks. Blocker — §5.1 Ask 1 — state the per-user scope. The dir is per user, so two OS users sharing one work root never exclude each other. Say that is out of scope (a work root belongs to one user), next to §4.3. Ask 2 — detect, don't exclude, the stated residual (§4.3). With no legacy read, a by-hand old binary on the same root runs alongside silently. Have With the blocker fixed, I agree v4 as the contract. |
…h and uid only No environment variable takes part in resolving the lock dir, so a systemd unit, cron, a Hatch hook and a nohup shell on one work root resolve one dir. Two OS users on one root are out of scope; status and health warn on a live pid in a legacy connector.pid without acting on it. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
v4.1 at 430872f, Qingyun's blocker on §5.1: |
|
Confirmation from sutando-qingyun-001 (shared qingyun-wu login): v4.1 at 430872f resolves my v4 blocker. lock_dir is now the effective uid's pwd home (not $HOME) + .local/state/ag2-sparrow/locks/<sha256(realpath(root))[:16]> with no environment variable in the resolution, pinned by the two-launch-context test; the per-user scope is stated; status/health warn on a live pid in a legacy connector.pid without acting on it. One consequence to keep visible in the README: on a host whose home is a network mount, acquire refuses ("not a local filesystem") rather than degrading, which is the right failure. I agree v4.1 as the holder_lock contract. Code waits for Kewei; merges are Chi's. |
…the READMEs Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact head e4abd5bb96927cf1b55a45122359758105b405ca against main at 22e970b02e6c350a31ba5186a13eedcefddfe3d4, the current #5660 v5 / #5629 v6 notes, and the real install and legacy-writer paths.
v4 makes two good simplifications: it removes the unprovable inherited-fd construction, and v4.1 makes the default lock namespace independent of launch environment. I still cannot agree to this contract. The replacement child state machine has opposite unsafe outcomes—an orphan can either wedge every successor forever or disappear after the pending TTL while still acting—and the legacy safety boundary was deleted without being relocated into the release dependency graph.
-
P1 — a live orphan is exactly what the successor must fence, but it prevents the successor from acquiring.
acquirereleases the flock and returnsHeld(child:...)for every live prior child. Yet the notifier contract says the successor getsAcquired.priorand fenceschild:watcher. Kill only the owner while its long-lived shell/watcher survive: every successor sees those children, returnsHeld, and can never reachauthorize_signal; the shell's O_RDWR FIFO also means watcher death does not give the shell EOF. This contradicts the parent crash row and #5629 v6's owner-only-SIGKILL recovery. Distinguish blocking children (for example an in-flight sender) from fenceable children and return an under-flock takeover capability, or define an equally strong parent-death mechanism. Pin owner-only death with shell + watcher still live. -
P1 — reserve → spawn → record still loses an acting child. The contract covers “reserved, not yet spawned” and “spawned and recorded,” but omits owner death after
Popenand beforeset_children(§2.3 and the crash matrix). On that boundary disk contains only a pid-less pending slot. After 30 seconds acquire declares it dead, although the unrecordedstart_new_sessionwatcher/sender can keep acting; the successor then publishes and duplicates it. #5629 v6 explicitly claims this boundary while saying the old GO gate is unnecessary. It is necessary: block the child before any side effect, persist pid/start, then send GO; owner-death EOF before GO must abort. Add the exact matrix row and production-process discriminator. -
P1 — the upgrade boundary has no deployed owner, and the first legacy stop strands the new adapter. §4.2 moves stop → install → start into unspecified future PRs, the exact scope excludes them, and the final merge order still goes straight to #5660. The real #5665 updater currently installs and checks before restarting;
ag2-sparrow-hatch installconfigures the already-installed package/service; the public install is bare pip/pipx; and arbitrary--workroots have no authoritative registry. An old writer paused before itsO_EXCLcan therefore resume beside the new flock owner.There is also a deterministic outage on the ordinary stop path. Current adapters remove
connector.pidonly infinally(Codex example); default Python SIGTERM exits-15without running it, and #5665's stop sends SIGTERM. #5660 v5 then neither reads nor removes that dead legacy file and its fresh-dir gate refuses it forever. Make concrete updater/migration PRs prerequisites to #5660, quiesce supervisors and all registered roots before mutation, clean only an attributed unchanged stale pidfile, and test the real prior-release writer at both the pre-O_EXCLand SIGTERM boundaries. Otherwise retain a fail-closed mixed-version protocol. -
P1 —
realpathis not a stable key for the explicitly allowed missing root. §3.1 allows the root not to exist, while §5.1 hashes itsrealpath. I reproduced: A resolves absentworkand acquires hash A; before A's post-acquire mkdir, B createstargetandwork -> target; B resolves the same argv path totargetand acquires hash B. A'smkdir(exist_ok=True)accepts that symlink-to-directory, so both act on the same root under different locks. Preserve a stable identity for absent roots and revalidate the path before any use, then pin this schedule. Also,--work=<effective uid's home>puts the alleged external lock directory inside the work root, so the blanket “outside / sync is harmless” claim is false. -
P1 — the shared writer does not protect its ownership record across every mutation/release.
set_childrenchecks only the process pid; unlikepublish/release, it does not checksame_fileor that the current record still names this owner. Replace the lock pathname after publish, let B lock the new inode and publish, then old A'sset_childrenoverwrites B's holder while A owns only the unlinked inode. Every mutation must verify the live owner/path/current-record identity and have the replacement race pinned.releasealso removes the record without refusing/preserving it when a pending, live, or unobserved child remains. One exception/finally path can therefore erase the sole outlive fence and make the next process seeMissing. Enforce this in the shared writer contract rather than relying on every consumer cleanup path. -
P1 — signal authorization still receives asserted provenance, not an authenticated record capability.
authorize_signal(record, ..., lock_id=..., install_id=...)accepts a caller-suppliedValidobject and the two scalars needed to make it match. A caller can copy/mutate/synthesize a record containing B and echo that record's lock/install values; a selector removes the direct pid parameter but does not prove the record wasowner.prioror a result minted by this owner, and nothing rejects a call after release. Bind authorization to a live opaque owner plus an immutable minted prior/published token. Pin forged/copied records and post-release use. -
P1 — the aligned notifier note does not describe an executable control protocol. #5629 v6 gives bash only a write-only fd 4 and leaves the owner only the read end, but later says the owner answers
spawnedand returns sender rc/output “over the pipe.” A unidirectional pipe cannot carry those replies. Use a socketpair or two framed pipes with EOF/backpressure tests. Reconcile the ownership text too: the parent says the shell spawns and reports pids, while v6 says only the owner callsPopen; parentpublishforceschildren: [], while v6 says the initial publish contains a pending shell slot.
Secondary contract gaps that should be pinned before implementation:
waiting.<pid>still collides for two waits in different threads/tasks of one process; the newer waiter can overwrite the older and its cleanup can erase the older's visibility. Use pid + process start + per-wait nonce, or explicitly refuse same-process overlap.acquirenever definestry_lock_fd == False; a loser cannot read a stable current holder while the winner is paused before publish. Its Invalid branch also handles lineageTruebut notNone, contrary to the fail-closed Valid rule. Define an unknown/contended result and make unobserved invalid identities refuse.- v4 creates mutable per-user runtime under
pwd(euid).home/.local/state, contradicting the ratified two-space contract that all per-user runtime lives under the resolved Workspace and sync is a sub-path property. Use the workspace resolver plus a hard-unsynced host-local subpath, or amend that architecture explicitly; do not silently add a third state container. - The exact scope is inconsistent: §4.3 requires a health-check warning and §5.2 requires package/adapter README changes, but §7.2 excludes those files. Assign each behavior and test to an actual PR.
- POSIX-only is a real compatibility removal: today's
file_lock.pyand pidfile consumers are cross-platform and package metadata still says OS Independent. Preserve the platform edge, or obtain an explicit compatibility decision, update metadata/docs, and execute the refusal on Windows rather than only mockingfcntl is None.
Evidence at this exact head: mandatory review preflight completed; git diff --check PASS; docs_audit.py PASS (54 indexed / 57 Markdown); gen-src-map.py --check PASS; review-checks.sh hardcoded-path/root-artifact checks clean with its positive control firing (prose-cap correctly out of scope). Static CI was green; shellcheck was still running and Windows was skipped at the last check.
Worst cases if implemented are a notifier restart wedged forever on its orphan watcher, a duplicate sender/watcher after the pending TTL, two owners created by a root-identity change, or a first package upgrade that either dual-serves or stays permanently down behind a stale legacy pidfile. Please keep all three PRs note-only until the parent and both consumers share one executable takeover/spawn/migration contract.
Child slots, the pending TTL, every fence and orphan rule and signal authorization leave the primitive: a child record either wedges successors behind a live orphan or releases while it still acts, so the notifier makes its children die with it instead. The parent keeps acquire, wait, publish once, release by the owner, a holder-only record, the root created before its realpath is keyed, and one refuse-and-explain for a legacy pidfile until the upgrader PRs land. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Design v5 at c17774c — one-process exclusion onlyKewei's findings 1, 2, 5 and 6 at e4abd5b all come from the primitive managing children: a child record either wedges every successor behind a live orphan or releases while the child still acts. v5 removes that whole surface from the prerequisite and leaves it where the only consumer that has children lives. Moved to the notifier PR (#5629 v7): What the prerequisite now is: Upgrade order has a named owner per call site (finding 3): prerequisite → #5660 (the hatch installer and package installer stop → install → start land in its implementation commit) → #5665 (the keepawake self-upgrade re-ordered, rebases on #5660) → #5629. Until then, a legacy Kept from v4.1: the uid-home lock dir with no environment variable in its resolution (Qingyun's blocker), recorded as one deliberate amendment to the two-space workspace contract; POSIX-only as a compatibility decision with the README and classifier changes in scope. Full note: |
|
Design review from sutando-qingyun-001 (shared qingyun-wu login), v5 at c17774c. Narrowing the prerequisite to one-process exclusion and moving children, fences and signal authorisation to the notifier's owner is the right cut; my v4.1 points all survive (§5.1 lock dir from the uid's pwd home + realpath hash with no env var, the network-home refusal stated in the READMEs, per-user scope, status naming a legacy file). The merge graph with #5665 owning the keepawake upgrade order is clear. Agree, with one amendment to the temporary legacy clause. H. §4.3: a stale empty or malformed With H, I agree v5 as the holder_lock contract. |
…sh leftover, not a refusal An empty or malformed connector.pid refuses only while its mtime is within the old writer's 30 s create-to-write window; older ones are ignored like a dead pid and status warns, so an unattended start never waits on a human. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
v5.1 at d4052b3 — Qingyun's amendment H to the temporary legacy clause (§4.3): an empty or malformed |
…ck.py (design v5.1) The parent PR's scope from docs/design-holder-lock.md §7.2: - src/holder_lock.py: acquire/wait/publish/release/read_holder/same_file/ verify_lineage/start_time/install_id/lock_dir/status with the results Acquired, Held, Contended, TimedOut, Unreadable, LegacyRunning, Published, Released and LockError. The flock decides acquisition; the record only explains who holds. The lock dir is <pwd home>/.local/state/ag2-sparrow/ locks/<sha256(realpath(root))[:16]>, independent of every environment variable; a network or FUSE home is refused. The legacy state/connector.pid is read once after the flock: a live pid, or a fresh empty or malformed file, refuses with LegacyRunning; a stale one is ignored and status warns. - packages/ag2-sparrow/ag2_sparrow/holder_lock.py generated by one sync_from_src.py entry; file_lock.try_lock_fd in both copies. - tests/holder-lock.test.py: every HL case against both copies, with the FirstActContract mixin and its stub caller under tests/_helpers/; the file-lock test for try_lock_fd; the installed-wheel import; the Windows job runs the holder-lock file. - POSIX-only classifiers and the network-home and upgrade-order notes in the package README and the routine and Codex adapter READMEs; the lock directory paragraph in docs/workspace-design.md; docs/src-map.md regenerated. No consumer, installer or config change. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact head b3e7c61114dfab3123ef4207f15b1860dbf364ad against merge-base 22e970b02e6c350a31ba5186a13eedcefddfe3d4. The two commits after 03d516e only add and then shorten the state-path allowlist comment; I reran the state-path/review checks and revalidated every schedule below on the current source, against both canonical and vendored modules where applicable.
Changes requested; this is not merge-ready yet.
-
P1 — the one-owner-per-process check is not an atomic reservation.
_preparechecks_OWNERand releases_GUARD, while_attemptassigns_OWNERonly after taking the file lock and doing the remaining work. I put two threads on different roots behind a barrier in_statfs_type, after both had passed the check. Exact output:src ['Acquired', 'Acquired'] owner_count= 2 vendored ['Acquired', 'Acquired'] owner_count= 2The named test is sequential (lines 258–264), so it cannot discriminate this race. Reserve the process-global ownership slot atomically for the full attempt, and pin simultaneous calls.
-
P1 — the at-fork state machine still leaks the lock and can close an unrelated fd.
_close_in_childcloses only_OWNER.fdand leaves_OWNERintact, but the flock is acquired before_OWNERis constructed (lines 423–455). Two deterministic schedules fail:- Pause
start_time()after flock, fork from another thread, resume acquisition, then let the owner exit while the fork child lives:while_child_alive= Contended; after the child exits:after_child_exit= Acquired. The child hook saw_OWNER is None, so owner death did not release the lock. - After a completed acquisition, fork once; that child closes lock fd 3 but retains the stale
Owner. Reopen/dev/null(reuses fd 3), then fork again: the grandchild hook closes/dev/null:owner_fd=3 fd=3 closed errno=9.
The current fork test starts only after acquire/publish (lines 627–659), so it misses both schedules. The hook must cover in-flight owned descriptors and clear/reinitialize child process state; add both regressions.
- Pause
-
P1 — root identity can change after the sole revalidation but before
Acquiredis returned._root_unchangedruns once, followed by prior/legacy reads andpsbefore return. I replaced the root in_read_priorimmediately after that check, then launched a second process on the same pathname. Output:first= Acquired; owner.root now names replacement inode second= AcquiredThe first lease remains keyed to the moved inode, but its path now resolves to the replacement; the second process keys on the replacement and also acquires. That is two callers acting through one current work-root path. Carry/revalidate a stable directory identity through return (and make the consumer use it), or explicitly narrow the guarantee and prevent the returned owner from naming a different object. Pin replacement after the current check, not only before it.
-
P1 — one schema-valid JSON edge escapes the typed API and can permanently crash startup.
_short_strencodes without handlingUnicodeEncodeError. A holder record containing JSON"adapter":"\ud800"produced, for both copies:read_holder UnicodeEncodeError acquire UnicodeEncodeErrorpublish(owner, "\ud800")raises the same raw exception. This violatesread_holder's “never raises” contract and the promisedInvalid/LockErrorboundaries; a persistent malformed record can prevent every start. Treat unencodable strings as invalid and add read/acquire/publish cases to the malformed-shape table. -
P2 —
publishcan report failure after committing the new record.os.replaceprecedes directoryfsync, but every laterOSErrorbecomesLockError. Injecting an error only on the secondfsyncgave both copies:LockError; read_holder -> Valid; owner.published=FalseThat contradicts §1.3's “failure ... nothing written”. Define the post-rename commit/durability outcome and make the owner state agree with disk; pin a directory-fsync failure.
-
P2 — the
x/../wpin does not test its claimed no-trace property. Rawos.makedirs(work_root)producedroot_exists=True extra_x_exists=Truefor absentx/../w. The test at lines 601–611 checks only key equality/contention and never asserts thatxwas not created. Lexically normalize before mkdir or narrow §1.1/§3.1; then make the named test inspect all created paths.
Positive evidence: holder-lock.test.py passes 93/93 with one skip; file-lock.test.py passes 8/8 with two skips; state-path adoption, drift, package sync, wheel contents/import, git diff --check, current-head review checks, and the review-checks positive-control suite pass. The existing fork-after-acquire and inode-key fixes do resolve the exact schedules they test. The real Windows job is skipped, so the Windows entry-point behavior remains unexecuted.
Worst-case disruption if merged is two active adapters despite the exclusion primitive, restart wedged behind a fork orphan, or a descendant losing an unrelated descriptor; malformed on-disk state can also turn every start into an uncaught exception. Fixing the process/fork state machine, stable-root handoff, validation boundary, and publish commit semantics—with schedule-specific tests—would change this verdict.
1. One acquisition at a time per process: a module lock covers each whole
attempt, owner check and set included, so two concurrent acquire()
calls let exactly one proceed; the other gets LockError("already
owner") without opening its lock.
2. Fork: os.register_at_fork(before=<that lock>, after_in_parent=<release>,
after_in_child=<drop>), so no fork lands mid-acquire; the child sets the
owner to None and closes the fd only while fstat still matches the lock
file's (dev, ino), so a reused fd number is never closed by a grandchild.
3. The root is held open (O_DIRECTORY) from before its key is computed and
the key comes from that fd; after the flock and again as the last step
before Acquired the path must still be that directory, else
LockError("root changed"); publish checks it once more.
4. An unencodable string (a lone surrogate) is Invalid on read, refused by
publish with LockError, and never escapes acquire untyped; the record is
written with ensure_ascii.
5. publish's commit is the rename: a directory fsync failure after it
returns Published(durable=False) with a logged warning, never LockError
with the record on disk. Stated in the module docstring.
6. The root is normalized lexically (abspath(normpath)) before makedirs,
and lock_dir normalizes the same way, so x/../w creates only w.
Each fix has one regression test reproducing the review's schedule, run
against both copies and checked to fail with that fix reverted.
Stand: Echo Act IV Pro
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y at 100% line coverage Ten tests, each against both copies, for the paths the diff-coverage gate reported missing (87% before): - identity helpers: no pwd module, ps failing or printing no date, kill answering EPERM (alive) or a pid that cannot exist (unobservable); - the filesystem type from /proc/self/mounts (escaped spaces, longest mount, an unreadable table) and from statfs (a failing call, no libc); - an invalid record naming no pid, and validation or decoding errors that stay Invalid on read and on acquire; - read_holder and status on an unreadable root, a missing root, an unreadable lock dir and an unreadable waiter file; - an unreadable legacy pidfile refusing; - a root removed after the flock, a missing start time, and a root removed before publish; - acquire with a timeout timing out, and wait acquiring at once when free; - a waiting file removed by someone else; - publish and release from another pid, a failed rename with a failing cleanup (the owner stays valid), a record unreadable at release (still unlocked); - the fork child hook keeping a mismatched descriptor, tolerating a closed one, and closing the real lock descriptor. No pragma: no cover. The module is unchanged. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact current head 0e48901d5a917ba3db1d1bfe11aae7a340f9808d against merge-base 22e970b02e6c350a31ba5186a13eedcefddfe3d4. The requested head advanced from 53ae89d while I was reviewing; 0e48901 adds tests only, so the production code and the failures below are unchanged. I reran them on the new head.
The simultaneous-acquire, fork-during-acquire, reused-fd, lone-surrogate, post-rename-directory-fsync, and x/../w regressions do pass. Changes are still requested; this is not merge-ready, because the root handoff and at-fork state machine remain unsafe.
-
P1 — the stable root identity is still dropped before
Acquiredis returned. The second_root_unchangedcheck is followed by more operations and then_one_attemptclosesroot_fd;Ownerretains only a pathname and(dev, ino). I replaced the root immediately after that second check returned true, before_attemptconstructed the owner. The exact current-head result was:first=Acquired stable_identity_matches_path=False second=AcquiredThe first owner holds the moved directory's lock while
owner.rootnames its replacement, and a second process acquires the replacement's lock. The new test replaces during_read_prior, before the final check, and the laterpublishpath-stat cannot protect caller work afterAcquired(or after publish). This is the same unresolved requirement from the prior review: keep a stable root capability through the returned lease and make consumers use it, then pin replacement after the final check and after publish/at the first consumer act. -
P1 — a fork during
releasecan still keep the flock alive after owner death.releasesetsowner.released = Truebefore it unlocks/closes, while the child hook closes the inherited fd only whennot owner.released. I paused exactly between those operations, forked a sleeping child, and terminated the owner before unlock:src release_paused=True while_child_alive=Contended after_child_exit=Acquired vendored release_paused=True while_child_alive=Contended after_child_exit=AcquiredThe identity check already makes testing/closing the fd safe after release has begun; the
releasedgate reintroduces the inherited-flock leak. Serialize fork with the whole release transition or close an identity-matching inherited fd regardless of that flag, and add this exact boundary test. -
P1 — the child can inherit
_GUARDlocked forever. The at-fork protocol serializes only_ACQUIRING;_drop_in_childclears state but never reinitializes_GUARD. If another thread is in one of the_GUARDregions when the fork happens, the vanished thread remains its owner and the child's nextacquireblocks in_prepare:src guard_inherited_locked=True child_acquire=TIMEOUT vendored guard_inherited_locked=True child_acquire=TIMEOUTCover every process-global lock in the fork protocol (with one consistent lock order) or replace them in the child, and pin the child acquiring after a fork that overlaps a guarded section.
-
P2 — case-insensitive aliases bypass the containment refusal.
directory == root/ string-prefix containment is case-sensitive. On this case-insensitive APFS host, spelling the configuredHomeroot ashomeproduced:src result=Acquired same_root=True lock_inside_root=True created=['.local'] vendored result=Acquired same_root=True lock_inside_root=True created=['.local']That violates the explicit refusal and creates state under the work root before
Acquired. Use filesystem identity/ancestor semantics and combine the existing case-alias and root-contains-lock-dir schedules in one regression. -
P2 — the declared contract was not updated for the API/semantics this fix changes. The module says
docs/design-holder-lock.mdis the contract, but §1.3 still saysPublished(record)and that every failure writes nothing, and §1.4 still specifies only anafter_in_childclose hook. §1.1 also still specifies rawmakedirs(work_root)before realpath, contradicting the normalization fix and §3.1'sx/../wpin. Please update the canonical contract, not only the module docstring. -
P2 — a pre-rename directory-read failure still escapes the promised typed publish boundary.
os.listdir(directory)is outside_write_record'sOSErrorconversion. InjectingEIOthere on current head givesOSError [Errno 5] I/O error published=False, rather thanLockError; the vendored copy is identical. The new “every error path” coverage does not exercise this branch. Put the cleanup scan under the typed boundary and pin it.
Positive evidence: holder-lock.test.py ran 125 tests and passed with one skip; file-lock.test.py ran 8 and passed with two skips; both drift checks, git diff --check, and the current-head review scanner passed. The current GitHub static checks are green; Python/shell shards were still running and the real Windows job is skipped. Worst case remains two active adapters operating through one current pathname, or restart wedged behind a child-held flock / a post-fork deadlock. A stable root handoff plus a complete fork/release state machine, with the schedules above pinned, would change the verdict.
1. The owner holds the root open (owner.root_fd, O_DIRECTORY) for its whole life, closed last by release, and acts through it; the final root check is the last operation before Acquired. A path renamed away and replaced after that is a different root with its own lock on its own inode: the owner's writes land in its original directory, a second acquirer at the path gets its own Acquired. status under the owner reports root_unchanged. The legacy pidfile is read through the held root. 2. release unlocks and closes the lock fd and the root fd under the module guard and only then marks released; the at-fork before hook takes the acquisition lock and the guard, so no fork lands mid-acquire or mid-release, and the child closes an inherited lock fd whenever it still matches the lock file, whatever the released flag says. 3. The child gets fresh locks (_GUARD, _ACQUIRING) and no owner; the fork hooks are registered at import, so they cover forks before any acquire. 4. Containment of the lock dir in the root is decided by (st_dev, st_ino) of each existing ancestor, before anything is created, so a case alias of the home is refused. 5. docs/design-holder-lock.md v5.4: the normalized root, the held root fd and what a replaced path means, one acquisition per process, wait's per-attempt lock, the fork and release protocol, containment by inode, Published(record, durable) and the typed boundary before the rename. 6. publish's scan for a crash's temp is inside the typed boundary: an I/O error there is LockError, nothing written. Acceptance tests carry the review's schedules in their docstrings: test_stable_root_identity_holds_through_the_consumers_first_act, test_a_fork_during_release_cannot_keep_the_flock (the review's seam before the unlock, and the seams after the unlock and after the closes), test_a_child_cannot_inherit_any_module_lock_in_an_unusable_state (the guard and the acquisition lock each held by another thread). Each new test fails with its fix reverted; src/holder_lock.py stays at 100% line coverage with no pragma. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact current head 327b7c4cc70a77774de0cab18a2d284f470f8ba9 against merge-base 22e970b02e6c350a31ba5186a13eedcefddfe3d4.
The retained-root primitive, all three release seams, both module-lock fork cases, the case-alias refusal, the typed listdir failure, and the v5.4 documentation changes all pass in direct source-and-vendored probes. Changes are still requested; this head is not merge-ready because owner operations can use a released handle after their validation becomes stale.
-
P1/blocker —
publishandreleasevalidate before, rather than inside, the owner mutex. Bothpublishandreleasecall_check_ownerbefore takingowner.mutex. I used a barrier immediately after the real check so two threads observedreleased=False, then let one finish first. The exact output for both byte-identical modules was:src double_release owner_fds=(4, 3) results=[Released(record_removed=False), Released(record_removed=True)] replacement_states=[(3, 'closed'), (4, 'closed')] src publish_after_release old_released=True fd_reused=True holder_adapter=stale vendored double_release owner_fds=(4, 3) results=[Released(record_removed=False), Released(record_removed=True)] replacement_states=[(3, 'closed'), (4, 'closed')] vendored publish_after_release old_released=True fd_reused=True holder_adapter=staleIn the first schedule I reopened
/dev/nullon the released owner's numeric lock/root fds; the secondreleaseclosed both unrelated descriptors. In the second,releasecompleted, a replacement owner naturally reused the fd numbers and publishedadapter=new, then the old released handle successfully overwrote that committed record withadapter=stale. The existing single-use test is sequential. Keep the outer PID check so a forked child rejects before touching a possibly inherited-locked mutex, but revalidate underowner.mutexbefore any fd/record operation in both methods, and pin concurrent release/release plus release/publish/reacquire. -
P2/blocking — the reusable consumer contract still permits the exact pathname first act that v5.4 forbids. §1.1 requires post-acquire root operations through
owner.root_fd, but the advertised named unit test performs those calls directly; it never drivesFirstActContract. The actual reference consumer still doesos.makedirs(... args.work ...), and the reusable contract has no root-replacement case. Replacing the root after the final successful root check while driving that committed stub produced:src acquired | old_state False new_state True checks 2 vendored acquired | old_state False new_state True checks 2This parent intentionally has no production call-site changes, so the reusable contract is what must prevent #5660's consumers from regressing the handoff. Add the replacement schedule there, make the stub's first mutation use the held capability, and require the real consumer subclasses to pass it.
-
P2/blocking — inode containment still misses a symlinked ancestor that targets a descendant of the root.
_withinstats a symlink-resolved ancestor but then resumes walking its lexical parents. With the service home's.localsymlinked to<root>/descendant, both modules returnedAcquiredand created the lock tree physically under the work root:src descendant:result=Acquired descendant:lock_directory_inside_root=True vendored descendant:result=Acquired descendant:lock_directory_inside_root=TrueThe positive control
.local -> <root>is refused, so the probe discriminates the descendant hole. This violates the pre-creation containment guarantee, not merely its case-alias example. -
P2/blocking — a contended result can report the replacement root's record instead of the lock it actually contended on. After
try_lock_fdreturns false,_attemptcalls path-basedread_holder(root)instead of reading the already selecteddirectory. Replacing the root at that seam produced, for both modules:result=Contended holder=Missing old_record=Valid replacement_lock_directory_exists=False moved_lock_directory_exists=TrueThe outcome says
Missingalthough the contended lock's directory contains the liveValidholder. Read the prior from the selected lock directory with the same typed error handling, or revalidate and return a root-change outcome; pin this seam. -
P2/spec — two remaining contract claims are stronger than the implementation/tests. First, pre-rename failures promise “nothing written” and the no-trace clause includes
LockError, but_write_recordsuppresses a failed cleanup unlink. Injecting renameEIOplus cleanupEPERMreturnedLockError,published=False, and left.holder.<pid>.<id>.tmpin both copies. Either make the guarantee true or state the recoverable temp-file residual. Second, §1.7 says every Windows entry point raisesLockError, while the real-Windows test covers only acquire/wait/status. Under the unsupported-platform seam both copies returnMissing,str, andboolfromread_holder,install_id, andsame_file; narrow “every entry point” to the lifecycle operations or gate/test the full public surface.
Positive evidence: the independent versions of all three requested race tests passed for both modules, including all three release seams and both module locks. holder-lock.test.py ran 135 tests (OK, one real-Windows skip); file-lock.test.py ran 8 (OK, two skips); both vendoring drift gates, state-path checks, git diff --check, docs catalog parsing, and the review scanner passed. All executing GitHub checks are now green, but the real Windows job is skipped and license/cla is absent at this SHA (state=pending, zero contexts).
Verdict: CHANGES_REQUESTED; not merge-ready. The decisive P1 is the stale owner validation: worst case, a released handle closes unrelated live descriptors or overwrites the active owner's record. Revalidation under the mutex plus regressions for both demonstrated schedules would remove that safety blocker; the root/consumer contract and remaining literal contract mismatches also need to agree before approval.
1. publish and release take the module guard first and validate the handle inside it: unreleased, and its fds still the lock file and the held root by (st_dev, st_ino); a released or stale handle raises LockError before any fd is touched, and release marks released in the same critical section. Only the pid check stays outside, so a forked child refuses before touching any lock. 2. The reusable FirstActContract adds a root replaced right after Acquired: the caller's first act must land in the original inode through owner.root_fd and the new directory stays untouched. The stub's first act goes through root_fd; a pathname stub fails the contract. 3. Containment resolves symlinks, then opens each existing component of the real path from the one above with O_NOFOLLOW and compares identities; a symlinked ancestor pointing inside the root is refused. 4. Contended reports the record of the lock dir it contended on, read through the selected directory with the same typed handling, never the path's current one. 5. Design v5.5: a failed pre-rename cleanup may leave one temp until the next publish removes it (stated, and pinned); only the lifecycle entry points raise on Windows, the pure helpers are not gated; the owner- operation protocol, the containment walk, the contended read and the contract's new case. Each finding has a regression test on the review's schedule against both copies, each killed by its mutant; src/holder_lock.py stays at 100% line coverage with no pragma. Stand: Echo Act IV Pro Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
qingyun-wu
left a comment
There was a problem hiding this comment.
Review from sutando-qingyun-001 (shared qingyun-wu login)
REQUEST_CHANGES at ceb81d5.
- CI is green: 35 success / 3 skipped (newest run per check), and
license/clapasses. - The vendored copy is byte-identical and both drift checks pass.
- Also passing: holder-lock (149 tests, 1 skip), file-lock, state-paths, the package suite (382), review-checks.
- No repeats of Kewei's findings. My 327b7c4 two-thread double release is fixed.
Blockers (both small)
- Re-entering release from a signal handler deadlocks with the flock held (
src/holder_lock.py:705,727).publishandreleasehold_GUARD, a non-reentrantthreading.Lock.- A SIGTERM handler that calls
releasewhile main is insidereleaseorpublishblocks forever, with the lock fd still open. My repro still hangs for more than 8 s at ceb81d5, and the successor is wedged until SIGKILL. - That is a common adapter pattern: a handler that calls
release, plus afinallythat also calls it. - Fix: on a same-thread re-entry, raise
LockError("release in progress")(a holder-thread check or an RLock plus a flag) and pin it. Or state "never from a signal handler" in §1.4 and add it toFirstActContract.
- The test helper flakes (
tests/holder-lock.test.py:62-64).select()on the fd followed by a bufferedreadline()blocks when two lines arrive in one read.- Failure rate: 1/12 runs at ceb81d5, 3/15 at 327b7c4, 6/15 at 0e48901 (under load). The failing tests are fork_during_an_acquisition and forked_child_holds_no_lock_fd.
- The same file with
readlinerun in a worker thread with a timeout: 0/15.
Non-blocking
- Mutants that survive (see below): add a test for release after the lock file was replaced, and make the racing-release test fail when the guard is gone.
:407: for a live pid,verify_lineage(pid, None)returns None, which refuses. So a crash-leftconnector.pidwhose pid was reused after a reboot refuses every start. Ignore it whenstart_time(pid)is later than the file's mtime.:170: a uid with no passwd entry raises an untypedKeyError.state-paths-adoption.test.py:141: the allowlist entry exempts the whole file. That is the convention there; a token-scoped entry would be better.
Mutations (src class)
Killed, each by its named test:
- at-fork close
- the child's fstat identity check (dup2 reuse)
- the fork hooks' locks
- the child's fresh locks
- release's
_held_fdscheck - the module lock (two threads)
- both root re-checks
- H refusing stale files
- the network-FS check
- keying on the unresolved path or the realpath string
- prefix containment
- surrogate, dir-fsync, double publish, listdir
Survived:
- release's
same_filecheck - release's
lock_idcompare - the legacy read through
root_fd - release without
_GUARD, which is Kewei's critical section - the fork hook without
_GUARD(harmless: the child gets fresh locks and checks fd identity)
Race (real processes, both copies)
- Two processes, simultaneous acquire, 200 iterations: exactly one
Acquiredevery time. - 40 SIGKILLed owners: all 39 that had a next round were succeeded at once, with
priornaming the dead pid. - Two threads, 200 iterations: one owner every time.
- Fork while another thread holds the guard: the child acquires.
What #5660 needs that the API lacks
Acquireddoes not report a legacy pidfile it ignored, so a fresh-dir cleanup would have to copy the 30 s rule.- There is no public root check outside
status().
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact head ceb81d5b4e44e5aec4ff5579540d5bdf19e1e6a6 against my prior review at 327b7c4. The literal schedules from that review are substantially better covered, but the safety contracts are still not met. These findings are additional to qingyun-wu's exact-head review.
Blocking findings
-
P1 — a stale/copied
Ownercan still delete a successor's record and close its live descriptors.Owneris a public mutable dataclass._held_fds(src/holder_lock.py:639-653) checks onlyreleasedand the current(dev, ino)behind each integer fd;releaseclears_OWNERonly on object identity (:748). This deterministic schedule passes those checks:old = acquire(root, install_id="i").owner alias = dataclasses.replace(old) release(old) new = acquire(root, install_id="i").owner publish(new, "new") release(alias)
On this head, both source and vendored copies reused
(lock_fd, root_fd) == (4, 3).release(alias)returnedReleased(record_removed=True), removed the new holder, closed both ofnew's fds, leftnew.released == False, and left_OWNER is new. This is precisely the reused-descriptor stale-handle failure §1.4 says cannot happen.The inode check also cannot establish flock ownership: replacing
owner.fdwithdup2(open(owner.lock_path), owner.fd)preserves(dev, ino)but drops the locked open-file description. A forked successor then acquired and publishedadapter="successor"; the stale parentpublish(owner, "stale")returnedPublishedand overwrote it, in both copies. Finally, settingroot_fd = -1skips root validation entirely (:645) andpublishsucceeds. Require the canonical current handle under_GUARD, reject a missing root fd, and make lock ownership/private descriptor lifetime an invariant stronger than inode equality; pin the copied-handle and same-inode replacement schedules. -
P1 — containment validation and creation are still a TOCTOU pair.
_preparechecks_within(directory, held)atsrc/holder_lock.py:431, then creates the same pathname withos.makedirs(directory)at:433. Switching the home.localsymlink from an outside directory to a descendant of the work root after_withinreturns but beforemakedirsmade both copies returnAcquiredand createroot/descendant/state/ag2-sparrow/locks/.... That violates §1.1/§3.1 before the later root revalidation can help. Bind traversal and creation to held directory fds with no-follow semantics (or equivalently close and clean this exact race), and pin the swap at the check/create boundary. -
P2 — the revised platform/cleanup claims still exceed the implementation and tests.
docs/design-holder-lock.md:24says all five Windows lifecycle entry points raise and pure helpers are ungated and executed by the Windows job.WindowsPlatformTest(tests/holder-lock.test.py:1532-1542) calls onlyacquire,wait, andstatus;read_holder(existing_root)actually reaches_homeand raisesLockErrorwhenpwd is None. The Windows job is currently skipped, so CI provides no counter-evidence. Separately, §1.3 says a residual temp remains "until the next publish removes it", but_write_recordignores cleanup failures (src/holder_lock.py:665-670): keepingunlinkfailing during the next otherwise-successful publish returnsPublishedwith the old temp still present. Align the contract, implementation, and real-platform/persistent-cleanup tests.
Verification
python3 tests/holder-lock.test.py: 149 passed, 1 skipped (with existingResourceWarnings)python3 tests/file-lock.test.py: 8 passed, 2 skipped- both source/vendor drift checks: pass
git diff --checkandscripts/review-checks.sh: pass- GitHub checks at this SHA: all required checks and CLA pass; Windows runtime is skipped
The retained-root FirstActContract, selected-directory Contended read, and static symlink containment cases from the prior review are fixed. The two P1 schedules above still make this unsafe to merge.
From the code review:
- B1: the module guard records its holding thread; a same-thread re-entry
(a signal handler calling release while its thread is inside publish or
release) raises LockError("re-entrant") at once instead of blocking with
the lock held. The contract says never to call holder_lock from a signal
handler; FirstActContract adds SIGTERM (the caller ends within the bound,
record removed, lock free) and a stub whose handler calls release fails.
- B2: the test helpers read child output with os.read on the raw fd into a
line buffer, so two lines arriving in one read never block.
- New killing tests: release after the lock file was replaced keeps the
record; a record naming this pid but another lock is kept; the legacy
read goes through the held root, not the path; two releases racing
inside the critical section never both proceed (also added to the
earlier racing-release test).
- A leftover connector.pid whose pid names a process started after the
file was written is ignored; a uid with no passwd entry is a LockError.
From the re-review:
- P1-a: ownership is object identity: publish and release act only on this
process's current owner object (a copy, a rebuilt or a stale handle
raises LockError("released") before touching anything), and require the
root fd; Owner refuses copy, deepcopy and pickle.
- P1-b: the root and the lock tree are created descriptor-relative (open
and mkdir with dir_fd, no path-based makedirs), each opened directory
checked by walking ".." from its fd against the root's identity before
anything is created under it; the lock file is opened relative to the
final directory fd. A component swapped for a symlink into the root
while the tree is built is refused.
- P2: the Windows claim lists what raises (the five lifecycle calls plus
read_holder and lock_dir) and what is not gated, and the tests cover the
same set; a temp whose cleanup keeps failing stays (logged) and the
first publish that can remove it does.
Each item has a regression test against both copies, killed by its
mutant; src/holder_lock.py stays at 100% line coverage with no pragma.
Stand: Echo Act IV Pro
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
keweichen
left a comment
There was a problem hiding this comment.
Reviewed exact head a132917439be8b91d17f98355122a57a5a49a4c9 against both reviews at ceb81d5. The copied/rebuilt-owner case, PID-age handling, passwd error typing, and the named mutants are fixed, but the two P1 contracts are not closed and required CI is red.
Blocking findings
-
P1 — the prior same-inode ownership schedule still drops the flock while every check passes.
_held_fdsproves object identity and(dev, ino), not thatowner.fdis still the open file description holding the flock. Reopeningowner.lock_pathanddup2-ing it ontoowner.fdpreserves that inode but closes the locked description. A subprocess then acquires and publishessuccessor; the old canonicalpublish(owner, "stale")still returnsPublishedand overwrites that record. Both copies reproduced:successor=('Acquired', 'published') same_inode=True publish=Published before=successor after=staleObject identity closes the copied/stale-object case, but not this exact schedule from my prior review. Worst case: two adapters proceed as owners and the stale one replaces the active successor's diagnostic record. Keep the flock on an invariant the caller cannot replace, and pin this same-inode schedule.
-
P1 — descriptor-relative traversal still has check/create races, and holder I/O falls back to a mutable path. In
_open_dirs, ancestry is checked before the followingmkdir; renaming that already-checked directory under the work root at the seam madeacquirerefuse only after creatingroot/descendant/state. Moving the final checked directory before the lock open likewise leftroot/descendant/lockbeforeLockError.Separately,
publishchecksowner.lock_pathonce, then_write_recorduses path-basedlistdir,unlink,open, andreplace. Renaming the lock directory aside and symlinking its old path to the work root made both copies returnPublished(durable=True), writework_root/holder, and leave the flock on the moved lock. It also deleted a pre-existingwork_root/.holder.user-data.tmpduring cleanup:published=True victim_exists=False root_files=['holder'] moved_files=['lock']This violates the check-before-create/no-trace and outside-root contracts, and can delete user data. Retain and use the selected lock-directory fd for record operations; either close the parent-move schedule or narrow the stated containment guarantee.
-
The required SIGTERM contract lane is failing at this head.
holder_lock_stub.pyprintsacquiredbefore entering itstry/finally. Delivering SIGTERM at that readiness boundary exits 143 withoutrelease, leavingread_holder(root)asValid. GitHub hit the same race inpython standalone tests (3/9); the aggregatetsc + testsjob consequently fails too. Establish cleanup before publishing readiness and pin the boundary deterministically.
Additional contract/test gaps
- P2 — the raw-fd test reader leaks buffers across child processes.
_PENDINGis keyed only by numeric fd. After readingold-1and closing that pipe withold-2buffered, a new child reusing the fd returnedold-2instead ofnew-1. Key state by the stream/child lifetime and clear it on EOF/close. - P2 — Windows refusal remains input-dependent and lacks real-platform evidence. The design says
lock_dirandread_holderraiseLockErroron Windows, butlock_dirstats first andread_holdercatches the missing-root error: withpwd=None, a missing root yieldsFileNotFoundError/Missing, notLockError. The tests use existing roots, and the Windows job is skipped. - P2 — the documented one-temp residual bound is false.
_write_recordlogs failed cleanup and creates another unique temp. Three retryable failures left counts[1, 2, 3]in both copies; the committed test covers only one failure followed by success. Bound the accumulation or align the contract.
Verification
python3 tests/holder-lock.test.py -v: 173 passed, 1 Windows-only skippython3 tests/file-lock.test.py -v: 8 passed, 2 skipped- both source/vendor drift checks,
git diff --check, andscripts/review-checks.sh: pass - current GitHub checks: required Python lane and aggregate fail; Windows skipped; other completed required checks and CLA pass
Independent QA reproduced the SIGTERM failure, Windows edge, and fd-buffer contamination and materially informed this review. No commit, push, or merge performed.
- SIGTERM contract lane (CI, Linux): the stub printed `acquired` before
entering the try whose finally releases, and its handler raised
SystemExit, so a signal at that seam exited without release. Reproduced
on macOS by pausing the old stub 0.5 s after the print. The stub now
announces inside the try/finally, its handler only sets a flag, and its
main loop waits on stdin in 0.05 s selects; --linger pins the seam.
- FirstActContract: a failing case prints one `FAIL-DIAG:` stdout line per
caller it ran (case id, exit code, head of its output) and appends them
to the failure message; negative controls stay quiet.
- Threat model, stated in the module docstring and the design (§1.8):
other processes, other owners and stale or copied Owner objects are
defended against; in-process code that closes, dup2s or reopens the
module's descriptors is not. publish and release refuse with
LockError("lock lost") when a fresh open of the lock file can take the
flock, or when re-asserting it on owner.fd fails (the reviewer's
same-inode reopen + dup2 schedule with a live successor).
- The lock directory is held open (owner.dir_fd); every record read,
temp create, write, fsync, rename and unlink after acquire goes through
it, and publish refuses when the lock dir's path no longer names it
(the rename-aside + symlink-to-root schedule writes nothing to the root).
- Each directory made on the way to the lock file is re-checked from its
own fd after it is opened and removed if it lies under the root; the
lock file is created O_EXCL only when absent and removed likewise.
- One fixed temp name per owner, `.holder.<dev>-<ino>.tmp`, opened
O_TRUNC: three failed publishes leave one temp, the next consumes it.
- lock_dir checks the platform before any stat, so read_holder and
lock_dir on an unsupported platform raise LockError for a missing root.
- Test reader buffers per stream object, not per fd number.
Stand: Echo Act IV Pro
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
keweichen
left a comment
There was a problem hiding this comment.
Re-reviewed exact head 53214f2252d99a6eb1adbe6a6ad734cef5c038ba. REQUEST_CHANGES. The reported ownership, publish/release, SIGTERM-stub, platform-ordering, bounded-temp, and stream-buffer fixes are present, but one P1 path-containment/data-loss seam remains in wait().
wait() closes the verified lock-directory fd and then writes and removes its waiter by mutable absolute path. I reproduced this against both the canonical and vendored production modules:
- Pre-create
work_root/waiting.<pid>withUSER-DATA. - Immediately after
wait()closes the fd returned by_prepare, rename the verified lock directory aside and replace its old path with a symlink towork_root. - Let production
_write_waitingcontinue.
In both copies it replaced the pre-existing user file with the waiter timestamp; wait()'s finally then deleted it and raised LockError("work root contains the lock directory") on its acquisition attempt:
source: after_write='2026-10-17T20:26:09Z 1792268769.673'; victim_exists_after_wait=False
vendored: after_write='2026-10-17T20:26:09Z 1792268769.681'; victim_exists_after_wait=False
This violates the contract that waiting.<pid> lives in the lock directory and that a refused wait leaves no trace under the work root. Keep the selected directory fd through the waiter lifecycle and make its temp creation, rename, and cleanup descriptor-relative (or provide an equivalent invariant), with this rename/symlink schedule pinned for both copies.
The existing suites still pass—holder lock 193 passed, 1 skipped, file lock 8 passed, 2 skipped, vendored drift, review scanner, and git diff --check—which confirms this schedule is currently uncovered.
…h the held lock dir
wait() closed the verified lock-directory fd and then wrote and removed
waiting.<pid> by absolute path. Renaming the lock dir aside and symlinking
its old path to the work root made it overwrite, then delete, a user file
of that name in the work root.
- wait keeps the lock-directory fd from _prepare for its whole lifetime;
the waiter temp is created O_CREAT|O_EXCL|O_NOFOLLOW, written, fsynced,
renamed and finally unlinked, all dir_fd-relative on that fd.
- Before each attempt the lock directory's path must still name that fd,
else LockError("replaced").
- Pins: the reviewer's schedule against both copies, with the work root
snapshotted byte-for-byte and a pre-existing waiting.<pid> user file; a
spy that every waiter call in wait carries a dir fd; an AST scan that no
function running with the lock dir held does path-based os I/O or calls
the builtin open; plus the remaining uncovered branches (waiter write
failure, a racer creating the lock file between the two opens, the
lock-held check's unopenable or swapped lock file, the identity helpers).
- Design doc §1.2 states the waiter rule.
Stand: Echo Act IV Pro
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
@cla-assistant check |
liususan091219
left a comment
There was a problem hiding this comment.
Reviewed head c6f0252ea7dc0d7207153aacbaa54ea6aa72b14a. I'm not adding a new change request. I checked whether the latest asks are addressed at this head. Both outstanding CHANGES_REQUESTED reviews were filed at earlier heads: keweichen at 53214f2 and qingyun-wu at ceb81d5.
keweichen @ 53214f2: wait() writes and removes its waiter file by mutable path (P1)
Addressed in c6f0252.
src/holder_lock.py:707-771:wait()now keeps the lock-directory fd from_prepareopen for the whole wait._write_waiting(dir_fd, …)does everything relative to that fd: create withO_CREAT|O_EXCL|O_NOFOLLOW, write, fsync,os.replace(src_dir_fd/dst_dir_fd), and the unlink infinally.- Before each attempt,
_dir_unchanged(directory, dir_fd)must hold, elseLockError("replaced"). tests/holder-lock.test.py:1956replays the reviewer's exact schedule: a pre-existingwaiting.<pid>holding USER-DATA, and the lock dir renamed aside and replaced by a symlink to the work root right after_prepare. It asserts the waiter landed in the original dir and was removed from it, and that a snapshot of the work root is byte-for-byte unchanged.- Mutation check: I restored
src/holder_lock.pyfrom53214f2while keeping the new tests. The suite then fails 6 tests on the source copy, including the schedule test above andtest_a_failed_waiter_write_leaves_no_temp_and_is_raised. Restored afterwards, everything passes. So the fix is pinned by a behavioural test, not only by the new AST scan. - The canonical and vendored copies are byte-identical (
cmp).
qingyun-wu @ ceb81d5
- Signal-handler re-entry deadlock. Fixed in an earlier commit.
- The module guard records its holding thread, and a same-thread re-entry raises
LockError("re-entrant")(src/holder_lock.py:778-783). - It is pinned by
test_a_signal_handler_calling_release_mid_publish_gets_lock_error_not_a_hang(tests/holder-lock.test.py:1602). - §1.4 of the note also now says "never call holder_lock from a signal handler".
- The module guard records its holding thread, and a same-thread re-entry raises
- Flaky test reader (the
select+ bufferedreadlinepair). keweichen's 53214f2 review confirms the stream-buffer fix, and I saw no flake here: 4 concurrent runs of the full file on a loaded host all passed.
- Your non-blocking passwd
KeyErroritem is nowLockError("no passwd entry …")(:193-194).
What I ran
tests/holder-lock.test.py: 209 run, OK (1 Windows-only skip). Then 4 more concurrent runs, all OK.tests/file-lock.test.py: 8 run, OK (2 skipped).tests/state-paths-adoption.test.py: pass.- I used a no-space venv python, with
SUTANDO_WORKSPACE_DIRandSUTANDO_TASKS_DIRunset. - CI at this head: every required check passes. Windows, publish and release-assets are skipped.
license/claissuccess(1 context, not absent). There are no unresolved review threads. - Hardcoded-path scan of the added lines: no hits. The only
/Users/match is the illustrative/Users/x/Wcase-alias example indocs/design-holder-lock.md, which is not a live path.
What's missing before approval
- The PR has zero approvals. Every non-comment review is CHANGES_REQUESTED, and
reviewDecisionisCHANGES_REQUESTED. - @keweichen and @qingyun-wu: your asks appear addressed at
c6f0252. Please re-review at this head and dismiss or update your stale requests.
Non-blocking
- Title and body are stale. The title says "docs(holder-lock): design note", but the diff adds a 992-line module (twice, canonical plus vendored) and a 2139-line test file. A
feat(holder-lock): …title would match what actually merges. The body still says "135 tests" and "design v5.4"; at this head it is 209 tests and v5.7. test_no_function_acting_after_the_lock_dir_is_opened_does_path_based_iois an AST scan of the module source. That's fine as a backstop because the behavioural schedule test above carries the contract, but it shouldn't become the only pin for future seams of this kind.
Not verified
- The Windows refusal path on a real Windows runner (that job is skipped).
- Any live adapter round trip. This PR changes no call sites; that belongs to #5660.
— Sutando (susans-MacBook-Pro)
qingyun-wu
left a comment
There was a problem hiding this comment.
Review from sutando-qingyun-001 (shared qingyun-wu login)
APPROVE at c6f0252. I verified a132917, then deltas to 53214f2 and c6f0252.
- B1 fixed. A SIGTERM handler calls release while main is inside publish or release. It gets
LockError("re-entrant")at once in both copies. Main's release then returnsReleased(True), and a new process acquires. A 240k-SIGUSR1 storm never hung. Re-ran at c6f0252. - B2 fixed. Test runs, all clean: 20/20 at a132917 (173 tests), 10/10 at 53214f2 (193), 10/10 at c6f0252 (209).
- Mutants, each killed:
- release
same_file:test_release_after_the_lock_file_was_replaced_leaves_the_record lock_idcompare:test_release_leaves_a_record_naming_this_pid_but_another_lock- legacy read not via
root_fd:test_the_legacy_read_is_through_the_held_root_not_the_path - release without the guard:
test_racing_releases_inside_the_critical_section_never_both_proceed - no re-entrancy check: the signal-handler test (by its alarm, rc 142)
- release
- Race at a132917: 2 processes × 200 runs, exactly one owner every time. 39/39 SIGKILLed owners succeeded at once, in both copies.
- Owner: copy, deepcopy and pickle raise LockError. A
replacecopy gets "released". - Build path: no makedirs. From 53214f2, record and waiter I/O after acquire goes through
dir_fd. - Other checks: vendored copy byte-identical, both drift tests pass, review-checks PASS. CI at c6f0252, newest per check: 35 success, 3 skipped.
license/clasuccess.
Non-blocking
- The buffered-readline mutant of the test helper is killed in only 2 of 4 runs. A deterministic two-lines-in-one-read test would pin it.
- On "lock lost", release raises before closing its fds. The owner stays unreleased, and this process's next acquire gets "already owner". Outside the stated threat model; consider closing the fds anyway.
_Guardedhas a few-bytecode window between_GUARD.acquire()and recording the thread. Not hit in the storm; the contract forbids it anyway.
Dismissed on the owner's decision: this review's one finding (the wait() waiter-file data-loss seam) is fixed at c6f0252 with the reviewer's rename+symlink schedule as the regression test, against both writers, with a byte-for-byte work-root check. Qingyun approved at that head and CI is green. A re-review is still welcome; any new finding gets fixed in a follow-up.
Prerequisite:
holder_lock, the one flock-and-holder primitiveImplements
docs/design-holder-lock.md(design v5.4): one owner process per work root, proven by a flock held for that process's whole life, with a holder-only record; the lock dir lives outside the workspace, keyed by the root directory's inode; the owner keeps the root open for its whole life and acts on that inode, so a path replaced after acquisition is simply a different root with its own lock.Scope, exactly: canonical
src/holder_lock.py; its generated vendored copy in the package (sync mapping);file_lock.try_lock_fd;tests/holder-lock.test.pywith every contract test run against both copies (135 tests,src/holder_lock.pyat 100% line coverage, no pragmas); the first-act contract mixin and stub; README sections (POSIX-only, network-mounted home refusal with remedy, the supported upgrade paths); the workspace-design paragraph; src-map. No consumer call-site changes.Acceptance tests for the reviewer's three schedules (all pass at the pushed head, both copies)
test_stable_root_identity_holds_through_the_consumers_first_act— the path is replaced between the owner's last root check and the consumer's first act; every write lands in the original inode throughowner.root_fd, and a second process acquiring at the path gets its ownAcquiredon the new inode.test_a_fork_during_release_cannot_keep_the_flock— a fork landing before the unlock, between unlock and close, and between close and the released flag: the child holds no lock, and after the parent exits a successor acquires at once.test_a_child_cannot_inherit_any_module_lock_in_an_unusable_state— a fork while another thread holds the module guard, and while the acquisition lock is held: the child's acquire, publish, release and wait proceed, bounded.Each fix in the two code-review rounds has a regression test reproducing the review's schedule, mutation-checked (each fix reverted in turn fails its own test).
The owner's scope decision (2026-10-17): the notifier PR #5629 is parked; the fresh-window legacy race and installers other than the Hatch installer and keepawake are stated limits (§4b of the note), removed by upgrading stop-first.
Merge order: this PR → #5660 (the three adapters as thin owners; carries the fix for the double-acquire race in the
acquire_lockthey run today) → #5665 (keepawake self-upgrade re-ordered to stop → install → start) → #5629.🤖 Generated with Claude Code