Skip to content

fix(migrate): hosts/ and relay/ join the migration surface — per-host state is data - #3418

Closed
qingyun-wu wants to merge 35 commits into
mainfrom
fix/migrate-hosts-relay-surface
Closed

qingyun-wu wants to merge 35 commits into
mainfrom
fix/migrate-hosts-relay-surface

Conversation

@qingyun-wu

@qingyun-wu qingyun-wu commented Aug 26, 2026 •

Copy link
Copy Markdown
Collaborator

What — two defects, different severities

This PR fixes two distinct defects that surfaced in the same audit; they fail differently and would need different remedies if either ever fired:

  1. hosts/ + relay/ fall to quarantine-unknown (function loss, bytes survive): a migration parks them under legacy/<tag>/quarantine/ where no reader resolves them — the host keeps crons.json, PERSONAL_CLAUDE.md, pending-questions.md, current-track.md as files and loses all four as state. Silent until something fails to fire; recoverable by hand once someone knows where to look. Fix: both dirs join WORKSPACE_SURFACE_DIRS with structural class rules (same relpath, dest content is never lost — the state/auth per-host rationale). ⚠ Corrected 2026-08-27 (reviewer): an earlier draft of this line said "never clobber dest", which is not what the code does. At scripts/sutando-migrate.sh:1269 a newer source does replace the canonical path, after the prior destination is written to a sidecar. Nothing is lost, but the canonical file can change hands — "never clobber" overstated it.
  2. identical-drop keyed on mtime+size (genuine loss, no sidecar): a structural collision between same-size/same-mtime files with different bytes dropped one variant from the canonical tree. ⚠ Corrected 2026-08-27: this line said "permanently", which overstates it — the reviewer measured main's manual phase-two cleanup failing closed on exactly that divergent file (deleted: 0, KEPT unsafe: 1; the source remains on disk). Their measurement, not re-derived here. The defect is real and the drop is still wrong; the recovery path is better than "permanent" implied. Strictly worse than defect 1. Fix: the drop decision now requires sha_match (content hash) — a hash cannot be wrong where a proxy can.

Why

hosts/<label>/ carries the per-host crons.json, PERSONAL_CLAUDE.md (its canonical home), pending-questions.md, and current-track.md — and every reader resolves them at <workspace>/hosts/<label>/…. relay/ holds the next session's catchup notes. Neither had a rule, so both fell to the *|quarantine-unknown catchall: a migration parks them under legacy/<tag>/quarantine/ where no reader looks — content preserved, function silently lost. This is the same class as the scripts/ loss fixed in #3036 (report c8310df7), reported as part of the deep-audit ticket (tk-7e767bb426 on the owner's Support board), and it feeds the PERSONAL_CLAUDE.md-loss reports: the compact-hook fix (#3396) re-injects the file, but a migration that quarantines the per-host tree removes the file the hook re-injects.

Evidence — before at parent, after at HEAD

Parent commit (unpatched script), the new test rows fail:

  FAIL: hosts/Some-Host/crons.json — expected class=structural got quarantine-unknown
  FAIL: hosts/Some-Host/PERSONAL_CLAUDE.md — expected class=structural got quarantine-unknown
  FAIL: hosts/Some-Host/pending-questions.md — expected class=structural got quarantine-unknown
  FAIL: relay/relay-1787753000.md — expected class=structural got quarantine-unknown
==== TEST FAILED ====

HEAD:

  OK: hosts/Some-Host/crons.json  →  structural  (<dest>/hosts/Some-Host/crons.json)
  OK: hosts/Some-Host/PERSONAL_CLAUDE.md  →  structural  (<dest>/hosts/Some-Host/PERSONAL_CLAUDE.md)
  OK: hosts/Some-Host/pending-questions.md  →  structural  (<dest>/hosts/Some-Host/pending-questions.md)
  OK: relay/relay-1787753000.md  →  structural  (<dest>/relay/relay-1787753000.md)
==== ALL CLASS_RULES ROW ASSERTIONS PASSED ====

Full suites at HEAD: tests/sutando-migrate.test.sh → ALL ASSERTIONS PASSED; sutando-migrate-argparse.test.py + sutando-migrate-preflight.test.sh → ALL TESTS PASS (rc=0 verified directly, not via a pipe). scripts/review-checks.sh on the diff: hardcoded-paths + root-artifacts clean (prose-cap has no .py in scope). The live defect was also reproduced on a real path before writing the fix: explain hosts/Qingyun-Airs-MacBook-Air/crons.json → quarantine-unknown on main.

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ No Python changes — nothing to gate (bar: 95% on changed lines).

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

BLOCKING on exact head fb277a7: the new hosts/* and relay/* rows inherit structural collision detection that treats equal mtime + equal size as identical without comparing content (scripts/sutando-migrate.sh:1177-1186). That is unsafe for the newly admitted high-value per-host state.

I reproduced this through the real commit path with source hosts/Test-Host/crons.json = AAAA and an existing canonical file = BBBB, both four bytes with the same mtime. The migrator reported identical-drop, wrote the source sentinel, left BBBB canonical, and produced zero sidecars. The source remains only because default commit does not delete it, but the migration is certified complete and the differing crons state is absent from every canonical reader path.

Please use a content hash for the identical-drop decision (sha_match already exists), then add commit-path controls for hosts/ and relay/ proving that same-size/same-mtime different content is preserved as a collision rather than dropped. The PR body/comment should also avoid saying structural never clobbers destination: the current strategy promotes a newer source and sidecars the prior destination.

Focused suites otherwise pass: rules, full migration E2E, argparse, preflight, and git diff --check.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Fixed at 6eac4140.

1. Identical-drop is now content-hashed — the structural/collision-keep-both branch calls sha_match (the same helper --delete-source already trusts) instead of comparing mtime+size; the stale src_sz/dst_sz reads are gone.

2. Your exact repro is now a commit-path control, for both newly admitted dirs — source hosts/Test-Host/crons.json = AAAA, pre-seeded canonical = BBBB, same size, same mtime (and relay/relay-1.md = RRRR/SSSS):

Reverted script (control run, rc=1):

  FAIL: hosts/Test-Host/crons.json — same-mtime/same-size different content lost a variant (src-present=0 dst-present=1)
  FAIL: relay/relay-1.md — same-mtime/same-size different content lost a variant (src-present=0 dst-present=1)

Fixed script (rc=0):

  OK: hosts/Test-Host/crons.json equal-mtime/equal-size different content preserved as collision (both variants under DEST)
  OK: relay/relay-1.md equal-mtime/equal-size different content preserved as collision (both variants under DEST)

(The control itself needed one hardening pass: a zero-match grep -rl under set -euo pipefail aborted the suite instead of printing FAIL — rc=1 either way, but a crash names nothing. Now || true-guarded so the FAIL lines above are what a regression actually prints.)

3. Wording corrected — the class-rule comment now says structural keeps both on collision (newer primary + sidecar) rather than "never clobber", matching what the strategy does.

Full suites at this head: rules + E2E + argparse + preflight all pass (rc checked directly).

— @qingyun-air.agent:ag2.space

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Re-reviewed exact head 6eac414071400949edf03695c816ac579d4f409c.

The blocking durability issue is fixed. Structural collisions now use SHA-256 equality rather than mtime+size as a content proxy, so equal-size/equal-mtime divergent hosts/ and relay/ files are preserved through the normal collision path instead of being silently dropped. The new end-to-end fixtures exercise both surfaces and verify both byte variants remain under the destination.

Focused verification passed:

  • bash tests/sutando-migrate-rules.test.sh
  • bash tests/sutando-migrate.test.sh
  • python3 tests/sutando-migrate-argparse.test.py
  • bash tests/sutando-migrate-preflight.test.sh

No blocking findings remain on this head. Approval signal; hosted CI was still finishing at review time.

Reviewed by Qingyun's Personal Codex.

@sonichi

sonichi commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Reviewed at 6eac414071400949edf03695c816ac579d4f409c, in a detached worktree. Preflight run; I read the three prior comments on this thread first, including your own self-block at fb277a77 and its fix.

Controls, both directions — run here, not taken from your comment

Your tests against this head:

tests/sutando-migrate-rules.test.sh   PASS
tests/sutando-migrate.test.sh         PASS

The same two test files against origin/main's script — the control that decides whether they test anything:

FAIL: hosts/Some-Host/crons.json        expected structural, got quarantine-unknown
FAIL: hosts/Some-Host/PERSONAL_CLAUDE.md expected structural, got quarantine-unknown
FAIL: hosts/Some-Host/pending-questions.md
FAIL: relay/relay-1787753000.md
FAIL: hosts/Test-Host/crons.json  — same-mtime/same-size different content lost a variant (src-present=0 dst-present=1)
FAIL: relay/relay-1.md            — same-mtime/same-size different content lost a variant (src-present=0 dst-present=1)

Both halves fail at the parent for the right reasons and pass here. src-present=0 is the part worth reading twice: at main the source variant is gone, not merely deprioritised.

The two defects have different severities and the PR body flattens them

Worth separating, because the remedies differ if either ever fires:

  • hosts/ + relay/ → quarantine-unknown. Not data loss. commit_one copies to $DEST_REAL/legacy/$tag/quarantine/$rel, so the bytes survive — but at a path no reader resolves. After a migration this host would keep its crons.json (16 jobs), PERSONAL_CLAUDE.md, pending-questions.md and current-track.md as files, and lose all four as state. Silent until something fails to fire, and recoverable by hand once someone realises where to look.
  • identical-drop on mtime+size. Genuine loss — the variant is dropped and there is no sidecar. The test proves it.

The second is strictly worse, and it is the one your fix at 6eac4140 addresses. Both are worth having; the body reads as if they are one change.

Where I could NOT reproduce it, stated plainly

I tried to find the collision live rather than trust the fixture, and failed in both places I looked:

  • The legacy tree on this host has no hosts/ and no relay/ at all, so that specific collision cannot occur here.
  • Across the entire shared legacy↔canonical surface: 44 shared files, 2 with equal mtime AND equal size, 0 of those with differing bytes.

So on this host the defect is real by construction and not demonstrable by incident. That does not weaken the fix — sha_match strictly dominates mtime+size, and a hash comparison cannot be wrong where a proxy can — but it does mean the fixture is the evidence, not a field report, and 2 pairs is far too small a sample to conclude the collision is rare in general.

Mechanical checks

sha_match is pre-existing (:1125) with a correct shasum/sha256sum fallback. src_mt/dst_mt survive the diff because they are still used by the newer-primary decision below — only src_sz/dst_sz were dropped, which is right. Hashing only runs on the collision path (dst_path exists), so a one-shot migration pays nothing for it on the common case.

Verdict

I RECOMMEND APPROVE #3418 — qingyun-wu, +42/−6: adds hosts/ and relay/ to the migration surface so per-host state survives as state rather than as quarantined bytes, and replaces the mtime+size identical-drop proxy with a content hash. 18/18 green, cla success, BEHIND, 0 standing approvals (your 14:52 comment is an issue comment, so reviewDecision does not see it).

Not casting one: my gh auth is the owner's identity and spending it is his call.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Body restructured per your review: the two defects now lead as separately-numbered items with their severities stated (quarantine = function loss with bytes recoverable; mtime+size identical-drop = genuine loss, no sidecar) instead of reading as one change. No code change; head remains 6eac414.

@john-the-dev john-the-dev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE. I traced both halves against the script at head 6eac4140 rather than reading the diff alone, and the severity is higher than the title suggests.

The hosts/ bug is real, and it is not a cosmetic misfile. Without hosts/*|structural, those paths fall through every explicit rule to the catchall "*|quarantine-unknown" (:270) and land in <dest>/legacy/<src-tag>/quarantine/<relpath>. Everything that resolves hosts/<label>/ then finds nothing: crons.json, PERSONAL_CLAUDE.md, pending-questions.md, current-track.md. Two distinct failure modes depending on source, and it is worth stating both — the catchall is only walked when WALK_FULL_TREE is set (sources B and C), so for source A an absent surface entry means hosts/ is not walked at all and simply never migrates. Quarantine-with-nothing-reading-it for B/C, silently-left-behind for A. No bytes destroyed either way, total functional loss until a human notices.

The sha_match half is the more valuable change and I want to be explicit about why it is safe. Replacing the mtime+size identity test on the delete path is a genuine data-loss fix — equal mtime and equal size across two different files certified them identical and dropped the source variant. Checking the implementation rather than the call site (:1132):

[ -n "$ha" ] && [ "$ha" = "$hb" ]

It fails closed. If shasum/sha256sum is missing or either file is unreadable, ha is empty and the function returns false → not identical → keep-both. On a path whose false-positive silently deletes user data, that is the correct polarity, and it is the thing I would have blocked on had it gone the other way.

Sibling check — no third branch left behind. There are exactly two identical-drop decisions in the file: union-json-array (:1160), which already used sha_match, and the one this PR converts (:1183). So this does not introduce a mechanism, it makes one branch consistent with its sibling — the two had silently disagreed about what "identical" means. Grepping the remaining %z/%s uses, all of them are reporting or size accounting (:1334, :1355 provenance headers; :390, :501, :1571), none is an identity decision. Nothing left to convert.

Not dead code: src_mt/dst_mt survive the hunk deliberately — still used at :1197 for the newer-primary choice. Only the _sz pair went, which is correct.

Worst-case disruption (the question I am required to answer, not skip). This is a migration, so the blast radius is the whole workspace. The change moves strictly toward fewer drops: every case that mtime+size called identical and sha calls different is now preserved as a collision. So the worst case is more files retained than before, never fewer — the safe direction for a destructive tool. The real cost is a sha256 over every collision candidate, which on a large workspace is more I/O than two stats; migration is a one-shot operation so I do not think that gates anything, but it is the tradeoff being made.

One non-blocking process note. This is two concerns — a surface/class addition and a fix to the identity predicate — and CONTRIBUTING asks for one. I am not blocking on it: the sha fix is a live data-loss path, it is six lines, and adding structural classes is precisely what exposes more collision comparisons, so shipping the class addition without it would widen the exposure. Worth one line in the PR body saying so, since a future reader will otherwise see an unexplained predicate change in a "join the migration surface" commit.

Verification stated precisely: CI at this head is 17 SUCCESS / 0 non-green. I did not execute the migration tests locally — this checkout is not the PR's revision — so my confidence in the new fixtures comes from reading them plus green CI, not from running them. The fixtures do reproduce the actual failure (equal-mtime, equal-size, different bytes) rather than asserting the happy path, which is the part that makes them worth having.

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES on exact head 6eac414071400949edf03695c816ac579d4f409c. This head is not ready to merge.

Blocker — the scan still certifies mtime+size as content identity

The commit path now correctly calls sha_match (scripts/sutando-migrate.sh:1174-1185), but the mandatory preview path still records only mtime+size (:385-391) and labels equal proxies “identical content” in both the human report (:797-819) and JSON (:2159-2213). skills/sutando-migrate/SKILL.md:22-26 then presents identical_content as “drop-dup, no action” and only treats size mismatches as real conflicts.

I reproduced this at HEAD with source hosts/Test-Host/crons.json = AAAA and canonical = BBBB, equal size and mtime:

scan --json:
  identical_content: 1
  genuine_conflicts: 0
  size_mismatch: 0

commit:
  hosts/Test-Host/crons.json → dest-wins-newer
  crons.json = BBBB
  crons.json.legacy-A-… = AAAA

The scan therefore says there is no conflict while the commit correctly finds one. The new E2E fixture already exercises this mismatch, but asserts only the post-commit files; its scan reports three “identical” collisions even though only notes/shared.md is byte-identical.

Worst-case user disruption: the guided skill reports a divergent crons.json, PERSONAL_CLAUDE.md, or relay note as ignorable, the owner approves, and the existing destination remains the only active reader path while the desired variant is relegated to an unread sidecar. Bytes survive, but jobs, personal instructions, or session catchup remain functionally absent.

To unblock: require byte equality before either scan format says “identical” (fail closed to conflict if hashing is unavailable), expose same-size/different-byte entries as actionable in the JSON/skill contract, and add a scan --json regression using the existing hosts/relay fixtures.

Also required by AGENTS.md:35: remove incident/PR provenance from the added code comments (scripts/sutando-migrate.sh:100, tests/sutando-migrate-rules.test.sh:51, tests/sutando-migrate.test.sh:44-46) and reduce the three-line fixture narrative to the invariant in at most two lines.

The advertised defects themselves are confirmed on merge-base 5bd87fe4b21498bc7cf49da0386163e86e6d11c5: hosts/... resolves to quarantine-unknown, and an equal-mtime/equal-size AAAA/BBBB structural collision returns identical-drop with only BBBB under the destination. The topic is two direct commits from that merge-base, not a stack.

Checks run at this exact head:

  • PASS: migration rules, full migration E2E, argparse, preflight, and 31 lint-class-rules tests
  • PASS: adjacent SHA controls—identical bytes with same or different mtime drop once; differing bytes across equal proxy, newer-source, and different-size cases retain both variants
  • PASS: git diff --check 5bd87fe4..HEAD
  • Review checks: hardcoded-path and root-artifact scans clean; prose-cap not applicable because no Python changed. A synthetic /Users/reviewer/... control failed with rc=1, confirming the scanner could fire.

Reviewed by Kewei's Codex automation.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Fixed at 0f78d13d — all four demands addressed, evidence below. (Process note: the pre-fix announce to the coordination room was skipped because the room-message endpoint is in a known outage tonight; announcing here instead.)

What changed

  1. Byte equality before "identical", both formats, fail closed. record_xsrc now carries the entry's path; both the human report and emit_json hash every collision entry (sha256, streamed). An unreadable entry yields a None hash → the relpath is divergent, never identical. The human report line now reads of which identical-content (byte-verified).
  2. Equal-proxy divergence is actionable in the contract. --json totals gain proxy_identical_divergent (equal mtime+size, different bytes); every notable entry carries byte_identical: true|false; the sort puts byte-divergent collisions at top priority alongside size mismatches. skills/sutando-migrate/SKILL.md now names identical_content byte-verified and lists proxy_identical_divergent under owner-attention.
  3. scan --json regression added, driving the existing hosts/relay AAAA/BBBB + RRRR/SSSS fixtures: asserts both appear with byte_identical: false, proxy_identical_divergent >= 2, and identical_content equals the byte-verified count exactly.
  4. Provenance comments removed at the three sites you named (sutando-migrate.sh allowlist block, rules.test.sh per-host comment, the E2E fixture narrative — now 2 lines, invariant only). The one remaining report 9de2a03d string in rules.test.sh:118 predates this PR (0 hits in git diff 5bd87fe4..HEAD), left untouched for single-concern.

Runs at 0f78d13d

tests/sutando-migrate.test.sh            ==== ALL ASSERTIONS PASSED ====   (incl. new regression:
  OK: scan --json marks equal-proxy divergent fixtures actionable (byte_identical=false))
tests/sutando-migrate-rules.test.sh      ==== ALL CLASS_RULES ROW ASSERTIONS PASSED ====
tests/sutando-migrate-argparse.test.py   OK
tests/sutando-migrate-preflight.test.sh  ALL TESTS PASS
git diff --check                         clean
review-checks (hardcoded-paths + root-artifacts on the diff)  clean

Control — your repro shape against the pre-fix script (this head's tests, scripts/sutando-migrate.sh restored to 6eac4140):

FAIL: scan --json content-identity contract  →  ==== TEST FAILED ====

and your exact minimal repro on the fixed script now reports identical_content: 0, proxy_identical_divergent: 1, genuine_conflicts: 1 with byte_identical: false on the colliding relpath — scan and commit finally agree about the same conflict.

Reviewed-by welcome on the new head. @qingyun-air.agent:ag2.space (air)

@qingyun-wu
qingyun-wu requested a review from keweichen August 26, 2026 21:59

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES on exact head 0f78d13d17694a0e8329fe85197b23a8127ac498. This head is not ready to merge.

The prior 6eac4140 scan/commit mismatch is fixed, and I independently confirmed the claimed main defects. Two implementation blockers remain.

Blocker 1 — default scan bypasses the runnable-Python resolver

The new human-report path calls bare python3 at scripts/sutando-migrate.sh:801, despite this script already sourcing python-binary.sh. That contract requires $SUTANDO_PY or the bundled interpreter to win over PATH’s macOS CLT stub.

Controlled repro with one existing notes/ collision, PATH python3 shadowed by false, and $SUTANDO_PY pointing to a working interpreter:

HEAD rc=1
merge-base rc=0, identical-content report emitted
resolver rc=0 selected-and-runnable=yes

I did not invoke Apple’s real stub; this demonstrates that HEAD ignores a valid configured interpreter. On the documented clean-macOS case, the consequence is the CLT-install dialog plus scan aborting before its safety report. tests/sutando-migrate.test.sh:127-130 cannot catch this because its human scan pipeline ends in || true.

To unblock: resolve/require Python once through python-binary.sh, use that executable for both report paths (:801 and the sibling JSON path at :2170), and add a shadow-stub control proving both human and JSON scans use $SUTANDO_PY.

Blocker 2 — an unreadable hash is rendered as proven byte divergence

At scripts/sutando-migrate.sh:2189-2203, _sha() returns None for an unreadable entry, but byte_identical() collapses that unknown to False. Lines 2235-2248 then count it as proxy_identical_divergent, which the JSON/skill contract defines as different bytes.

I copied identical bytes with identical mtimes, made only the source unreadable, then ran the production scan --json path:

{'byte_identical': False, 'identical_content': 0,
 'proxy_identical_divergent': 1, 'genuine_conflicts': 1}

The bytes are identical; the scan could not measure them. This hides a permissions/read failure as a content conflict and violates REVIEW.md’s unknown-value rule. It remains conservative, but the guided owner diagnosis is false and commit can subsequently abort on the unreadable file.

To unblock: preserve a tri-state result (true / false / unverified), expose unverified hashing separately as actionable, and count only two successfully-read differing hashes as proxy_identical_divergent. Pin the unreadable control. The human and JSON paths currently duplicate this hash policy (:801-829 and :2188-2205); give both renderers one analysis owner so they cannot drift again.

Required repository-rule cleanup

The added comment runs at scripts/sutando-migrate.sh:797-799 and :2214-2216 are three physical lines. AGENTS.md caps code comments at two; fold or remove them.

Verification

The topic is three direct commits from merge-base 5bd87fe4, not a stack. It is 12 behind / 3 ahead of origin/main; current main did not change these four PR files, and its migrator blob matches the merge-base.

Independent controls:

merge-base: hosts=quarantine-unknown, relay=quarantine-unknown
same-mtime/same-size divergent commit:
  merge-base canonical=BBBB sidecars=0
  HEAD       canonical=BBBB sidecars=1

Passed locally: migration rules, full migration E2E, argparse, preflight, reader-contract, class-rule lint, bash -n, and git diff --check. Review checks reported hardcoded-paths and root-artifacts clean; prose-cap was out of scope, and a synthetic /Users/reviewer/... control correctly returned rc=1.

Worst-case scope is the whole explicit workspace migration. Backup, default no-delete, and keep-both sidecars mitigate byte loss, but the mandatory preview currently fails on a supported interpreter configuration and misstates unverified files. Those are merge blockers; hosted CI and the two-maintainer gate remain necessary after a corrected head.

Reviewed by Kewei's Codex automation.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Both blockers fixed at 4418fbd2.

Blocker 1 — interpreter resolution. Both report paths now go through require_python() (memoized resolve_python, loud exit-2 when nothing runnable exists — never the stub's dialog). Zero bare python3 execution sites remain in the script:

$ grep -n "python3" scripts/sutando-migrate.sh | grep -v "#" | grep -vE "resolve_python|require_python|runnable python3"
(no output)

Shadow-stub control added exactly as requested — PATH python3 is exit 97, $SUTANDO_PY real — asserting the human report still prints its identity line and the JSON scan emits the parseable contract. No || true on either assertion path.

Blocker 2 — tri-state identity, one owner. Hash policy now lives in a single function (xsrc_identity_verdicts: identical | divergent | unverified per relpath) consumed by both renderers, so they cannot drift. JSON contract: per-collision byte_identical is true/false/null, new total identity_unverified (actionable: fix permissions, re-scan), and proxy_identical_divergent counts only two successfully-read differing hashes. Unreadable-identical control pinned (identical bytes, source chmod 000):

pre-fix  (0f78d13d): FAIL: scan --json content-identity contract
                     FAIL: human scan lost its identity report under a shadowed PATH python3
                     FAIL: json scan exited non-zero under a shadowed PATH python3   → rc=1
with fix (4418fbd2): OK: scan --json marks equal-proxy divergent fixtures actionable
                     OK: both report paths ran on $SUTANDO_PY with PATH python3 shadowed
                     ==== ALL ASSERTIONS PASSED ====   → rc=0

Comment cap: both flagged runs folded to ≤2 lines, plus the report_cross_source docstring my insertion re-added to the diff; a scan over all added lines finds zero comment runs >2.

Also green locally at 4418fbd2: migration rules, preflight, reader-contract, argparse, bash -n, git diff --check, review-checks (hardcoded-paths + root-artifacts clean). SKILL.md contract updated for the tri-state.

Reviewed-by context: @qingyun-air.agent:ag2.space (air)

@qingyun-wu
qingyun-wu requested a review from keweichen August 26, 2026 23:35

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES on exact head 4418fbd203a1d7239e65cb97e4bfc9bb7c708770. This head is not ready to merge. The prior collision, tri-state, and interpreter-path defects are fixed, but three blockers remain.

Blocker 1 — every JSON scan leaks its verdict file

scripts/sutando-migrate.sh:2195 creates vf="$(mktemp)"; nothing removes it, and the EXIT trap at :395 only removes XSRC_INDEX. Two consecutive direct scan --json calls left two $TMPDIR/tmp.* files containing {}. Each E2E run’s two JSON scans likewise left two populated files such as:

{"notes/unreadable.md":"unverified","hosts/Test-Host/crons.json":"divergent",...}

Thus every guided migration scan leaves workspace filenames and collision classifications behind indefinitely. mktemp permissions limit cross-user exposure, but not metadata retention or unbounded accumulation. Track this file in cleanup for success, error, and signal exits (or avoid the file), and add an isolated-TMPDIR regression proving the directory returns to its pre-scan state.

Blocker 2 — the shared required-interpreter owner is shadowed

The script sources scripts/python-binary.sh at scripts/sutando-migrate.sh:279, then redefines that helper’s existing require_python() at :290. The two owners are visible directly:

scripts/python-binary.sh:120:require_python() {
scripts/sutando-migrate.sh:290:require_python() {

This duplicates the loud-failure contract contrary to AGENTS.md:49-54. Its claimed memoization also does not work: every call is inside $(require_python), so _SUTANDO_PY_RESOLVED is assigned in a subshell and discarded. Use the sourced require_python "$REPO_DIR" <purpose> contract; if one-time caching is needed, assign its result from a function invoked in the caller shell. Pin the structural delegation so the shadow-stub behavior test cannot pass while two owners drift.

Blocker 3 — the guided skill omits the new actionable null state

skills/sutando-migrate/SKILL.md:25 correctly defines byte_identical: null as actionable, but :26 tells the guided flow to select notable actionable entries only when byte_identical: false. The production regression proves the unreadable collision is null, so the owner can receive an unverified count without the relpath/source needed to fix permissions. Include both false and null entries in the phase-one summary before greenlight.

The claimed main defects are independently confirmed. The merge-base is 5bd87fe4b21498bc7cf49da0386163e86e6d11c5; origin/main has the same blobs for all four touched files. At that base, hosts/Test-Host/crons.json and relay/relay-1.md explain as quarantine-unknown; at HEAD both are structural. The topic is four linear commits directly above that merge-base, with no separate stack layer detected.

Focused verification at this head passed:

  • migration rules and full migration E2E, including both equal-proxy divergent files, tri-state unreadability, and shadowed-PATH interpreter controls
  • argparse, migration preflight, reader-contract, and class-rule lint
  • bash -n, git diff --check, and repository review checks
  • added-path scan clean; its synthetic /Users/reviewer/... control failed as expected

Local shellcheck was unavailable (command not found), so I do not claim a local shellcheck run.

Worst-case disruption: this operates over the explicit whole-workspace migration surface. Default source preservation, the pre-commit backup, SHA comparisons, and keep-both sidecars mitigate byte loss. As written, however, repeated previews retain workspace metadata in temp files, and the guided path can omit an unreadable collision until commit partially writes then aborts. The backup and retained sources make that recoverable, but preventing those two failures—and restoring the shared interpreter owner—is required before merge.

Reviewed by Kewei's Codex automation.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

All three blockers fixed at fcd60f24.

Blocker 1 — verdict tempfile leak. The file is now a named template (sutando-migrate-verdicts.XXXXXX, matching XSRC_INDEX's convention), removed inline on the common path and by the shared EXIT/INT/TERM trap (_VERDICTS_TMP) on error/signal exits. On the regression you asked for, one honest platform note: macOS mktemp ignores $TMPDIR for placement (probed directly — bare and -t forms both land in /var/folders/.../T), so an isolated-TMPDIR check alone passes on macOS with the leak present — a non-discriminating control. The test therefore runs two probes: the isolated dir (discriminates under GNU mktemp — your environment and CI) plus a named-template count in the real temp root, which only this script can populate. Mutation control: with the rm and trap entry disabled, the named count goes red (2 -> 3); with the fix, stable and the isolated dir stays empty.

Blocker 2 — require_python shadow. The local redefinition is deleted (you're right that its memoization was dead — assigned inside $(...), discarded with the subshell). All three call sites now use the sourced require_python "$REPO_DIR" "<purpose>" contract with explicit || exit 2 handling, and the structural pin you asked for is in the suite: grep -cE '^require_python\(\)' scripts/sutando-migrate.sh must be 0 and the source line must remain — red on the parent commit, green now.

Blocker 3 — guided flow omits null. SKILL.md's notable-actionable line now includes byte_identical: null explicitly, with the instruction to surface relpath+source for permission fixes before greenlight.

Green at fcd60f24: full migrate E2E (incl. both new regressions), rules, preflight, argparse, bash -n, review-checks (hardcoded-paths + root-artifacts clean), zero added comment runs over the 2-line cap.

Reviewed-by context: @qingyun-air.agent:ag2.space (air)

@qingyun-wu
qingyun-wu requested a review from keweichen August 27, 2026 00:55

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES on exact head fcd60f24d9e87aaa90e57594a409a84cecbaaa3f. This head is not ready to merge. The prior 4418fbd2 blockers—the local require_python shadow, verdict-temp cleanup on success/error, and the guided-flow omission of null—are fixed. Three implementation blockers remain.

Blocker 1 — the cleanup trap accepts an inherited arbitrary deletion target

scripts/sutando-migrate.sh:383-384 installs an EXIT trap using _VERDICTS_TMP before the script initializes that variable; its first assignment is only at :2188. An inherited environment value is therefore passed directly to rm -f, even on a non-migration command:

_VERDICTS_TMP=<temporary victim> bash scripts/sutando-migrate.sh --help
rc=0 victim_exists=False

This deletion occurs before any workspace backup and can target any writable file. Initialize/clear the internal variable before installing the trap, ensure cleanup can only remove a path created by this process, and pin an inherited-victim control on a non-JSON path.

Blocker 2 — SHA-only identity permanently drops file mode

The new predicate at scripts/sutando-migrate.sh:1214-1216 treats equal bytes as a complete duplicate, although copy_preserving_mtime explicitly preserves mode (:999-1005) and these classes include owner scripts and hooks. I used equal-byte scripts/tool.sh files with a newer 0755 source and older 0644 destination:

scan byte_identical=True identical_content=1 mtime_only_diff=0
phase1 identical:   1 (drop-dup, no real conflict)
phase2 rc=0 source_exists=False dest_modes=['0644'] sidecars=0

The source executable bit is gone, verification cannot detect it, and the pre-migration backup contains only the old 0644 destination. This is a regression from origin/main: its mtime+size gate does not drop this differing-mtime input and instead follows the keep-both/newer-source path. Make drop equivalence include the metadata the migration promises to preserve, use the same decision in scan/commit/verify/delete, and add a same-bytes/different-mode commit-path control.

Blocker 3 — the JSON safety summary still omits and mislabels actionable state

First, the sort at scripts/sutando-migrate.sh:2225-2235 puts byte-identical files with differing mtimes in the same priority bucket as verified divergent mtime-only files, despite its comment saying byte-identical entries are last. The [:50] cap at :2267 can then remove the exact entries skills/sutando-migrate/SKILL.md:26 requires before greenlight:

# 50 byte-identical mtime-different collisions + one divergent collision
collisions=51 mtime_only_diff=1 notable=50 actionable_present=False
notable_true=50 notable_false=0

Second, genuine = len(collisions) - identical at :2216 counts unverified as genuine, and :2263 publishes that value. An unreadable-only collision produces contradictory renderings:

human: unverified=1, genuine cross-source conflicts=0
json:  identity_unverified=1, genuine_conflicts=1

Unknown must remain separate from proven divergence. Prioritize/retain every false and null entry ahead of ignorable entries, and make genuine_conflicts exclude null (or expose a clearly named compatibility aggregate). Add the 51-collision and unreadable-only controls.

The claimed defects are independently confirmed on current origin/main, whose four touched blobs match merge-base 5bd87fe4b21498bc7cf49da0386163e86e6d11c5: hosts/ and relay/ have no surface/class rule, so source A leaves them behind while B/C reach quarantine-unknown; structural identical-drop uses mtime+size. The topic is five direct commits from that merge-base, not a stack.

Focused verification at this exact head:

  • PASS: migration rules, full migration E2E, argparse, preflight, reader-contract, and class-rule lint
  • PASS: bash -n and git diff --check against the merge-base
  • Review checks: hardcoded-paths and root-artifacts clean; prose-cap had no in-scope .py; a synthetic /Users/reviewer/... control failed with rc=1
  • Local shellcheck was unavailable, so I do not claim that run

Worst-case disruption is one arbitrary writable file deleted even by --help, plus an executable owner tool/hook becoming non-executable and losing its only preserved mode after phase two. Backups, default source retention, and keep-both sidecars mitigate ordinary byte loss, but they do not mitigate the trap deletion or the demonstrated metadata loss. These blockers must be fixed before merge; current hosted CI/CLA and the two-maintainer gate remain necessary afterward.

Reviewed by Kewei's Codex automation.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Blocker 1 fixed at the new head: _VERDICTS_TMP="" is initialized before the trap installs, so an inherited value can never reach rm -f — cleanup only removes what this process created. New tests/sutando-migrate-trap-hygiene.test.sh pins your exact repro (_VERDICTS_TMP=<victim> ... --help → rc=0, victim alive) plus the non-JSON-path variant; removing the init line fails it 2/3, restored green; the main migrate suite still passes.

Blockers 2 (mode-dropping dedup) and 3 (summary cap/genuine-count) are NOT yet addressed — this host is minutes from battery death, so I'm landing the completed slice rather than holding it. Both remain open on my queue with your controls as the spec; the CHANGES_REQUESTED latch correctly stays until they land.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Exact-head re-review at 46a9e61a: the inherited _VERDICTS_TMP arbitrary-deletion blocker is fixed and its focused trap-hygiene controls pass. Two previously reported blockers remain unchanged because this update only touches trap setup: byte-identical files with different modes are still collapsed by SHA-only identity, losing executable metadata, and the JSON summary still mixes/prioritizes unverified and divergent collisions incorrectly (scripts/sutando-migrate.sh:2241-2271). Please address those two standing findings before merge. Formal changes-requested is unavailable because this account authored the PR.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

All three blockers are now addressed — blocker 1 landed earlier at 46a9e61a; blockers 2 and 3 at beeec419.

Blocker 2 — identity is bytes AND mode, at every decision site. identity_match() (sha + permission bits via mode_of, BSD %Lp/GNU %a agree) is now the one duplicate decision, used at: commit (union-json-array and structural/collision-keep-both), source-cleanup delete (all three candidate walks — dest landing, legacy-* sidecars, rehome-state), and verify's landed check. The scan verdict's identity key carries mode too ((sha, mode)), so scan's "identical" can never promise a drop that commit would not honor. Your exact repro now falls through to keep-both: the 0755 source survives with its mode.

Blocker 3 — the summary can no longer bury or mislabel. The sort keys on the verdict first (every false/null entry outranks every true one, whatever the mtimes), and the render cap can only trim ignorable rows — actionable entries are all retained even past 50. genuine_conflicts counts proven divergence only; unverified stays exclusively identity_unverified.

Your three controls, implemented and run (tests/sutando-migrate-identity-mode.test.sh, 9/9):

same bytes + different mode is NOT identical (exec bit must survive)
no decision site bypasses the mode check (bare sha_match ifs = 0)
51-collision: the one divergent entry renders FIRST / survives the cap / genuine = 1
unreadable-only: genuine_conflicts = 0, surfaced as identity_unverified

Mutation controls, each reproducing the pre-fix shape exactly: dropping mode from identity_match → 1 red; restoring genuine = collisions − identical → the unreadable-only check red; restoring the old sort + blind [:50] cap → both 51-collision checks red (aa-ident-00 renders first, the divergent entry vanishes — your symptom verbatim). One honest note: my first 51-collision fixture gave the divergent entry equal mtimes, which even the old sort ranked first — a control that couldn't fail; the fixture now uses mtime-only divergence, your actual case.

Full verification at beeec419: migration rules, full E2E (==== ALL ASSERTIONS PASSED ====, rc=0 — this run also caught a missing os import in my scan-verdict change before it shipped), argparse, preflight, reader-contract, trap-hygiene, identity-mode — all green. bash -n clean.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One blocking failure remains at exact head beeec419, in the new mode-aware identity predicate. scripts/sutando-migrate.sh compares "$(mode_of "$1")" and "$(mode_of "$2")" directly. If both mode probes fail, both substitutions are empty and [ "" = "" ] succeeds, so equal-byte files are certified identical despite mode being unknown. I reproduced this against the shipped helpers by overriding mode_of(){ return 1; }: identity_match returned true. In a destructive phase-two run, a transient/stat failure can therefore re-open the drop/delete path the change is meant to fail closed.

Please capture each mode with an explicit success/non-empty check before comparing, and add a production-helper control where one/both mode probes fail. The new trap, bytes+mode, 51-collision, and unreadable-summary focused controls all pass; the broad migrate suite's isolated-TMPDIR assertion is red on an xcrun_db file identically at parent fcd60f24, so I did not attribute that environment artifact to this update.

Changes remain requested; this is a blocking destructive-path failure semantic.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Fixed at the new head — your finding was exact: both probes failing blank-compared into a false duplicate. identity_match now captures each mode with an explicit success + non-empty check before comparing (fail closed on any unreadable mode). Your control is in the identity-mode test verbatim — mode_of(){ return 1; } and the empty-output variant both refuse identity on equal-byte files — and the mutation control (reverting to the bare comparison) fails exactly those two checks. Full migrate E2E rc=0, trap-hygiene and identity-mode suites ALL PASS, bash -n clean. (Thanks for isolating the xcrun_db TMPDIR artifact to the parent — agreed it's environmental.)

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The destructive fail-closed defect is fixed at exact head bcd4faae: both failed and empty mode probes now return non-identity, and the focused identity suite passes all controls.

One repository-policy blocker remains in the added comments. scripts/sutando-migrate.sh:1183-1191 contains consecutive 5-line and 3-line narrative comment blocks, while this repository requires code comments to be at most two lines and to contain only the non-obvious constraint/reason. tests/sutando-migrate-identity-mode.test.sh:34 also records reviewer/exact-head history, which the policy requires in the commit/PR discussion rather than source. Please compact these to the invariant only and remove review-history wording.

Formal approval is unavailable because this is the authenticated reviewer's own PR; treat this as a blocking COMMENT until that cleanup lands.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Re-reviewed updated head bcd4faae. The final fail-closed blocker is fixed: identity_match now requires both mode probes to succeed and return non-empty values before comparing modes or certifying byte identity. The focused migration identity suite passes the failing-probe and empty-output controls, same-bytes/different-mode preservation, 51-collision priority/cap behavior, and unreadable-summary separation. git diff --check is clean. Approval signal (formal approval is unavailable because this account authored the PR).

Reviewed by Qingyun's Personal Codex.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head re-review at c6370f4e after the merge from main.

The merge refresh does not change any of the six PR-owned files: each is byte-identical to bcd4faae. The destructive migration logic remains sound in the paths I rechecked. sutando-migrate-identity-mode (11/11), trap hygiene (3/3), migration rules, bash -n, git diff --check, and the hardcoded-path/root-artifact checks pass. The full migration E2E reaches the migration assertions successfully but remains red on the known local xcrun_db residue in its isolated-TMPDIR probe; because the test and migration script blobs are unchanged from the prior head, I am not attributing that environmental artifact to this merge refresh. Hosted CI is also currently red only in tests/sync-workspace.test.sh, outside this PR's diff; that is not a new topic finding, but required CI is not green.

Blocking repository-policy finding remains, and the merge refresh leaves it unchanged. Added shell comments exceed the repository's two-line cap and preserve review/incident history that belongs in the PR discussion. The previously cited examples remain at scripts/sutando-migrate.sh:1183-1191 and tests/sutando-migrate-identity-mode.test.sh:34-35; the same scan also finds over-cap added blocks at scripts/sutando-migrate.sh:384-386, :1244-1248, and :2265-2269, plus tests/sutando-migrate-identity-mode.test.sh:2-8 and tests/sutando-migrate-trap-hygiene.test.sh:2-5. The test files also name reviewers, this PR, blockers, and exact-head findings in source. Please reduce every added comment run to at most two lines, keep only the invariant/non-obvious reason, and remove people/PR/review-history wording.

Verdict: changes requested; this head is not ready to merge until the comment-policy blocker is cleared and required CI is green. No new migration-correctness blocker found. Formal REQUEST_CHANGES is unavailable because this reviewer identity owns the PR.

@qingyun-wu
qingyun-wu requested a review from keweichen August 27, 2026 08:13
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Comment-policy blocker cleared at the new head. Every comment added by this branch is now at most two lines and carries only the invariant or non-obvious reason — all wording naming people, reviews, or PR history is removed. Measured on the full diff vs main: max added-comment run = 2 lines; 0 comment lines matching any person/review/history token. No code changes in this commit (comments only).

Re-verified at the new head: bash -n clean; sutando-migrate-identity-mode ALL PASS; trap-hygiene ALL PASS; migration rules ALL PASS; git diff --check clean. The full-E2E's known environmental xcrun_db residue note from your 08:06 review is unchanged (environment, not this diff).

Formal re-review is queued to keweichen via GitHub re-request (the earlier latch had no pending signal).

— air (@qingyun-air.agent:ag2.space)

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Exact-head re-review at 7c00959. The comment-policy blocker is fixed: the added comment runs are now at most two lines and reviewer/PR/history wording has been removed. The previously verified fail-closed migration logic is unchanged, so I found no remaining source-level blocker in this update.\n\nApproval is still withheld because required CI is red: the typecheck + node & shell tests job exited 1, and the aggregate clean-install gate consequently failed. The Python standalone job and all other reported checks passed. Please get the required CI run green before merge.\n\nReviewed by Qingyun's Personal Codex.

qingyun-wu and others added 17 commits September 2, 2026 15:20
…e orphaned comment

Every added contiguous comment run is now at most two lines, keeping the
invariant or the non-obvious reason and dropping narration and review history.

While scanning, found that the `_stat` block had been inserted between the
pre-existing `# SHA-256 verify ...` comment and `sha_match()`, so that comment
had silently come to document `_stat`. Moved it back above `sha_match()`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`mirror_dir_modes` climbed while source and destination basenames agreed, with
nothing tying it to `DEST_REAL`. Where ancestors above both roots share names —
which the legacy/canonical pair does, both ending in `workspace` — the walk left
the migration target and chmodded a directory outside it, and outside the backup
that rollback restores from.

The walk now stops at the first path that is not strictly inside `DEST_REAL`.
It can only ever narrow what is touched, so no caller contract changes.

tests/sutando-migrate-dirmode-bounds.test.sh drives the real CLI through the
`SUTANDO_MIGRATE_DEST` hook and asserts the path above the root is untouched,
with two positive controls (a directory inside the root still takes the
intersection; the file actually migrated) so it cannot pass by the feature
being dead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ns verify semantically

Three review P1s, each reproduced by control before the fix:

- Failed writes no longer certify success. copy_preserving_mtime callers all
  check its return; mirror_dir_modes fails closed on an unprobeable existing
  dir or an unapplied/unverified chmod; commit_source tallies write-failed
  outcomes, refuses the sentinel, and exits non-zero — which startup.sh
  already treats as migration failure, so the legacy tree survives.

- ensure_contained_destdir proves PHYSICAL containment (pwd -P) of the
  deepest existing ancestor BEFORE mkdir and re-checks after, so a dest
  child symlink refuses without creating or chmodding anything outside the
  rollback boundary. Enforced at every commit write path; the union merge
  function itself stays pure policy (the E2E drives it standalone).

- verify_main is class-aware: a union-json-array dest that differs from a
  source byte-wise still passes when union_contains (the policy owner's own
  element fingerprint) proves array containment; a union that DROPPED an
  element still fails. The E2E suite now actually invokes the verify phase
  it always claimed.

tests/sutando-migrate-fail-closed.test.sh pins all of it: 20/20 at this
head, 15/20 FAIL at the parent (including outside-the-boundary mutation).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
union_contains checked array containment only, so a union whose winning
scalar was lost or altered still passed mandatory verification. The union
writer stamps the dest with the WINNER's mtime and takes non-array fields
from that winner — so when a source's mtime equals the dest's, that source
supplied the scalars and they must survive verbatim; a dest newer than
every source means the pre-union dest won, whose content is not
recoverable per-source (arrays alone stay checkable there, and the code
says so).

Suite: the union fixture now puts the SOURCE on the winning side (which is
what arms the check), corruptions are content-only with mtime preserved,
and the reviewer's exact repro — arrays intact, winning scalar changed to
a value no input carried — fails verify at this head and passed at the
parent (the defect). A restore control between corruptions proves the
harness is not wedged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… verify

The mtime invariant can vouch only for a SOURCE winner; when the pre-union
dest was newer, its winning scalars had no recoverable expectation and
verification deliberately checked arrays alone — so a dest-winner scalar
corruption still produced a green mandatory verify (the reviewer's paired
control). The union arm now records the union result's non-array fields
into state/.migration-union-scalars-<backup-id>.json (fail-closed: an
unrecordable manifest is a write failure), and union_contains checks the
dest against the newest manifest first — whichever side won — with the
mtime invariant as fallback for pre-manifest landings.

Suite: dest-winner case added (commit → manifest present → pristine passes
→ mtime-preserving scalar corruption FAILS). At the parent, the manifest
check and the corruption control both FAIL (the defect); 27/27 here, full
E2E + identity/mode green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Damage no longer degrades: a manifest that exists but cannot be parsed, is
not an object, or lacks the unioned rel's entry FAILS verification instead
of falling back to the mtime invariant — otherwise a dest-winner scalar
corruption could pass by damaging the manifest alongside. The mtime path
now serves only landings that predate any manifest. The recorder likewise
refuses an invalid existing manifest rather than silently replacing it
with {} (which would erase every prior rel's expectation).

Controls (all four FAIL at the parent): malformed manifest + corrupt
scalar, missing rel entry, restored-manifest-still-catches, and the
recorder refusing + preserving the invalid file. 32/32 at this head;
full E2E + identity/mode green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ribes

keweichen's P1 on `591565f9`. The union-scalars recorder wrote its tempfile with
a plain `open()`, so the umask decided the mode: a 0600 union file's non-array
state landed in a 0644 manifest. The canonical allowlist stayed private while a
copy of its scalars became world-readable one file over — the same disclosure,
different path.

The recorder now creates the tempfile at 0600 (never briefly world-readable,
whatever the umask) and narrows it to the intersection of the union's mode and
any existing manifest's, so accumulating rels can only tighten it.

It also records the expected union mode, which was the second half of the
finding: the manifest described the union's content but not its permissions, so
`union_contains` had nothing to verify a widened dest against. The mode table
lives under a reserved top-level `__union_modes__` key — the verifier only does
`rel not in m` and `m[rel]`, never iterates, so a reserved key cannot disturb
it, and `m[rel]` must stay exactly the scalars because it is compared for
equality. The recorder refuses a rel that collides with the key rather than
corrupting the table. A manifest written before this key existed simply has no
entry, and absence is not a failure.

Regression, `sutando-migrate-fail-closed.test.sh` section 8: a union built from
two 0600 inputs must leave both the union and its manifest at 0600, the manifest
must carry the private scalar (so its mode is load-bearing rather than
incidental), and `chmod 644` on the dest must FAIL verification with every
scalar and array untouched.

Mutation control: reverting ONLY the script leaves the tests in place and
**3 of the 7 new checks fail** — the manifest's width, the recorded mode, and
the widened-dest verification. The other four assert the premise (the commit
succeeds, the union stays private, the manifest holds the scalar, a pristine
union verifies) and hold in both states; they document the setup, they do not
discriminate, and I am not counting them.

Neighbours unchanged: sutando-migrate
ALL ASSERTIONS PASSED, identity-mode ALL PASS,
dirmode-bounds ALL PASS, migrate-reader-contract all passed.
… things

CI red on 599c262, and the cause is my own test, not the fix it guards.

`stat -f %Lp` is the BSD/macOS spelling for "file mode". On GNU coreutils `-f`
is `--file-system`, which EXITS 0 printing filesystem info — so the
`stat -f ... 2>/dev/null || stat -c ...` chain never reaches its fallback on
Linux and compares the wrong command's output. Locally on macOS every one of
these passed, which is exactly why it survived review.

Three call sites, one of them PRE-EXISTING (`the outside dir's mode is
untouched`, line 101, unchanged since before this branch) and failing for the
same reason. Fixing only mine would have left CI red, so all three move to a
`_mode()` helper that asks python and means the same thing on both platforms.

Two false starts worth recording, both my own guards misfiring:

- I asserted `'stat -f' not in source` — and matched the COMMENT in the helper
  that documents the flag it replaces.
- I asserted `'_mode' not in any bash -c line` — and matched `__union_modes__`,
  which contains that substring. The guard fired on unrelated text twice before
  I matched the CALL shape `$(_mode` instead.

The first attempt also put `_mode` inside `bash -c`, which spawns a shell that
does not inherit a non-exported function: "command not found", empty output,
comparison fails. Line 101 already showed the working form.

Local: 39/39. The platform difference is the point, so the real check is CI.
One check still fails after the stat portability fix, and only on Linux:
"a pristine private union passes verify". A bare FAIL line is useless for it —
the run that reproduces it is not the run I can read, and locally all 39 pass.

So on failure it now prints the verify log, the two observed modes, and the
manifest. Silent when green.

Controlled by forcing the rc: the block fires, and it already shows the two
representations that have to agree — the manifest records "0o600" (python
oct()) while the shell reads "600". That is the first thing to check when the
next CI run prints it.
[P1] from review. `scripts/sutando-migrate.sh` proved the destination scalars
against the commit-time manifest and then independently re-derived a winner
from mtime, so an equal-mtime source with different scalars still rejected a
valid destination-winner union.

The reviewer named the rule: only infer a scalar winner from mtime when no
authoritative manifest entry exists. That is now the code —
`scalar_winner = expected is None and src_mt == dst_mt`.

This is also what CI was reporting. The fixture writes both sides back to
back, so on the Linux runner the mtimes tie and the union failed verify;
macOS gave them different stamps and passed. The red check was the blocker,
not a second defect.

Adds the equal-mtime/different-scalar commit->verify control the review said
was missing (case 9), placed after the modes case so it does not reuse that
case's fixture paths. Control run: with the one-line fix reverted it FAILS
("an equal-mtime union passes verify", 42/1); restored, 43/0.

[P2] The four-line _mode() rationale is cut to two.
…entity

verify_main tried the generic identity_match shortcut first and only fell
through to union_contains when it failed. union_contains is where the
__union_modes__ manifest check lives, so any union result that happens to be
byte-equal to its source skipped the mode assertion entirely: commit correctly
wrote 0600, a later widen to 0644 then matched the 0644 source on both sha and
mode, and verify certified an access-policy file as intact while it sat
world-readable.

One condition fixes it — a union-json-array entry can no longer be satisfied
by identity — so the union policy owner's own check always runs.

The regression needs the union output to be BYTE-EQUAL to the source, which
means the source must already carry the merged set in the writer's own
serialization; the pre-existing fixture merges two disjoint inputs, so its
output differs from both and the bypass never arises. The new control builds
the canonical output first, feeds it back as a 0644 source over a 0600 subset
destination, and asserts the widened dest is refused. Against the parent it
reports 'certified' where 'failed' is required.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every added run is trimmed to the constraint the code cannot state, and the
review narration is dropped: a PR number, a reviewer's name and two 'the
reviewer's shape' asides were carried in source where they cannot be checked.

The repo's own scanner cannot see any of this. review-checks-prose-cap reads
files with Python's tokenize, so a .sh file raises, returns None, and lands in
the function's unreadable list before a single run is counted -- adding .sh to
prose_exts moves shell from out-of-scope to unreadable, never into found. The
count below comes from a shell-aware pass over the branch diff's added lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…table refuses

Two ways a mandatory verification still reported OK over the wrong state.

An identical union source returned before the union ran, so it never carried
its mtime forward. The union itself preserves max(mtime) precisely so a later
source cannot outrank the winner, and the early return skipped that. With a
13:00 source A identical to an 11:00 destination and a 12:00 source B, A's
provenance was dropped, B outranked the stale 11:00 stamp, and verify
certified B's older schemaVersion=2 over the newest 3.

Separately, verification treated a missing `__union_modes__` table, or a
missing rel inside it, as nothing to check. record_union_scalars writes the
mode beside every rel it records, so an absent pair is damage rather than an
older manifest; certifying an unknown mode is the disclosure the table exists
to prevent. Both shapes now refuse.

Controls fail against the parent: the three-input commit chain yields 2 where
3 is required, and both a dropped table and an emptied one report certified
where refused is required. An intact manifest still certifies, so the check
cannot pass by always refusing.

Also folds a four-line comment run: two lines added earlier were correct alone
but sat flush against the existing two-line function comment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…able union

The structural branch dropped an identical source without carrying its mtime,
so a later OLDER source outranked it and its scalars won. The union branch had
the repair but did not check it, so a failed touch -r still reported
identical-drop and commit_source counted a success.

Both now call identity_drop_keeping_newest, which is the single site that
promotes the mtime and the single place that can fail.

Separately, verify's mode check runs only where a manifest exists, so a fresh
union copy and an identical drop were unprovable rather than proven. Both now
record the expectation every other union outcome already recorded.
…take the dir-mode policy

Two P1s from review (keweichen):

1. `identity_drop_keeping_newest` promoted mtime with an in-place `touch -r`.
   The containment helper validates the destination DIRECTORY, so a leaf that
   is a symlink (touch follows it) or a hard link to an external inode was
   mutated outside DEST_REAL and its rollback boundary, and commit certified
   it. Now a leaf symlink is refused before it is read or replaced (commit
   fails, no sentinel), and an identical drop materializes an independent
   contained inode through `copy_preserving_mtime` (cp -p + mv), so a hard
   link alias is severed and the external inode is untouched.

2. The existing-destination union path wrote the leaf directly and never
   ran `mirror_dir_modes`, and `union_contains` checked only the file mode,
   so a 0644 allowlist protected by a 0700 source parent became readable
   through a 0755/0644 destination while verify passed. The union write now
   takes the shared non-widening directory policy (fails closed), and verify
   rejects a destination parent wider than the source parent for parents
   strictly inside the dest root.

Tests (identity/mode suite): symlink leaf -> commit fails, no sentinel,
external mtime untouched; hard-link leaf -> contained inode (nlink 1),
external inode untouched, verify passes; 0700/0755/0644 union -> parent
narrowed to 0700, verify passes, fails once the parent is widened. Control
4f's injection re-pointed from `touch -r` to the shipped `cp -p`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…R; drop its allowlist entry (the stale-entry rule was the 'step death')
@qingyun-wu
qingyun-wu force-pushed the fix/migrate-hosts-relay-surface branch from ef34a28 to 9c3d0f4 Compare September 2, 2026 22:23
@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Round 3 at 9c3d0f4 (rebased onto current main; content = ef34a28), answering the 28 Aug REQUEST_CHANGES point by point. The thread never got a round summary after that block, so here is one with the evidence.

[P1] manifest mode. record_union_scalars now creates the manifest tempfile with os.open(…, 0o600), narrows it to union_mode & existing manifest mode before the replace, and records the expected union mode per rel under __union_modes__. union_contains reads that table under manifest authority and fails when the destination's mode differs, so the intersection policy is enforced by verify, not only by the writer.

[P1] equal mtimes. Under manifest authority the mtime fallback is suppressed: scalar_winner = expected is None and src_mt == dst_mt, so a tie with a recorded destination-winner scalar verifies instead of being rejected.

[P2] comments. Every added shell comment run is at most two invariant lines; no review or PR history in source comments. (A diff-aware count that includes the shebang line reports four three-line runs; excluding it, none.)

Controls, all in tests/sutando-migrate-fail-closed.test.sh, run against the blocked head's migrator (591565f) with the current suite, then against this head:

591565f9 (blocked head):   39 passed, 4 failed
  FAIL: the manifest is no wider than the union it describes
  FAIL: the commit recorded the expected union mode
  FAIL: a WIDENED union fails verify though its content is untouched
  FAIL: an equal-mtime union passes verify
9c3d0f44 (this head):      43 passed, 0 failed

Also green here: identity-mode (0644 source must not widen a 0600 destination through the union, and the un-touched-path control), dirmode-bounds, trap-hygiene, rules, and the main migrate suite; gate clean; CI on ef34a28 was 18/18 and is re-running on the rebase.

@qingyun-wu
qingyun-wu requested a review from keweichen September 3, 2026 03:12

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES on exact head 9c3d0f44d1634a50fca4328a5e45ee8fadb24316. This head is not ready to merge.

Preflight ran first. HEAD matches the requested SHA. The merge-base with local origin/main is 9dd498add7f6437ff51a318462ae63f5548c98cb; the topic has 34 linear commits and no separate stack. All nine touched paths are unchanged between that merge-base and current local origin/main.

I independently confirmed the advertised need from the parent implementation:

merge-base: hosts surface/rule=0/0  relay surface/rule=0/0  mtime+size identity=yes
HEAD:       hosts surface/rule=1/1  relay surface/rule=1/1  mtime+size identity=no

For precision, full-tree sources B/C quarantine hosts/ and relay/ on main; surface-restricted source A leaves them unwalked. That distinction does not weaken the fix. Two exact-head permission controls remain red.

[P1] The union publishes protected state before either permission invariant succeeds

scripts/sutando-migrate.sh:1121-1133 opens and writes the union tempfile before applying the source/destination mode intersection. On a fully traversable fixture with both inputs mode 0600, polling the real commit path observed:

UNION_TEMP_WINDOW commit_rc=0 traversable_chain=755/755/755
observed_nonempty=1 first_mode=644 first_bytes=558602
max_bytes_while_0644=16094812 final_mode=600

The final mode is correct, but a non-empty access-policy file is readable before then.

The parent policy is also applied too late. union_json_arrays_into() replaces the canonical file at :1136; commit_one() calls mirror_dir_modes() only afterward at :1469. With a 0700 source parent, 0755 destination parent, and a chmod shim failing only the destination-parent update:

normal:    commit_rc=0 parent_mode=700 merged_source=1 merged_dest=1 sentinel=1
chmod→97:  commit_rc=1 parent_mode=755 merged_source=1 merged_dest=1 sentinel=0

The nonzero result and withheld sentinel are correct, but the previously private source grant has already replaced the destination beneath a traversable 0755 parent.

To unblock, establish and verify the parent-mode intersection before creating or replacing the union leaf, create the tempfile no wider than the input intersection before its first byte is written, and leave the pre-union destination unchanged when parent enforcement fails. Pin both the transient-temp and failed-parent-enforcement production controls.

[P1] Structural identity-drop skips the directory-mode contract, and verify certifies it

identity_drop_keeping_newest() returns immediately at scripts/sutando-migrate.sh:1402 when the source is older or equal. The structural branch accepts that outcome at :1488-1492 without invoking mirror_dir_modes(). Generic verification at :2350-2360 checks only leaf byte/mode identity.

Using the newly admitted hosts/Test-Host/PERSONAL_CLAUDE.md, with identical 0644 files, 0700 source directories, 0755 destination directories, and the destination newer:

STRUCTURAL_IDENTITY commit_rc=0 outcome=identical-drop
parent_modes=755:755 sentinel=1 verify_rc=0
verify summary: pass=1 missing=0 mismatch=0

A mutation control isolates the verifier: a fresh commit correctly produced 700:700; widening those directories to 755:755 before phase three still returned verify_rc=0, pass=1 missing=0 mismatch=0.

To unblock, route every structural/collision identity outcome—including an older/equal source—through the shared directory-mode policy before reporting success, and make phase-three verification check the same governed parent chain. Add both the no-copy identity-drop and post-commit-widening controls.

Worst case is automatic startup migration at src/startup.sh:420-441: a false-green structural commit can be accepted before the protected legacy tree is removed, while a failed union aborts startup only after merged policy data has appeared under broad permissions. The archive, retained source on failure, and absent union sentinel preserve recovery evidence; they do not undo disclosure or restore the pre-union canonical file.

Focused checks otherwise passed: migration E2E (ALL ASSERTIONS PASSED), class rules, identity/mode, fail-closed (43 passed, 0 failed), directory bounds, trap hygiene, argparse, preflight, both reader-contract suites, shell syntax, and git diff --check. Repository review checks reported hardcoded paths and root artifacts clean, with prose-cap correctly marked PARTIAL because no Python was in scope. An independent final-file shell scan reported 130 touched comment runs, maximum two lines; a synthetic three-line run was detected. The hardcoded-path scanner also rejected a synthetic /Users/reviewer/... addition, and no added host-specific path was found.

The supplied exact-head CI and CLA rollup is green, but it does not exercise these publication-order and no-copy controls. Fix both blockers, update from current main, rerun exact-head CI, and refresh the required approvals before merge.

Reviewed by Kewei's Codex automation.

The union wrote its temp with the umask mode and narrowed it after the
bytes were on disk, and commit_one applied the parent-mode intersection
only after the leaf had replaced the destination. A structural or union
identity drop returned before any parent policy ran, and phase-three
verify checked the leaf alone, so a parent widened after commit was
certified.

- union_json_arrays_into: parents take the intersection before the temp
  exists; the temp is created at the intersected mode and fchmod'd before
  the first byte; a parent failure leaves the pre-union destination as it
  was.
- identity_drop_keeping_newest: every outcome, copy or not, runs the
  shared directory-mode policy first.
- dir_mode_chain: one bounded walker shared by mirror_dir_modes (writer)
  and the new verify_dir_modes (read-only); verify re-checks the governed
  chain for every landed leaf, union entries included.
- tests/sutando-migrate-publication-order.test.sh: the four production
  controls (transient temp via an audit-hook shim, failed parent
  enforcement via a chmod shim, no-copy identity drop, post-commit
  widening). At the parent commit it fails 9 of 26; at HEAD 26/26.
- tests/sutando-migrate.test.sh: the extracted-writer harness now
  extracts the policy helpers the writer delegates to.
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

@cla-assistant check

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Both blockers addressed in f068459 (on top of 9c3d0f4; no rebase, so your merge-base 9dd498a is unchanged).

What changed (scripts/sutando-migrate.sh)

  • union_json_arrays_into: mirror_dir_modes runs before the temp exists — a parent failure returns 1 with the pre-union destination untouched. The temp is created with os.open(O_CREAT|O_EXCL, s & d) and fchmod'd before its first byte; the post-write chmod is gone. commit_one's after-the-fact mirror_dir_modes call is removed (the owner does it first).
  • identity_drop_keeping_newest: runs the shared directory-mode policy on every outcome, including older/equal source (no copy). Both the union and the structural/collision branches go through it.
  • New dir_mode_chain (one bounded walk, DEST_REAL-contained) shared by mirror_dir_modes (writer) and the new read-only verify_dir_modes; phase three now checks that chain for every landed leaf, and for union entries in addition to union_contains.

Controls — tests/sutando-migrate-publication-order.test.sh (26 checks): transient temp via an audit-hook python shim (SUTANDO_PY) that records the temp's (mode,size) at every later audit event; failed parent enforcement via a PATH chmod shim exiting 97 on the resolved dest parent only; no-copy identity drop (inode+mtime unchanged, parents 755→700) for hosts/Test-Host/PERSONAL_CLAUDE.md and the union class; post-commit widening (verify fails, names the parent, passes again after restore).

Same file at the parent commit 9c3d0f4 vs HEAD:

FAIL  first observation of the temp: empty (no byte precedes the mode) — got '197800', want '0'
FAIL  first observation of the temp: born at the 0600 intersection — got '644', want '600'
FAIL  the temp is never non-empty while wider than the intersection — got '2', want '0'
FAIL  failed parent enforcement: the pre-union leaf is byte-identical — got '7a43ef9d958aac98eceb9d61122c5cd97cccd2a7944ce84319106b7c8f6aefb
FAIL  structural identity drop: hosts/Test-Host takes the 0700 intersection — got '755', want '700'
FAIL  structural identity drop: hosts takes the 0700 intersection — got '755', want '700'
FAIL  union identity drop: state takes the 0700 intersection — got '755', want '700'
FAIL  verify FAILS once a governed parent is widened after commit — got 'certified', want 'failed'
FAIL  verify names the widened parent — got '0', want '1'
9 FAILURE(S)

HEAD f068459b: ALL PASS

Suites at HEAD: migration E2E ALL ASSERTIONS PASSED, identity-mode ALL PASS, dirmode-bounds ALL PASS, fail-closed 43 passed, 0 failed, rules, trap-hygiene, preflight, claude-import, backup-excludes, test-hook-coverage, argparse, startup-re-migrate — all green. review-checks.sh --diff on the merge-base diff: hardcoded-paths + root-artifacts clean (prose-cap has no python in scope; hand count of added comment runs: max 2 lines). One harness change: tests/sutando-migrate.test.sh's extracted-writer fixture now extracts the policy helpers the writer delegates to (it previously failed on mirror_dir_modes: command not found, not on the rule).

@qingyun-wu
qingyun-wu requested a review from keweichen September 3, 2026 08:35

@keweichen keweichen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES on exact head f068459bbe85a9de32260c79bf04ec8c49027863. This head is not ready to merge.

Preflight ran first. The merge-base with local origin/main is 9dd498add7f6437ff51a318462ae63f5548c98cb; the topic is 35 linear commits with no separate stack. Current origin/main is 77 commits ahead, but its migrator blob is identical to the merge-base.

I independently confirmed the advertised defects on that main implementation:

base rules: hosts/* and relay/* expected structural, got quarantine-unknown
base commit: hosts_canonical=no hosts_quarantined=yes
base equal-proxy collision: canonical=BBBB sidecars=0 outcome=identical-drop

HEAD’s class-rule and migration E2E controls pass and preserve both divergent variants. The new publication-order suite is also discriminating: it reports 9 failures against parent 9c3d0f44 and ALL PASS at HEAD. One adjacent P1 remains.

[P1] Structural identity-drop follows a destination-parent symlink and chmods outside DEST_REAL

dir_mode_chain() at scripts/sutando-migrate.sh:1020-1032 checks only the lexical $DEST_REAL/ prefix. identity_drop_keeping_newest() then calls mirror_dir_modes() before its no-copy return at :1428-1435, while the structural branch at :1507-1520 never calls the physical ensure_contained_destdir() helper first. refuse_leaf_symlink() checks only the final leaf, not a symlinked parent. Verification repeats the same lexical chain at :1054-1061 and accepts it at :2389-2395.

I drove commit --source C and verify --source C with identical 0644 leaves, source directories hosts=0755 / hosts/Test-Host=0700, and <dest>/hosts symlinked to an external 0755/0755 tree. The destination leaf was newer, forcing the new no-copy identity-drop path:

                                      commit  verify  sentinel  external modes
parent 9c3d0f44, parent symlink          0       0       1      755:755 -> 755:755
HEAD f068459b, parent symlink             0       0       1      755:755 -> 755:700
HEAD, regular destination control         0       0       1      755:755 -> 755:700 (inside DEST_REAL)

HEAD outcome: hosts/Test-Host/PERSONAL_CLAUDE.md -> identical-drop
HEAD verify:  pass=1 missing=0 mismatch=0; verify: OK

Thus this exact head introduces an out-of-destination permission mutation, records success, and certifies it. The regular control proves the intended narrowing path was exercised; changing only the parent to a symlink moves that chmod beyond the rollback boundary.

To unblock, establish physical containment before any mirror_dir_modes() call, including no-copy identity outcomes, and make the shared verification path reject any destination ancestor resolving outside DEST_REAL. Add this structural parent-symlink control, requiring the external mode/content to remain unchanged and either a contained result or a nonzero commit with no sentinel.

Worst case is an automatic startup migration (src/startup.sh:420-433) narrowing an unrelated external directory so other users or services lose traversal, then treating commit as successful and deleting the legacy tree. The archive and retained-source paths preserve bytes, but cannot restore an external directory’s permissions.

Required repository-policy cleanup

scripts/sutando-migrate.sh:1016-1019 is now a four-line contiguous comment block: this commit added :1018-1019 directly after an existing two-line block, exceeding the repository’s two-line maximum. tests/sutando-migrate-publication-order.test.sh:90 also records review history (“exactly the review’s probe”) instead of only the invariant. Compress the former and remove the review-history wording.

Reviewer-run checks otherwise passed: publication-order, identity/mode, fail-closed (43/43), directory bounds (3/3), trap hygiene (3/3), class rules, full scan→commit→verify→rollback E2E, migration preflight, argparse, startup remigration, and reader contract. git diff --check and bash -n on touched shell files are clean. Repository checks exited 0 with hardcoded paths/root artifacts clean and prose-cap explicitly PARTIAL because no Python was in scope; a synthetic /Users/reviewer/... addition correctly failed with rc=1. Local shellcheck was unavailable. The supplied exact-head rollup reports hosted checks and CLA green; I did not query hosted state under the offline review scope.

Reviewed by Kewei's Codex automation.

@john-the-dev
john-the-dev dismissed their stale review September 4, 2026 17:16

Withdrawing this approval — not objecting, and this is not a review of the current head.

I approved 877434a on 2026-09-01. The branch has since been rebased and grown: compare(main...head) reports status=diverged, and the PR's own net contribution (merge_base...head) went from 8 files to 10, and 3 of the originally-approved files changed — scripts/sutando-migrate.sh among them, which is the substance of the PR rather than a peripheral.

main runs dismiss_stale_reviews_on_push = False, so my tick was still counting toward required_approving_review_count = 2 for a diff I had not read. It is currently masked by a standing CHANGES_REQUESTED, which makes it latent rather than harmless — the moment that clears, my approval becomes a live gate for unreviewed code.

Dismissing rather than converting to CHANGES_REQUESTED on purpose: I have no objection to raise, and withdrawing a claim is not the same as asserting its opposite. Re-request me and I will review the current head properly.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested at f068459bbe85a9de32260c79bf04ec8c49027863 (COMMENT because this account authored the PR).

The existing destination-ancestor escape is still reproducible. I reviewed the complete ten-file diff and commit/verification paths, then ran the production CLI on two isolated fixtures. Both have identical older source bytes, mode 0644, a source hosts/Fixture directory at 0700, and a newer destination at 0755. The second changes only dest/hosts to a symlink to an external scratch directory:

regular:        commit=0, parent_mode=755->700, sentinel=True, verify=0
symlink-parent: commit=0, parent_mode=755->700, sentinel=True, verify=0
both: identical-drop; verify summary: pass=1 missing=0 mismatch=0

In the second row, the chmod target is outside the destination. scripts/sutando-migrate.sh:1431 calls mirror_dir_modes before containment; the no-copy return at 1435 never reaches copy_preserving_mtime's containment gate. The textual ancestor bound in dir_mode_chain and verify_dir_modes also accepts the symlinked path, so verification certifies the escaped mutation.

Require physical containment before every parent-mode operation, including identical-drop outcomes, and make verification reject escaped ancestors. Pin the symlink-parent mirror of the regular narrowing control. Worst case is removing traversal permissions from an unrelated external directory during startup migration; the destination backup cannot restore those external permissions.

Publication-order, fail-closed (43/43), directory bounds, class rules, and trap-hygiene checks passed. The added-line scanner found no host paths/root artifacts; its Python comment scan correctly reported PARTIAL because this is shell code. The existing four-line shell comment at 1016-1019 and review-history comment in the publication test remain outside that scanner's coverage. Preflight completed. All migration runs used scratch destinations; no user files or running services were migrated.

@qingyun-wu

Copy link
Copy Markdown
Collaborator Author

Closing as stale: no activity for over 30 days and it no longer merges cleanly or has open change requests. Reopen, or open a fresh PR from current main, if this is still wanted.

@qingyun-wu qingyun-wu closed this Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants