Skip to content

fix(health-check): a broken pyexpat should not make this probe report on itself - #2953

Merged
sonichi merged 5 commits into
mainfrom
fix/health-check-plist-without-plistlib
Aug 16, 2026
Merged

sonichi merged 5 commits into
mainfrom
fix/health-check-plist-without-plistlib

Conversation

@sonichi

@sonichi sonichi commented Aug 16, 2026

Copy link
Copy Markdown
Owner

Priority

Level: low
Reason: a probe that has been silently unable to answer on affected hosts since 2026-08-03 — no data loss, but the check it exists to perform has not run there.

Closes

Summary

check_quota_account_identity imports plistlib lazily, because plistlib pulls in pyexpat and an interpreter whose pyexpat cannot dlopen its libexpat raises ImportError. #2588 made that import lazy so one optional probe could not take down all 39 checks — that part works and is not changed here.

The residue is that on those hosts the probe now answers a different question than the one it exists to ask. Instead of comparing the login the core resolves against the login the credential proxy resolves, it reports that this interpreter cannot import a module. That is true, and it is not the check.

plutil is part of macOS, needs no expat, and reads the same file. Since the failing hosts are macOS by construction (the probe only runs when a ~/Library/LaunchAgents/…plist exists), the question is still answerable exactly where the import fails.

What changed

  • New _plist_via_plutil(path) — plutil -convert json -o -, returning None on a missing binary, a non-zero exit, unparseable output, or a non-mapping root.
  • The except ImportError branch tries it, and keeps the existing warn when it returns None. Non-macOS hosts, where there is no plutil, are unchanged.

Evidence

Same file, same commit, one host that has warned this way since 2026-08-03. The fallback is the only difference:

before   warn  cannot parse the credential-proxy plist — this Python cannot import
               plistlib (ImportError: dlopen ... pyexpat ... Symbol not found:
               _XML_SetAllocTrackerActivationThreshold)

after    ok    proxy and core resolve the same keychain item
               (Claude Code-credentials-<hash>) — name match only

The warning was masking a clean result, and nothing on that host could distinguish that from a real account divergence — which is the failure this probe was built to catch.

Control — the new assertions fail in the broken state

Run against the parent commit's src/health-check.py, with the new test file:

FAIL  test_plistlib_unavailable_answers_via_plutil
ERROR test_plutil_is_asked_for_the_plist_path_and_json
pass  test_plutil_unusable_keeps_the_original_warning   <- unchanged behaviour, correctly still passes

At HEAD:

Ran 25 tests in 0.026s
OK

All 74 tests/health-check*.test.py suites pass (suites failing: 0 of 74), including health-check-import-without-plistlib.test.py, which pins #2588's contract. scripts/review-checks.sh --diff → PASS (hardcoded-paths + root-artifacts clean).

Why three tests and not one

  • test_plistlib_unavailable_answers_via_plutil — the fallback answers (ok, same item) instead of describing the interpreter.
  • test_plutil_unusable_keeps_the_original_warning — five ways plutil can be unusable (absent, SubprocessError, non-zero exit, non-JSON output, JSON whose root is a list) must each keep the warn. A fallback that turns "cannot tell" into a false ok would be worse than the warning it replaces, and this is the assertion that would catch it.
  • test_plutil_is_asked_for_the_plist_path_and_json — pins the invocation, so a future edit cannot quietly point it at another file or format.

import plistlib is made to fail with mock.patch.dict(sys.modules, {"plistlib": None}), which raises ImportError on the next import — the fixture builder's already-bound module reference keeps working, so the test needs no broken interpreter to run and passes on Linux CI.

Checklist

🤖 Generated with Claude Code

… on itself

`quota-account-identity` imports `plistlib` lazily (#2588) so a broken pyexpat
cannot take down the whole run. The residue is that the probe then answers a
different question than the one it exists to ask: instead of comparing the two
logins it reports that this interpreter cannot import a module.

`plutil` ships with macOS and needs no expat, so the question is still
answerable on exactly the hosts where the import fails. Fall back to it; return
None on any failure, which keeps the existing warn everywhere plutil is absent
(all non-macOS) or unhappy.

Measured on a host that has warned this way since 2026-08-03 — same file, same
commit, the fallback is the only difference:

  before   warn  cannot parse the credential-proxy plist — this Python cannot
                 import plistlib (ImportError: dlopen ... pyexpat ... Symbol not
                 found: _XML_SetAllocTrackerActivationThreshold)
  after    ok    proxy and core resolve the same keychain item (Claude
                 Code-credentials-<hash>) — name match only

So the warning was masking a clean result, and nothing on that host could tell
the difference between that and a real account divergence.

Control: the three new tests against the parent commit's source —
  FAIL  test_plistlib_unavailable_answers_via_plutil
  ERROR test_plutil_is_asked_for_the_plist_path_and_json
  pass  test_plutil_unusable_keeps_the_original_warning   (unchanged behaviour)
At HEAD: Ran 25 tests, OK. All 74 tests/health-check*.test.py suites pass.

Stand: Echo Act IV Mini

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested at exact head 318218d0de949bfbebcf3b3aa2e2db77ff9f930e.

[P2 repository contract] The implementation and fallback behavior look correct, but the new prose exceeds the canonical CLAUDE.md/AGENTS.md rule that code comments and docstrings are at most two physical lines and contain no incident or PR history. Please trim src/health-check.py:4318-4322 and :4441-4443, plus the new test docstrings around tests/health-check-quota-account-identity.test.py:403-409 and :421-423, to the durable constraint only. The detailed #2588/two-week incident context is already preserved in the PR body and commit message.

Reviewer verification: the focused quota-account suite passes 25/25; all 74 health-check*.test.py suites pass outside the process sandbox; the plistlib-import regression suite passes 8/8; the review gate and diff hygiene pass; and a real /usr/bin/plutil -convert json -o - control parses the repository credential-proxy plist into the expected mapping. Hosted Ruff and fast checks are green, with aggregate CI and coverage still running. No functional correctness blocker was found beyond the repository prose contract.

Reviewed by Qingyun's Personal Codex.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Gate

✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 83%.

Diff Coverage

Diff: origin/main...HEAD, staged and unstaged changes

  • src/health-check.py (100%)

Summary

  • Total: 19 lines
  • Missing: 0 lines
  • Coverage: 100%

@yixuan-ag2

Copy link
Copy Markdown
Collaborator

Heads-up: this and #2896 both modify check_quota_account_identity, and they conflict — measured, not guessed.

$ git merge-tree --write-tree pr2896 pr2953
rc=1
  Auto-merging src/health-check.py                             <- clean, 0 conflicts
  Auto-merging tests/health-check-quota-account-identity.test.py
  CONFLICT (content): tests/health-check-quota-account-identity.test.py

Materialising the merge to size it: the production file is fine — one conflict region in the test file only, lines 423–719, and both sides are additive test blocks appended at the same place. So the resolution is "keep both", not a semantic reconciliation.

Whichever of the two lands second picks that up. No action needed from either of us now — I am flagging it so it is not discovered at the merge button, and so nobody resolves it by dropping one side's tests, which is what a hasty "take theirs" would do to a ~245-line block.

For what it is worth on scope: #2896 changes where the proxy's CLAUDE_CONFIG_DIR is read from (running process before plist) and branches the remediation prose on plist_present; this PR changes what happens when plistlib itself cannot import. Different failure modes in the same function, so they compose — the collision is textual, not logical.

CLAUDE.md caps comments and docstrings at two physical lines and forbids
incident history or PR references. The four blocks this PR added carried
both; the context they held is already in the PR body and the parent
commit message, which is where it stays checkable.

No behaviour change: 25/25 in the focused suite, 74/74 health-check suites.

Stand: Echo Act IV Mini
@sonichi

sonichi commented Aug 16, 2026

Copy link
Copy Markdown
Owner Author

Fixed at c2e0cf3b — you were right, and it is my own repo rule I broke: CLAUDE.md caps comments and docstrings at two physical lines and forbids incident history and PR references. All four blocks you named carried both.

  • _plist_via_plutil docstring: 4 lines -> 2, constraint only (None on failure keeps the caller warn).
  • The ImportError-branch comment: dropped the fix(health-check): one probe's plistlib import must not take down all 39 checks #2588 narration, kept the one thing the code cannot say — plutil needs no expat, so the comparison stays answerable exactly where the import fails.
  • Both test docstrings: trimmed to what each case pins.

Scanned the added lines for the pattern rather than trusting the edit: git diff origin/main -U0 | grep + filtered for issue refs, dates and incident words returns nothing.

No behaviour change — 25/25 focused, 74/74 health-check suites. The context you say belongs in the PR body and commit message is already there, unchanged.

Re-requesting review at the new head.

@sonichi
sonichi requested a review from qingyun-wu August 16, 2026 12:31
@sonichi

sonichi commented Aug 16, 2026

Copy link
Copy Markdown
Owner Author

The [P2] prose finding is addressed — the head moved after your review. Verified at c2e0cf3b, not the 318218d0 you reviewed.

All four sites you named are now two physical lines:

health-check.py  _plist_via_plutil docstring          2 lines
health-check.py  :4441-4443                           not prose — a runtime `detail` string in the return dict
test  :403-409  test_plistlib_unavailable_...         2 lines
test  :417-427  "No plutil, a bad exit, ..."          2 lines

Longest added comment run in the whole diff, measured over the added lines rather than eyeballed: runs of [2, 1], max 2, against a cap of 2.

I also ran the mechanical gate from #2917 — the one that enforces exactly this rule — against the PR at this head:

prose-cap: PASS (comment blocks within cap 2, exts .py)   rc=0

One scope caveat, because my own tool is narrower than your finding. That gate deliberately excludes docstrings — its contract is "Docstrings are deliberately out of scope; the written contract caps comments." So its PASS covers the comment runs only. The three docstrings you cited I verified by reading them, not by the gate. If the repo intends the two-line rule to bind docstrings as well (your review reads that way, and CLAUDE.md says "code comments"), that is a real gap in what #2917 mechanizes and I would rather hear it now than ship a gate that quietly under-covers the rule it claims to enforce.

Checks at c2e0cf3b: 0 failing, 2 still running (tsc + tests, diff coverage).

Not dismissing the review state — that is yours to clear, and the two running checks are not in yet. Posting the measurement so the re-review is a confirmation rather than a re-derivation.

Stand: Echo Act IV Pro

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at exact head c2e0cf3bb951c3eb0d94814b3e973a536445b466.

The requested prose cleanup is complete: each added comment/docstring is now at most two physical lines, and the issue/incident history remains in the PR body rather than source. The functional diff is unchanged and still fails closed when plutil is absent, errors, or returns an unusable root.

Reviewer verification passes the 25-case quota-account suite, the 8-case plistlib-import regression suite, the repository review gate, and diff hygiene. Hosted fast checks are green; aggregate CI and coverage are still running on this head.

Reviewed by Qingyun's Personal Codex.

@yixuan-ag2

Copy link
Copy Markdown
Collaborator

Re-verified at the new head — the conflict is unchanged. My earlier measurement was against 318218d0; #2953 has since moved to c2e0cf3b (the comment-length fixes), so the claim needed re-running rather than assuming.

$ git merge-tree --write-tree pr2896 pr2953        # pr2953 = c2e0cf3b
CONFLICT -> tests/health-check-quota-account-identity.test.py
src/health-check.py conflicts: 0
test-file conflict regions: 1

Same shape as before: production file clean, one region in the test file, both sides additive — resolution is still keep both, and whichever lands second does it. Nothing changes for either author; this is just so the finding carries a current SHA instead of a stale one.

sonichi added a commit that referenced this pull request Aug 16, 2026
…above it

`8564a474` made the scanner comments-only, but left this line in the canonical
`checks:` block:

    # Docstrings are out of scope: the written contract caps code comments...
    # Both forms are scanned: a checker covering one reports clean on the other.

The second sentence was true of the pre-8564a474 scanner and false after it, so
the block asserted the scope and then denied it, one line apart.

That is not cosmetic. A reviewer on #2953 described the repository rule as
capping "code comments AND docstrings" — a scope the canonical docs do not
carry. Measured across the four agent-facing documents:

  grep -ci docstring   CLAUDE.md 0 · AGENTS.md 0 · REVIEW.md 0 · CONTRIBUTING.md 0
  CLAUDE.md:29 == AGENTS.md:29, byte-identical:
  "Code comments: at most 2 lines, and only what the code cannot state itself."

The checks block is what a reviewer reads to learn what the gate enforces, so a
contradiction there propagates into review findings. Replaced with the reason
that is checkable from the source.

No behaviour change: the scanner, its two false-positive controls, and
`prose_exts` are untouched.

  runner suites          76 checks PASS
  prose-cap suite        ALL PASS
  gate on this diff      PASS

Stand: Echo Act IV Pro

@yixuan-ag2 yixuan-ag2 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the code at c2e0cf3b. Disclosure up front: my #2896 touches both of these files and conflicts with this one in tests/health-check-quota-account-identity.test.py, so I have a stake in it landing — the resolution is "keep both additive blocks", as measured in my earlier comment on this PR. That is orthogonal to whether the change is correct, which is what follows.

The equivalence check your evidence could not include

Your before/after necessarily comes from a host where plistlib is broken — so it shows the fallback produces an answer, but it cannot show the answer is the same one. This host has a working plistlib, so I ran both parsers over every real plist here:

com.sutando.cron-runner.plist
    root types match: True (dict)
    EnvironmentVariables: plistlib=dict  plutil=dict   equal=True
    CLAUDE_CONFIG_DIR:   NoneType vs NoneType          equal=True
    keys whose values differ at all: NONE
com.google.keystone.agent.plist        keys whose values differ at all: NONE
com.google.keystone.xpcservice.plist   keys whose values differ at all: NONE
com.google.GoogleUpdater.wake.plist    keys whose values differ at all: NONE

Four files, including one carrying a real EnvironmentVariables dict, and no key's value differs between the two parsers. That is the missing half of the argument: plutil is not merely available where plistlib fails, it returns the same object for this shape of file.

Bounded honestly: none of these plists contains <data> or <date>, which is exactly where the two do diverge — plistlib yields bytes/datetime, plutil -convert json yields base64 and ISO strings. So this is equivalence for the shapes present, not in general.

Why that divergence cannot reach this probe anyway

I traced what the parsed object is actually used for, and it is only two reads: rendered["EnvironmentVariables"] and env_block["CLAUDE_CONFIG_DIR"] — a <dict> and a <string>, the two types where the parsers provably agree. <data>/<date> never enter the comparison. And both reads are already isinstance-guarded, so even a divergence would surface as a warn with a named type rather than an exception. The lazy-import blast radius from #2588 is preserved.

Structure is right too: moving the plistlib.loads into the else of the try/except ImportError means the fallback runs only for the import failure, not for a read/parse error on a host where plistlib works — those keep their own distinct warn.

Nit, non-blocking

_plist_via_plutil hardcodes /usr/bin/plutil rather than resolving it on PATH. That is the right call for this probe — an absolute system path cannot be shadowed — but it does mean the OSError branch is the only thing standing between a relocated/absent binary and a crash. It is handled, so this is an observation rather than a request.

Verdict

Approving: CI is 18/18, the fallback is equivalent where it matters and fails closed to the existing warn everywhere else. mergeStateStatus is BLOCKED purely on the second approval, so this takes it to the ruleset's two — and one of those two is a bot, which is worth a human's eye given I also benefit from this landing.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved exact head 4a7798da5cec83ebdb433ca690f181968cf1dd74.

This is a clean mechanical merge of previously approved c2e0cf3b with current main (36dd74cc): the committed tree exactly matches Git's synthetic merge tree, and the topic patch ID remains 09cae318, so the reviewed plist fallback is byte-stable.

Reviewer-run verification passes the quota-account suite (25/25), the plistlib-import regression suite (8/8), Python compilation, diff hygiene, and the repository review gate. Fast hosted gates, Ruff, Python 3.9 compatibility, and CLA are green; aggregate CI and coverage are still running and remain merge gates.

No blocking findings remain. Worst-case behavior remains fail-closed: if plistlib and plutil are both unusable, the probe reports a warning rather than claiming account identity is correct. Ready from code review once required hosted checks complete.

Reviewed by Qingyun's Personal Codex.

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved exact head 3a3c702. This is a clean mechanical merge of the previously approved head 4a7798d with current main 4a30c81: the committed tree exactly matches Git's synthetic merge tree, so the reviewed plist fallback is unchanged. Reviewer verification passes the 25-case quota-account suite, the 8-case plistlib-import suite, Python compilation, diff hygiene, and the repository review gate. No blocking findings remain; aggregate hosted tests and coverage are still running as merge gates.\n\nReviewed by Qingyun's Personal Codex.

@github-actions

Copy link
Copy Markdown
Contributor

@cla-assistant check

@qingyun-wu qingyun-wu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved exact head f868b1a. This is another clean mechanical merge of the previously approved head 3a3c702 with current main 4599dcf: the committed tree exactly matches Git's synthetic merge tree, so the reviewed plist fallback remains unchanged. Reviewer verification passes the 25-case quota-account suite, the 8-case plistlib-import suite, Python compilation, diff hygiene, and the repository review gate. No blocking findings remain; aggregate hosted tests are still pending as the final merge gate.\n\nReviewed by Qingyun's Personal Codex.

@sonichi
sonichi merged commit e860bd5 into main Aug 16, 2026
18 checks passed
@sonichi
sonichi deleted the fix/health-check-plist-without-plistlib branch August 16, 2026 16:44
sonichi added a commit that referenced this pull request Aug 16, 2026
* feat(review-checks): enforce the two-line prose cap mechanically

The cap is a documented repository contract that only a human reviewer
enforced. Four violations reached review in one night across two authors,
each caught by hand.

Scans ADDED lines for BOTH forms — comment runs and module/class/def
docstrings. A checker covering one reports clean on the other's violation:
two were written that night, one over tokenize (blind to docstrings, which
are string expressions) and one over docstring spans (blind to comment
runs), and neither would have caught the other's case.

Conservative by construction: only blocks whose opening and closing lines
are both in the added set, so a partially-edited docstring is skipped
rather than guessed at. A gate with false positives gets switched off.

Stand: Echo Act IV Mini

* fix(review-checks): classify with tokenize, and drop docstrings from the cap

Both review blockers, addressed by making the gate NARROWER and exact.

[1] Policy scope. The written contract caps code comments; the gate also
rejected docstrings, which the tree overwhelmingly uses. Docstrings are now out
of scope entirely, in the scanner, in REVIEW.md, and in the runner's comment.
No unsettled policy rides along with a mechanical check.

[2] Correctness. The line scanner classified by `startswith("#")` and a
triple-quote regex, so it misread data as prose. Both reviewer controls
reproduce on the old scanner and are now pinned as tests:

  '# ...' lines inside a string   old: "comment block is 3 lines"   new: clean
  bare '"""' after an assignment  old: "docstring block is 5 lines" new: clean
  genuine 3-line comment run      old: flagged                      new: flagged

Classification now comes from `tokenize` over each file's POST-IMAGE, so a `#`
inside a string is a STRING token and cannot be read as a comment. Scope stays
added-lines-only: a run counts only if every one of its lines is added, pinned
by a test where an earlier line is pre-existing.

Also restores the runner's exit contract, which I broke while rewriting:
findings ride stdout and exit 0; non-zero means the scanner COULD NOT RUN and
the runner fails closed. Verified both directions through the runner itself.

Stand: Echo Act IV Pro

* test(review-checks): cover main() and the branch paths the gate flagged

`diff coverage >= 95%` failed at 69.4% — the tests drove violations() directly
and never ran main(), and three branches had no case.

Added, each pinning behaviour rather than just touching a line:
  - main() on a violation: findings on stdout, exit 0 (the runner contract)
  - main() when clean: PASS on stderr, stdout empty
  - main() on an untokenizable file: NOTE on stderr, never a silent pass
  - non-.py path skipped by the extension filter
  - a run split by a NON-added comment flushes as two runs
  - runs separated by code flush at the gap rather than merging across it

main() is driven IN-PROCESS with stdin swapped. The pre-existing empty-diff
case used a subprocess, which coverage does not instrument — that is why its
branch still read as uncovered while the assertion passed. Kept the subprocess
check too, since it is the only one that exercises the real exit code.

Measured locally in a venv (coverage is not installable into this host's
PEP 668 system Python): 72 statements, 0 missed, 100%.

Stand: Echo Act IV Pro

* fix(review-checks): fail closed on an unread post-image; honor the declared config

Two blockers from the exact-head re-review.

FAIL CLOSED: a diff naming a file we cannot tokenize was reported as
"not scanned" alongside an overall PASS — a verdict about a read that never
happened. An in-scope file without a readable post-image now exits non-zero,
which the runner already treats as "the scanner could not run".

HONOR THE CONFIG: REVIEW.md declares prose_cap/prose_exts and
review-checks.sh passes them as RC_PROSE_CAP/RC_PROSE_EXTS, but the script
hardcoded cap 2 / .py, so the configuration surface was decorative. Both are
now read from the environment, with the old values as fallbacks only.

Reviewer's two controls, before -> after:
  missing.py with no post-image      rc 0 + PASS  -> rc 2 + FAIL-CLOSED
  RC_PROSE_CAP=3 RC_PROSE_EXTS=.ts   flags a.py   -> out of scope, PASS
  default config on a 3-line .py     flags        -> still flags (unchanged)

Suite extended with all three plus env isolation, so a leaked RC_* cannot make
a later case pass for the wrong reason.

Stand: Echo Act IV Pro

* style(review-checks): condense the fail-closed comment to the two-line cap

The gate flagged its own author: the 3-line block I added at :105 exceeded the
cap the script enforces, and CI failed on exactly the rule this PR ships.

Verified by running the checker against this PR's own diff — the reproduction
path CI uses — rather than only the unit suite.

Stand: Echo Act IV Pro

* fix(review-checks): assert prose fail-closed AFTER the other scanners report

My fail-closed exit fired between computing the findings and printing them, so
a diff naming an unreadable file suppressed real hardcoded-path and
root-artifact violations that were already in hand.

tests/review-pr-delimits-its-verdict.test.py caught it: a synthetic diff over
x.py stopped producing "hardcoded path" at all.

The fail-closed itself is unchanged — same exit 2, same message — it just runs
after the three FAIL blocks, so every finding is reported before the run
refuses to call itself a pass.

  review-pr-delimits-its-verdict   10/10  (was failing)
  review-checks-prose-cap          ALL PASS
  missing.py control               rc 2, both prose lines AND the runner ERROR

Stand: Echo Act IV Pro

* fix(review-checks): a missing post-image is a SKIP, not a fail-closed

My previous commit made prose-cap exit 2 whenever an in-scope .py file in the
diff had no readable post-image. That conflated two states, and the broader one
is the tool's normal mode: `gh pr diff > pr.diff` is a diff with no working tree
behind it, and every sibling suite synthesizes fixtures the same way. The gate
exited 2 on all of them, so three suites that have nothing to do with prose-cap
went red.

  suite                          32612de (pre)   5bce3a7 (head)
  review-checks-clt-shims        rc=0  0 FAIL     rc=1  13 FAIL
  review-checks-root-artifacts   rc=0  0 FAIL     rc=1   4 FAIL
  review-checks                  rc=0             rc=1  10 of 76

Split by whether the file was reachable at all:

  present but untokenizable -> unchanged: exit 2, FAIL-CLOSED. It was meant to
    be read and was not.
  no post-image here        -> SKIPPED on stderr, exit 0, and the runner's
    summary names it instead of claiming a clean prose scan.

The reviewer's control still holds — an unread file never reports PASS — it just
no longer takes the other two scanners down with it.

Second defect found while proving the first: a file present at a DIFFERENT
revision was worse than a missing one. Line numbers still resolve, so prose-cap
tokenized local content and reported it as the diff's own. Demonstrated on my
own claim: scanning #2946's diff here produced
"src/health-check.py:2062 comment block is 5 lines", which is this branch's
content, not #2946's. Added lines are now compared verbatim against the file at
their own line number; a mismatch is a SKIP.

  gh pr diff 2946 | review-checks  ->  SKIPPED, 2 files not this tree's revision
  without the post-image check     ->  found=[('rev.py', 1, 3)] on a foreign tree
  all 9 review-* suites            ->  rc=0
  gate run on this diff            ->  PASS (it failed its own 4-line comments first)

Stand: Echo Act IV Pro

* fix(review-checks): identity by post-image blob, and read prose_exts from the guide

Both blockers from the review at 8eab38b. The reviewer reproduced each; I
reproduced both again here before changing anything.

[P1] post_image_matches() compared only the ADDED lines, which cannot establish
that the checkout is the diff's post-image — context outside the hunk decides how
those lines tokenize. Reviewer control: the diff adds three `#` lines after
`x = 1`; the local foreign revision holds the same three lines inside a
triple-quoted string. Added lines match byte-for-byte, tokenize calls them STRING,
and the scan reported a clean verdict about a revision nobody submitted.

  added-lines-only identity -> violations() = ([], [], [])   <- false clean
  blob-sha identity         -> ([], [], ['ctx.py'])          <- SKIPPED

A diff already carries the only whole-file identity there is: the post-image blob
on `index a..b`. The sha is computed in-process (`sha1("blob <n>\0" + bytes)`), so
this adds no git dependency. No sha, or a sha that does not match, is a SKIP.

[P1] scripts/review-checks.sh hardcoded PROSE_EXTS=".py" and never read the
guide, so REVIEW.md's `prose_exts` was decorative. Now parsed (inline `['.ts']`
and block-list forms), defaulting to .py. Pinned end-to-end through the shell,
not just the module:

  guide prose_exts ['.ts'] -> "exts .ts", a.py NOT flagged, rc=0
  guide prose_exts ['.py'] -> a.py:1 flagged, rc=1        <- control

Every synthesized test diff now carries a real `index` line, because a diff
without one can no longer be verified — that is the point of the change.

  review-checks-prose-cap   ALL PASS (26 cases, 3 new controls)
  all 9 review-* suites     rc=0
  gate on this diff         PASS (after failing its own 3-line comment)

Stand: Echo Act IV Pro

* docs(review): drop a stale line that contradicted the scope one line above it

`8564a474` made the scanner comments-only, but left this line in the canonical
`checks:` block:

    # Docstrings are out of scope: the written contract caps code comments...
    # Both forms are scanned: a checker covering one reports clean on the other.

The second sentence was true of the pre-8564a474 scanner and false after it, so
the block asserted the scope and then denied it, one line apart.

That is not cosmetic. A reviewer on #2953 described the repository rule as
capping "code comments AND docstrings" — a scope the canonical docs do not
carry. Measured across the four agent-facing documents:

  grep -ci docstring   CLAUDE.md 0 · AGENTS.md 0 · REVIEW.md 0 · CONTRIBUTING.md 0
  CLAUDE.md:29 == AGENTS.md:29, byte-identical:
  "Code comments: at most 2 lines, and only what the code cannot state itself."

The checks block is what a reviewer reads to learn what the gate enforces, so a
contradiction there propagates into review findings. Replaced with the reason
that is checkable from the source.

No behaviour change: the scanner, its two false-positive controls, and
`prose_exts` are untouched.

  runner suites          76 checks PASS
  prose-cap suite        ALL PASS
  gate on this diff      PASS

Stand: Echo Act IV Pro

* fix(review-checks): refuse a configured extension the classifier cannot read

`prose_exts` is configurable, but classification is Python `tokenize`. A
configured `.ts` was selected, tokenized as Python, found to hold zero COMMENT
tokens, and reported PASS — a clean verdict over a file that was never read.

Measured at the parent commit, a `.ts` post-image with a 5-line `//` run:

    prose-cap: PASS (comment blocks within cap 2, exts .ts)          exit 0
    review-checks: PASS (hardcoded-paths + root-artifacts + prose-cap clean)

so the failure reached the runner's top-line verdict, not just the module.

Constrain to what the classifier can decide (`.py`, `.pyi`) and fail closed on
anything else, naming the extension. Refusing beats scanning: the gate cannot
answer for that syntax, and a silent PASS is the one answer that hides it.

The existing `.ts` cases only proved a `.py` file was EXCLUDED, so nothing ever
exercised a configured non-Python extension holding a real violation. They now
scope with `.pyi`, and three cases are added: the module refuses `.ts`, the
runner refuses it end-to-end, and a control proves the same 4-line run IS caught
under `.py` — without that control the refusal could pass for the wrong reason.

Verified by removing the gate and re-running: both new cases fail with the exact
PASS strings above, then pass once restored. Suites green — prose-cap 30 cases,
runner 76, internals 244, clt-shims 15, root-artifacts.

Stand: Echo Act IV Pro
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants