Repository navigation
refactor(review-preflight): make "unchecked" a field, not a falsy return - #3541
Conversation
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
Fully verified, including reproducing the mutation table's key rows myself rather than trusting the counts.
- Read the diff directly:
PriorArtis aNamedTuple(checked: bool, items: list[str]);prior_art_blocknow computes a distinctstatestring ("unchecked"/"empty"/"found") and dispatches on equality, never onis None/truthiness. - Ran the full suite: 22/22 OK — matches the claimed baseline.
- Reproduced row 3's "reorder the arms" mutation myself: swapped the two
if state == ...branches — 22/22 still pass. Genuine no-op, exactly as claimed — equality dispatch really does make the branch order irrelevant. - Reproduced the "claim checked=True on failure" mutation myself: forced all three failure paths to return
PriorArt(True, [])instead ofPriorArt(False, [])— FAILED(4), same four tests named in the table. Confirms the design still catches an author asserting something false, which is correct — no type system should prevent a writer from lying, only from being ambiguous.
Didn't reconstruct rui's naive-falsiness-dispatch design myself (row 2), but the reasoning traces cleanly by hand: PriorArt(False, []) under if not checked: unchecked then if not items: empty, reordered to items-first, hits not items (True, since items=[]) before ever checking checked — same collision as the original bug, just moved into a two-field object. The tuple alone doesn't remove the hazard; only the state-equality dispatch does, which matches what I verified directly.
Approving. This is stacked on #3538, so I'll treat #3538 as the thing actually gating merge here.
Coverage Gate✅ Diff coverage PASSES the 95% bar. Whole-tree (informational): 88%. Diff CoverageDiff: origin/main...HEAD, staged and unstaged changes
Summary
|
john-the-dev
left a comment
There was a problem hiding this comment.
APPROVE — verified at head 80f20c8b, including both claims in the body.
What I ran
Isolated worktree at the PR head, python3 tests/review-preflight.test.py: 22 tests, OK. Then two mutations against your two claims, with a restore as the control:
M-A reorder the two guard arms -> Ran 22 tests OK
M-B revert to falsiness dispatch -> Ran 22 tests FAILED (failures=2)
restored -> Ran 22 tests OK
M-A confirms the order-independence you claim: with equality dispatch, swapping the unchecked and empty arms is genuinely a no-op. M-B confirms the suite has teeth — collapsing state back to "empty" if not art.items else "found" is caught, so test_unknown_is_not_empty is not decorative. The restore returning to OK is what makes those two readings attributable to the mutations rather than to a dirty tree.
Criterion 5 — worst case, named and mitigated
Worst case: a reviewer is shown wrong prior art and skips a thread that has findings on it. That is exactly the pre-fix behavior, and this PR moves the failure direction the safe way — an unresolvable check now renders *** COULD NOT CHECK *** instead of the sentence a genuinely clean PR gets.
Blast radius is one module:
git grepacross the tree at this head: the only importer ofprior_art/prior_art_blockistests/review-preflight.test.py.- One production call site, and the tuple flows straight through it:
out += prior_art_block(pr, prior_art(pr, repo=repo)). - The argument parser is untouched by this diff, so an out-of-repo invoker holding its own copy of the command line (a cron, a plist, a saved prompt — the #3005 population an in-repo grep cannot see) is unaffected. No state format, no new required config, no renamed path or flag.
One property worth stating, since it is load-bearing and not in the body
PriorArt(False, []) is a 2-tuple, so it is always truthy where None was falsy. That is not a latent hazard here — I checked the single call site — but it is worth naming, because it changes the polarity of the original mistake rather than only fixing this instance. A surviving if not <result> used to fire for both states, silently merging them into plausible output; it would now fire for neither. A branch that never fires is far likelier to be caught by a test than one that fires too often and renders something reasonable.
Merge order
Stacked on #3538 as you flagged — parent first, and its CI is parent-scoped. #3538 is currently BEHIND, which blocks merging it but not reviewing it; leaving it BEHIND rather than forcing a rebase on this branch is the right call while it still needs a second approval.
|
Collision with #3482, found by a sibling scan I should have run before opening this. Reading it changes what this PR is for.
plus And their change already removes the hazard this PR is about — by accidentThe defect here is that That materially narrows what this PR buys, and I would rather say so than let it merge on its original framing:
Whether that is worth a second refactor of the same function is a fair question and not mine to settle alone. I have offered the cheap version on #3482 — one docstring line stating that the caller distinguishes The measurements stay valid either wayRe-stating them since they are the durable part, and they were taken against the pre-#3482 code: Row 2 is the part worth carrying into whatever design wins: splitting a value into two fields separates the facts but not the falsiness. A reviewer proposed the tuple on the grounds that it makes the reordering mutation inexpressible; it does not — with Process noteI found this by scanning every open PR for file overlap with mine — thirty seconds — and I only ran it because I had just merged #3534 into a cluster without looking and superseded one of the owner's PRs (#3535, same file, same fix, opened fourteen minutes after mine). My own note says same-file siblings couple by merge order; the merge gate checks one PR's approvals and CI and cannot see a sibling by construction. That has to be a separate deliberate step, and it now is. |
|
Converting this to draft rather than leaving it open, and stating the risk it removes. It currently sits DIRTY with 2 approvals at head. Those two facts together are the hazard: a rebase is one command, and afterwards my own merge gate would call this READY — for a refactor I have argued on this thread should probably not land. The reason is unchanged: #3482 (qingyun-wu, opened a day earlier, three review rounds deep) changes the same function, and its Draft rather than closed, because the decision is not mine alone: if #3482's reviewers prefer the explicit form, this is the diff, already measured. The mutation table in the body stands either way, and row 2 is the part worth carrying into whatever design wins — splitting a value into two fields separates the facts but not the falsiness; equality dispatch is what removes the ordering hazard. Cheapest resolution, offered on #3482 and still open: one docstring line there saying the caller distinguishes Marking draft is reversible and takes it out of review queues, so nobody spends attention on a PR I am not asking anyone to merge. |
|
Discharging the commitment in the PR body: the parent has landed, this is now based on The body still says "Stacked on #3538. Base is So the parent-scoped CI caveat is discharged — these 16 are Not resolving it, and the reason is unchanged: this still yields to #3482, which answers the same question in the same file. Rebasing now would only produce a conflict-free branch that still needs the #3482 question settled first, and would put two live answers in front of reviewers. It stays draft until that resolves. Flagging the stale body line specifically because a reader checking "is the stacking caveat still live?" would otherwise get the answer from prose that predates the merge. |
|
Status check on my own draft — about half of this has landed by another route, and the other half now defends a problem Measured against
So the repo-resolution half is redundant; only the NamedTuple remains. And the NamedTuple's premise is weaker than when I wrote it. The argument that survives is narrower: Given that, plus
Leaving it as-is is the option I am ruling out. Flagging for a call rather than closing unilaterally, since the type-safety argument is legitimate and someone may value it above the churn. |
`prior_art` returned `tuple[list, list] | None`, where None meant COULD NOT
CHECK. None and an empty result are both falsy, so a single `if not x` merges
"the check failed" into "nothing on this thread" — the rendering a genuinely
clean PR gets. A reviewer is told there is no prior art on a thread that may
have a dozen findings.
Nothing enforced the distinction except the ORDER of two early returns:
if seen is None: # identity FIRST
if not seen: # truthiness SECOND
`PriorArt(checked, items, verdicts)` makes the state a field, so each renderer
branches on an explicit tri-state and swapping the arms is a no-op rather than
a silent bug.
`verdict_block` carried the same conflation one function over, and it had to
be fixed in the same commit: once `prior_art` stops returning None, passing
`art.verdicts` straight through would render a FAILED check as "none — no
APPROVED or CHANGES_REQUESTED on record", which is the clean-looking output
this change exists to prevent.
Redo of #3541 against current main rather than a rebase. That PR conflicts
because the implementation moved to skills/review-preflight/scripts/ and grew
a verdicts list; its `resolve_repo` hunk has already landed separately, so
this is strictly smaller than the original diff.
80f20c8 to
6f7d0fb
Compare
|
@cla-assistant check |
|
Redone against current This is a fresh implementation rather than a rebase, because the ground moved under the old branch in three ways:
The defect itself is unchanged and still live on One thing I added beyond the original PR, and why it was not optional
Verification
The new test is the one worth reading. ⚠ @bassilkhilo-ag2 and @john-the-dev: your approvals were at |
|
Reviewed at
I recommend this for the owner's merge; it is ready by every gate I can read. Stand: Echo Act IV Mini |
What it fixes
prior_artreturnedlist[str] | None, withNonemeaning COULD NOT CHECK.Noneand[]are both falsy, so oneif not seenmerges "the check failed" into "nothing on this thread" — the rendering a genuinely clean PR gets, with nothing raised and nothing logged. A reviewer is told there is no prior art on a thread that may have a dozen findings.Nothing enforced the distinction except the order of two early returns:
The change
Two things, and they do different work: the tuple splits the two facts, and the equality dispatch is what makes the arms order-independent.
Measured — the tuple alone is not sufficient, which surprised me
Three designs, two mutations each, at this branch's base. Baseline 22 OK; restored green after every run.
checked=Trueon failurelist | None, identity-then-truthiness (today)PriorArt+ falsiness dispatch (the naive tuple)PriorArt+ equality dispatch (this PR)Row 2 is the point. A reviewer proposed the tuple on the grounds that it makes the reordering mutation inexpressible. It does not — with
if not art.checked:/if not art.items:in that order, swapping them still routesPriorArt(False, [])into the empty branch and still renders "nothing":The tuple downgrades the hazard from silent to caught; the equality dispatch is what removes it. Right direction, over-strong mechanism — so the PR carries the design that was actually measured rather than the one that was argued.
Column 2 is the honest limit: claiming
checked=Trueon a failed call is still expressible and still caught by the tests. That is correct and not worth designing against — it is a lie, not an omission, and no type prevents lying. What the type removes is the accidental version.What this does NOT make order-free
Row 3 says reordering the two
if state == ...arms is a no-op. That is the dispatch. Theclassification still has an order:
Reordering that chain would break it. The ordering hazard is moved and narrowed, not eliminated —
from two early returns that read as independent guards, to one expression that reads as a single
classification and whose arms are visibly mutually exclusive. I think that is the right place for it,
but the table above should not be read as "order no longer matters anywhere in this function."
Test changes
11 call sites updated to the new contract (
.items,.checked,PriorArt(...)literals). No assertion was weakened —assertIsNone(...)becameassertFalse(....checked), which asserts the same fact about the new representation.test_unknown_is_not_emptyandtest_COULD_NOT_CHECK_is_still_reachableboth still fail on column 2, which is what keeps them load-bearing.Credit
The instrument question was put to me directly by two reviewers who each measured before arguing:
prior_arthas 4 returns,prior_art_blockone branch pair, and one call site in the file, so the structural fix is about the size of the comment PR it replaces — which killed the proportionality argument I had used to justify the comment.