ci(untrusted-input): P17's catalog count is one anchored block, not any **109** in the doc - #5891
Conversation
…ny **109** in the doc
test_catalog_table_matches_the_gate.py compared the number of CATALOG:
lines in specs/numeric/formats_catalog.t27 (the population of catalog-gate's
mandatory-field, 109) with docs/theory/IGLA-FORMAL-RESULTS.md by
`f"**{records}**" in doc` -- anywhere in 27,000 lines. T397's own "**109**
records" kept it green whatever P17's re-take said.
P17's figure is now one generated block,
<!-- catalog-count anchor=15ac5b5b1... -->109<!-- /catalog-count -->,
written by --write and checked against the CATALOG: count in the tree of
the anchor (fetched by SHA in CI), its RE-TAKEN AT heading, and the
re-take's mandatory-field row. Exactly one block, in P17. Nothing outside
the marker is read. The help string stays a live check (one build).
The pattern is #5801's, not a second one: the marker, fetch-by-SHA,
--write/--anchor and the block-to-re-take checks move into
scripts/ci/anchored_count.py, imported by both gates.
test_retaken_propositions_still_match.py prints byte-identical output and
its --write is still a no-op.
--self-check (second line of the CI step) runs the whole script on four
temp copies and asserts rc and the check named: correct block 0, wrong
block 1, wrong block + right number planted elsewhere 1, marker removed 1.
The old check returned 0 on all four.
Closes #5881
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check's words also appear on its own `ok` line, so "the text is in stdout" held on a copy that went green. Measured: with the block check made substring-tolerant, the wrong-block controls exited 0 yet reported "names it: True". They now look for the words on a FAILED line. Refs #5881 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
📓 NotebookLM Notebook linked to this PR
This notebook contains session context, decisions, and artifacts for this work. |
gHashTag
added
the
bee-reviewed
|
Reviewer bee: reviewed head Source of truth.
Catalog count change. I tested this in a throwaway #5801 gate. I ran both the master and head copies against the real doc. Their output is byte-identical, and Checks. 28 pass, 2 skip, none fail. |
1 participant
Closes #5881
What the check compared
scripts/ci/test_catalog_table_matches_the_gate.py(Untrusted Input Gate, "The published catalog table still matches the tree") countsCATALOG:lines inspecs/numeric/formats_catalog.t27-- the population oft27c catalog-gate'smandatory-field, 109 -- and then requiredover the whole 27,000-line
docs/theory/IGLA-FORMAL-RESULTS.md. T397 (line ~17104) says**109** records and **436** fields, so that alone kept it green whatever P17's re-take said (confirmed by the reviewer of #5801).What it compares now
The same pattern as #5801 (#5799), not a second one:
<!-- catalog-count anchor=15ac5b5b16c86076e3cf256ef78cf0371611e096 -->109<!-- /catalog-count -->, written bypython3 scripts/ci/test_catalog_table_matches_the_gate.py --write(run once to place it).CATALOG:count in the tree of the anchor (fetched by SHA when the depth-1 checkout lacks it), against itsRE-TAKEN AT \15ac5b5`heading, and against the re-take'smandatory-field` row. Exactly one block, and it must sit in P17. The "the check no longer exists" sentence must be in that re-take, not anywhere in the file.bootstrap/src/main.rshelp string must match today's catalog (one build, one number), andno-spurious-layoutmust be gone from the source.Shared helper. The marker regex, fetch-by-SHA,
--write/--anchorand the block-to-re-take checks moved fromtest_retaken_propositions_still_match.pyintoscripts/ci/anchored_count.py, which both scripts import. That script's output is byte-identical before and after (diffed), and its--writeis still a no-op on the committed doc. One tightening, the same for both scripts: the re-take's measured row is looked up across the whole quoted re-take (the block may sit above the table, as in P17), and a re-take with no such row is red rather than vacuously green.Negative controls
Copies of the document in a temp dir; the committed doc was not edited. Old = the script at
origin/masterrun on its own tree holding the copy; new = this script with--doc <copy>. Exit codes from the runs.110)states the record count of its anchor**109**/| **109** |planted elsewherestates the record count of its anchor109)**109**)is a marked catalog-count blockThe old check was green on every copy. The repo has no pytest harness for
scripts/ci; following its convention (tools/check_catalog_integrity.py --self-check), the four controls are--self-check, run as a second line of the same CI step. Each runs the whole script on a temp copy and asserts its exit code and the check it names (states the record count of its anchor,is a marked catalog-count block). The entry is dispatched beforemain()does anything and returns its own verdict, so amain()whosereturn 1becamereturn 0fails it -- the T86 lesson recorded in.github/workflows/catalog-count-gate.yml.Mutants, each committed first and reverted with
git checkout:... or f"**{n}**" in doc)--self-checkrc 1 (both wrong-block controls fail)main()'sreturn 1->return 0--self-checkrc 1 (all three red controls fail)1146->1147(the #5801 gate, now on the shared helper)test_retaken_propositions_still_match.pyrc 1,states the corpus of its anchorThe first mutant also caught a weak control: the check's words appear on its own
okline, so a "text in stdout" match passed on a copy that went green. A red control now needs the words on aFAILEDline (second commit).Not changed: T397's own prose
**109** recordsstays as T397's statement; it is simply no longer read by any check.🤖 Generated with Claude Code