You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Protect each eval set's manifest in the hooks, so a floor or mark edit asks first #696
A mistake guard, for whoever is free. The Factory Builder asked for it when ruling on the security read of #692 (a gate set holds a floor per criterion). It was already on the Maintainer's optional list.
What
Add each eval set's manifest, evals/*/cases.json, to protected in config/hooks.json, with an is such as "an eval set's manifest: its gate, bars, floors and natural marks". A session's edit to a manifest then asks first, as an edit to a rubric does.
Why
A gate set's floors (criterion_bars), bars, least of a kind and natural marks all live in its manifest under evals/, which no person need approve. A change to them between commits can lower a floor or, with the cases unchanged, publish groups whose difference is under ORIGIN_LEAST (#695: "Differences between a gate set's manifests across commits can isolate a group under ORIGIN_LEAST"). The hook only guards against a session's mistake; CI and review still hold the rule.
Notes
config/hooks.json is itself protected: the edit is pre-approved by the Maintainer 2026-11-08, ruled by the Factory Builder. Say so in the PR body.
Check that the pattern form matches what tools/hooks/protected.py reads (config/rubric-*.json is a precedent), and that the hooks' own tests cover the new path.
No case file is named or read; the manifest path alone.
Activity
Hooks: a second set's manifest is named in the test, so narrowing the…