You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Proofs: regrade stored runs when only tests/ changed #139
Is your feature request related to a problem? Please describe.
When a check changes after the first proof, the proof runs again from scratch: NOP, Oracle and every negative control, plus any stored agent runs. In an agent-optimization loop on a FreeCAD CAD agent, a run with 8 check changes spent about 226 minutes on proofs (4 to 85 minutes each). Most changes touched only tests/, so the agents' and controls' outputs were still valid; only the grades needed redoing. Harbor already supports this (harbor jobs regrade re-runs the verifier on stored trials). The task-create proof, however, fingerprints tests/ and rejects reused runs, so the skill cannot simply be told to regrade.
Describe the solution you'd like
When only tests/ changed, let a proof regrade stored control and agent jobs instead of re-running them, and record that it did (old and new tests/ hashes, the jobs regraded). Re-run only what the change can affect, for example a new negative control.
Describe alternatives you've considered
Regrading by hand outside the skill, which loses the proof record.
Activity