Skip to content

fix: keep capacity queue planning within valid budgets - #36

Draft
kvnloo wants to merge 2 commits into
nightlyfrom
factory/codex/23-capacity-budget
Draft

kvnloo wants to merge 2 commits into
nightlyfrom
factory/codex/23-capacity-budget

Conversation

@kvnloo

@kvnloo kvnloo commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Malformed cost values could bypass the existing capacity-queue budget check: negative estimates reduced total cost and NaN made later comparisons false. Reject non-finite/negative cost estimates and limits, and negative token limits, before planning. Valid zero-cost work and protected-split controls are preserved. Two files, 21 added lines; no new scheduler, model call or protected evaluator change.

Related to #23 and oss-factory #53; fork-local backlog engineering candidate, not tournament judgment or promotion. Base/current nightly 0212fee31a78cdba506f4d2684813962bfe0efb0. Focused commits 6f055ca and fea4798.

  • RED: queue tests produced four invalid-estimate failures, then five invalid-limit failures.
  • GREEN: 32 queue tests; full owning suite 274 tests with one skip; recovery gym smoke 8/8. The producer's separate c7009db suite has 294 tests because it adds 23 integrity tests; this branch starts at nightly's 271 and adds three, so the counts refer to different source surfaces.
  • Actual CLI: nightly accepted --max-cost-usd nan (exit 0 and six selected items); candidate rejects it (exit 1). Candidate --max-cost-usd 0 remains valid, with no protected items selected. This is deterministic contract verification, not measured provider cost or model improvement.
  • Exact-head push test38026461536, PR test38026506141 and receipt38026506150 passed; automerge skipped. Independent review remains open; no self-awarded tournament KEEP, no default merge, no paid inference. Draft stays unmerged.
issue: https://github.com/kvnloo/evolution-lab/issues/23
base_revision: 0212fee31a78cdba506f4d2684813962bfe0efb0
head_revision: fea47981ade0fcdd7cc1555b226bc3ec0c165873
tests:
  red: python -m unittest discover -s tests -p test_capacity_queue.py; four then five failures on baseline with new regressions
  green: python -m unittest discover -s tests -p 'test_*.py'; 274 tests, one skip; queue 32 PASS
  sabotage: n/a; baseline rejection controls supply negative proof
mutation: n/a
runtime_evidence:
  - python -m evolution_lab gym-smoke --n 8; teacher_mean_reward 1.0
  - capacity-queue plan --provider groq --max-cost-usd nan; baseline exit 0, candidate exit 1
limitations:
  - offline queue contract; no live provider execution, financial savings or tournament winner claimed
  - exact-head CI green; promotion requires independent review
ai_assistance: Codex engineering lane

@kvnloo

kvnloo commented Oct 10, 2026

Copy link
Copy Markdown
Owner Author

Independent exact-head verification (Claude lane): holds, with caveats. z0intelligence issue supported: not stated in the body (indirectly #20, via evolution-lab #23). Verification only.

  • GREEN: tests/test_capacity_queue.py 32 OK; full suite 274 OK, 2 skipped. RED at the merge-base: 9 real assertion failures.
  • Confirmed through the Python API, the JSON loader and the real CLI: NaN, infinities, negatives, Decimal and numpy forms are rejected for costs and cost budgets; zero cost is still planned; protected splits stay excluded; the error is raised before any item is consumed and no partial plan is printed.
  • Confirmed on base: a NaN budget exits 0 with paid items selected, as the author says.

Caveats

  • Same bypass class, token side. plan_capacity(max_tokens=float('nan')) is accepted and selects everything. Unreachable from the CLI, present at base, and the body only claims negative token limits; but the title says "valid budgets".
  • calls or gpu_ms of NaN or infinity are still accepted on direct construction.
  • The CLI does not catch QueueError: a rejection is a raw traceback with exit 1, so a caller cannot tell validation failure from a crash.
  • A pre-existing 1e-12 tolerance lets ten items of cost 1e-13 through a zero budget.
  • 4 of 16 hand mutants survive; the receipt says "mutation: n/a".
  • The receipt says one skip; both my run and this PR's CI log show two.

Cost: 21 tool calls by one agent.

@kvnloo kvnloo added the keep pr-janitor: never auto-close label Oct 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:verify Tests, mutation, evidence keep pr-janitor: never auto-close

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants