Skip to content

v1.91.36.0 fix(cso): Python and Rails repairs replay, Rails verification fits its budget, killed attempts leave no scratch - #3074

Merged
capy-ai[bot] merged 4 commits into
mainfrom
cso-python-replay-determinism
Oct 8, 2026
Merged

capy-ai[bot] merged 4 commits into
mainfrom
cso-python-replay-determinism

Conversation

@garrytan

@garrytan garrytan commented Oct 8, 2026

Copy link
Copy Markdown
Owner

The private evaluator's Lane R ran the real gstack-cso helper end to end against the staged runtime images. It found four runtime verification defects, each reproduced locally on amd64 with images from staging run 37555192492.

Defects fixed

  • Python repairs never replayed. Offline preparation wrote timestamped .pyc bytecode, so two preparations of the same app hashed differently, and replay reported changed preparation inputs. Preparation now sets SOURCE_DATE_EPOCH, so venv, ensurepip and pip write checked-hash .pyc files. Two preparations now differ in 0 files.
  • Rails replays drifted the same way. RubyGems and mini_portile build logs (gem_make.out, mkmf.log, ext/**/tmp/**/*.log) record random build directories. They're now left out of the prepared identity. Compiled extensions stay in it.
  • Rails verification couldn't finish on 4 vCPU. An attempt prepares the app twice, about 245 s each because native extensions compile. The attempt watchdog, the Docker executor and the assertion witness each capped the attempt at 300 s. They now share one bound, MAX_VERIFICATION_ATTEMPT_MS: 900 s for Rails attempts and replays, still capped by the run deadline. Other stacks keep 300 s.
  • A killed helper left scratch behind. resume and finish now reclaim the attempt's supervision, preparation-execution, archive-staging, archive-materializations and verification scratch. They do this under the run lock, and only once the watchdog has recorded cleanup.

Validation

On the tree merged with main (f67c478):

  • typecheck is clean;
  • the CSO suites plus the version and skill-docs tests pass (824 pass, 0 fail);
  • new tests cover the witness bound, scratch reclaim and preparation determinism.

Not yet confirmed: arm64, and the evaluator's full Lane R run on CI. Those happen in the next staging run and dry run after merge.

Open in Capy

… leave no scratch

- Python offline preparation sets SOURCE_DATE_EPOCH so .pyc bytecode is reproducible.
- Prepared identity hashes skip RubyGems/mini_portile build diagnostics that name random build dirs.
- Rails verification attempts and replays get 900 s; the attempt watchdog, Docker
  verification executor and assertion witness share MAX_VERIFICATION_ATTEMPT_MS.
- resume and finish reclaim a killed attempt's scratch once its watchdog recorded cleanup.

v1.91.34.0
@trunk-io

trunk-io Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

@capy-ai capy-ai Bot changed the title v1.91.35.0 fix(cso): Python and Rails repairs replay, Rails verification fits its budget, killed attempts leave no scratch v1.91.36.0 fix(cso): Python and Rails repairs replay, Rails verification fits its budget, killed attempts leave no scratch Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

E2E Evals: ✅ PASS

[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 0/0 · behavior 0/0 · judge 0/0 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 0m00s · cost $0.00

0 executed, 0 reused rule/judge records | 0 manual accepted (unscored; no score-cache credit) | $0 rule/judge cost | reconcile exit: 0

Profile: pr / mode pr; selected behaviors: 0, judges: 0. Deferred to scheduled/release coverage: 3 behaviors and 0 changed prompt files. Deferred checks did not run and receive no PR-pass credit.

Rule and judge shards
Shard Automated result Manual/unscored Executed Reused Status Cost
Fail-closed reconciliation
[test:paid] report: 1/1 slices, 0 planned shards, tier=gate, attempt 1
[test:paid] coverage: profile=pr mode=pr; selected E2E=0, judges=0
[test:paid] deferred: 3 broad behaviors, 0 changed prompts without quick live coverage; these are not PR passes
[test:paid] collector final outcomes: 0 executed, 0 reused; 0 passed, 0 failed, 0 manual accepted (unscored; no score-cache credit) (0 attempt records from 0 collectors; every record counts)
[test:paid] VERDICT GREEN — lane gate/pr, attempt 1
  rule 0/0 · behavior 0/0 · judge 0/0 · quarantined 0 (0 failing the lane)
  SKIPPED 0 · INFRA 0 · INCOMPLETE 0 · unattributed 0 · ACTION REQUIRED 0
  wall 0m00s · cost $0.00
[test:paid] report: every planned shard accounted and passed

Sliced lane: planner → duration-packed executors → fail-closed report. Behavior cases run a pre-registered 3-trial panel (PASS at 2/3 with no contract violation); a PASS 2/3 is shown with its failed trial, never as a clean pass. Reused results retain their original provenance and expiry.

@capy-ai
capy-ai Bot merged commit 1d1ce39 into main Oct 8, 2026
66 of 67 checks passed
BenjaminDSmithy added a commit to BenjaminDSmithy/gstack that referenced this pull request Oct 8, 2026
Upstream moved to 1.91.42.0 (garrytan#3074, garrytan#3076, garrytan#3078), above this
branch's 1.91.35.0, and the release queue holds 1.91.43.0 to
1.91.47.0, so the release moves to 1.91.48.0 (PATCH, from
bin/gstack-next-version: bumped past claimed 1.91.47.0).

- CHANGELOG.md: main's file byte-identical from line 44 down, with
  this branch's entry on top, re-headed [1.91.48.0].
- VERSION, package.json and the agents digest: main's side, then
  bin/gstack-version-bump write --version 1.91.48.0 --regen-digest.

bin/gstack-hook-check on the merged tree: 9 ok, rc 0. Focused set
(the hook-check, doctor, session-update, autoplan, version and
digest files): 609 pass, 6 skip, 0 fail across 16 files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
levi85 pushed a commit to levi85/gstack that referenced this pull request Oct 14, 2026
…ion fits its budget, killed attempts leave no scratch (garrytan#3074)

* fix(cso): Python and Rails repairs verify and replay; killed attempts leave no scratch

- Python offline preparation sets SOURCE_DATE_EPOCH so .pyc bytecode is reproducible.
- Prepared identity hashes skip RubyGems/mini_portile build diagnostics that name random build dirs.
- Rails verification attempts and replays get 900 s; the attempt watchdog, Docker
  verification executor and assertion witness share MAX_VERIFICATION_ATTEMPT_MS.
- resume and finish reclaim a killed attempt's scratch once its watchdog recorded cleanup.

v1.91.34.0

* Format lib/cso with prettier

* Take v1.91.36.0 (1.91.35.0 is claimed by garrytan#3073)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant