Skip to content

Each job the Factory writes stops its own work under a step limit, so a job too long fails and is never cancelled (#388, record 0098) - #434

Merged
don-petry merged 9 commits into
mainfrom
claude/ci-388-timeout-caee96
Oct 31, 2026
Merged

don-petry merged 9 commits into
mainfrom
claude/ci-388-timeout-caee96

Conversation

@don-petry

@don-petry don-petry commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Closes #388. Record 0098, which builds on 0091. By the Factory Builder's numbering it queues after #429 (0097); until 0094 to 0097 land, the records' gap check fails here, as on the others.

Why. P-42 found that GitHub concludes a job it stops at its own timeout-minutes cancelled, not timed_out, its check run and run with it. A required check cancelled gives no verdict, so a Lens that runs too long landed on the Maintainer as check_unreadable. The trial on a step's own limit found it concludes failure, its job and check run with it, and a later if: always() step still runs: rule (a) of the ruling.

What changes

  • tools/check_workflows.py's timeout rule holds every workflow here. Each run: step sets its own timeout-minutes, a whole number above none, and a job's run: steps together leave the job MARGIN, two minutes, for its other steps. A uses: step needs no limit of its own. continue-on-error, which turns a step's timeout into success, was refused already.
  • The workflows, their word-for-word templates, the shape checks, the example, the standard's copy and the counter-examples carry the limits. 46 counter-examples were brought back to breaking one rule each; 3 new ones hold the new parts. ci.yml's hash is recorded again.
  • The limits come from what each step needs.
    • The margin: in 116 jobs of the required checks' recent runs, set-up, uses: and post steps took at most 8 s together.
    • make verify took up to 23 minutes against its job's 25, so that job goes to 28 and its step has 26. A first limit of 22 would have failed passing runs.
    • eval.yml and release.yml change only in their limits. The eval job keeps its 120 minutes and BUDGET; its steps have 1, 3, 3, 110 and 1. Its run step's 110 cannot cut a case, since its cases end within 110 minutes of the plan and the plan, fetch and install go first. release.move has 10 of 15.
    • The Engine's step has 20 of 30, above the longest tier's 900 s and its two reads of the CLI's version.
    • A test holds the eval run step to BUDGET and the Engine step to its longest tier.
  • The design. Section 4.2 and its table, and section 3.10, say what a timeout concludes, and P-18's row says which part P-42 answered.
  • The simulator's killed Run concludes failure, its outcome left or not, or cancelled with no outcome, never timed_out. Its rule is renamed a_run_past_its_limit_fails, asked by P-18 and unverified until P-18 has a fixture.
  • The Kernel is unchanged.

What this does not change: a Run past its limit is no attempt. apply still runs, and leaves environment, run_failed, for any run job that did not succeed, as it did before this. So the automatic pause and the Run ceiling bound it, not the attempts. By the Factory Builder's ruling (option A) the code stays, since classing a failed run job as item would count outages as attempts. #433 asks whether it should count, and what that needs from the run job's outputs.

Checks

Reviews

Structured (review-findings/1, parsed by review_parse.py), on 4793d94, before the Factory Builder's conditions and the fix round. AGY (Argon) read again on 9315ab2, since the rule's logic changed: a clean read of all 12 parts.

Reviewer Head Unread parts Blocking Major Minor Info
agy 4793d94 none 0 0 0 0
muse 4793d94 none 0 1 1 0
claude-review 4793d94 none 0 3 1 0
claude-security 4793d94 none 0 0 2 1
Finding Severity Criterion Where What Disposition
muse#1 major PR-1 tools/check_workflows.py:774 uses: steps are exempt from step limits and uncounted in the sum, so a job can still be cancelled at its job timeout fixed in part in 1a7c585: a two-minute margin for a job's other steps, by the Factory Builder's condition. Declined for the rest: SonarCloud, a job of uses: steps alone, is no required check the Kernel reads, so its cancelling reaches no decision
muse#2 minor PR-3 tools/check_workflows.py:779 A non-ASCII digit timeout-minutes crashes lint with ValueError instead of being rejected fixed in 5fdac10: a limit is MINUTES, a whole number above none in ASCII digits
claude-review#1 major PR-5 .github/workflows/eval.yml:50 The step limits the record sets by what each step needs are held by no test fixed in 5fdac10: a test ties the eval run step to BUDGET and the Engine step to its longest tier, with a mutant each
claude-review#2 major none docs/architecture/software-factory-design.md:923 A Run stopped at a step limit is read as environment, not as an attempt, as the design and the draft record say fixed in 9315ab2, by the Factory Builder's option A: the code stays, section 3.10, the record and the sim's comment say environment; follow-up #433
claude-review#3 major none tools/check_workflows.py:776 The timeout rule accepts timeout-minutes: 0 on a run: step, which sets no step limit at all fixed in 5fdac10, as muse#2, with a counter-example and a mutant
claude-review#4 minor none tools/check_workflows.py:779 Only run: steps count toward the margin, so a one-minute gap does not ensure a step's limit is reached before the job's fixed in 1a7c585, as muse#1; the design says the guarantee holds within the margin
claude-security#1 minor PR-3 tools/check_workflows.py:776 A run: step with timeout-minutes: 0 passes the new 'timeout' rule, but the runner treats 0 as no step limit, so the job can still be cancelled fixed in 5fdac10, as muse#2
claude-security#2 minor none tools/check_workflows.py:779 The rule budgets only run: steps and allows their sum to reach job-1, so setup, uses: steps and post steps can still carry a job past its limit to cancelled fixed in 1a7c585, as muse#1
claude-security#3 info none tools/check_workflows.py:779 A timeout-minutes value that isdigit() but is not a decimal digit (e.g. '²') crashes the workflow check with ValueError fixed in 5fdac10, as muse#2

🤖 Generated with Claude Code

don-petry and others added 5 commits October 29, 2026 22:43
… a job too long fails and is never cancelled (#388)

GitHub concludes a job it stops at its own timeout cancelled, its check run
with it (P-42); a step stopped at its own limit concludes failure, its job
with it (P-42.step-timeout). So check_workflows' timeout rule now holds each
run: step to a timeout-minutes of its own, and a job's run: steps to fewer
minutes together than the job. Every workflow, its templates, its shape
checks, the example, the standard's copy and the counter-examples carry the
limits; ci.yml's hash is recorded again. The design's 4.2 and 3.10 say what
a timeout concludes, and P-18's row which part P-42 answered. The sim's
killed Run concludes failure, or cancelled with no outcome, never timed_out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n for a job's other steps, the checks job given 28

The run: steps' limits together now leave their job MARGIN, two minutes,
for its set-up, uses: and post steps, which took at most 8 s in 116 jobs of
the required checks' runs. make verify took up to 23 minutes, so the checks
job has 28 and its step 26. The eval job keeps its 120 minutes and BUDGET;
its run step has 110, within which its cases end. The standard says the
margin, and a counter-example leaves a job less than it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…step's limit holds what its step needs

The Claude review and the security pass: timeout-minutes: 0 sets no limit, and a
digit int() reads but YAML does not crashed or passed the check; a limit is now
MINUTES, [1-9][0-9]*, with a counter-example. The Claude review: no test tied the
eval run step to BUDGET or the Engine step to the longest tier; one does, with a
mutant each. The design says the guarantee holds while a job's other steps keep
within the margin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…code has it, and the record is 0098

The Factory Builder's ruling, option A. The Claude review found that apply leaves
environment, run_failed, for any run job that did not succeed, so a Run past its
limit was never an attempt. The code stays. Section 3.10, the record and the sim's
comment say so, and issue 433 asks whether it should count, and what that needs
from the run job. The record is renumbered 0098.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry
don-petry requested a review from a team as a code owner October 30, 2026 18:23
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-31T01:41:58.186101Z
We couldn’t complete this review. Comment @codex review to try again.
927bd56 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@don-petry
don-petry added this pull request to the merge queue Oct 31, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 31, 2026
don-petry and others added 2 commits October 30, 2026 20:39
…ts limit

The merge group ejected this: four form counter-examples that #428 (Workflow
check: a list under on: is one only where its events begin with an item) landed
have run: steps with no limit, so each broke the timeout rule beside its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@sonarqubecloud

Copy link
Copy Markdown

@don-petry
don-petry added this pull request to the merge queue Oct 31, 2026
Merged via the queue into main with commit 7ffd0e3 Oct 31, 2026
19 checks passed
@don-petry
don-petry deleted the claude/ci-388-timeout-caee96 branch October 31, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A job stopped at its timeout concludes cancelled, so neither a required check nor a Run reads it as timed_out

1 participant