Skip to content

Trial P-65: a cancelled run's suite concludes cancelled, its judging check failure, and the rollup gives each check its suite's conclusion and its run (#681, step 1) - #744

Merged
don-petry merged 6 commits into
mainfrom
claude/trial-p65-681
Nov 12, 2026
Merged

don-petry merged 6 commits into
mainfrom
claude/trial-p65-681

Conversation

@don-petry

@don-petry don-petry commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

User description

Part of #681, "A cancelled run of a Lens is superseded by a later run on the same head": step 1 of its plan, ruled option (B) by the Factory Builder (trial, then port, then Kernel). The fixture alone, in its own PR, as ruled.

Trial P-65: verdict pass

The question and the pass condition were written at 14:02:59Z, before anything ran (trials/P-65.json pass). The trial ran on canary pull request 102, as the Maintainer. Its workflow p65-cancel.yml was granted nothing and checked nothing out: p65 / work sleeps 240 s, and p65 / verify needs it, runs under if: always(), and fails unless work succeeded.

What ran:

  1. R1 on head A, cancelled mid-way.
  2. B pushed, and its run cancelled.
  3. A pushed back; its run R3 was left alone.
  4. R1 re-run, and that attempt cancelled.
  5. The pull request closed and reopened, and its run R4 cancelled.
Point Result
(1) A cancelled run Its suite and run conclude cancelled; p65 / work cancelled, p65 / verify failure
(2) The push back to A and the reopening Each starts a workflow run with a suite of its own on A, beside R1's, which stays
(3) A re-run Keeps its run's id and its suite, with new check runs (run_attempt tells them apart); the rollup gives only the latest attempt's
(4) The run left alone success
(5) One rollup read Asking checkSuite { conclusion } and workflowRun { runNumber }, beside the run's databaseId the port reads already, costs 1, as the port's query does. P-8's measure stands

The section 6 row also names three limits the Kernel step (3) must rule on, from the Claude review:

  • A run's id, number and suite follow the order runs were created, and a re-run keeps all three, so none of them orders a re-run attempt against a later run.
  • A job stopped at its own timeout concludes cancelled too (P-42).
  • Each judging check concluded about a second before its suite, a gap the reads did not catch.

Every cancel was by hand; a concurrency group's was not tried.

Removed: the authorization header gh sent, from each of the 8 exchanges. The redaction of the eight captures found no credential: it removed only response headers a fixture may not keep. The bodies are trimmed to p65-cancel.yml's suites, runs and checks, as each call says, to keep the PR under the diff budget.

Step 8, undone in the canary:

  • Pull request 102 closed, and its branch trial/p65-cancel deleted.
  • No token, secret, variable, key, runner or webhook was made.
  • The runs stay: they hold only the two jobs' logs of a sleep and a test, and the fixture names them.

Grounding: no rule of sim/grounding.py names P-65 yet. The port's PR (#681 step 2) adds the fake's cancelled-run case and grounds it on this fixture.

Verify

make checks passes, trials included. Full verify is CI's.

Reviews

Read Head State
Claude + security d2cc189 Nothing blocking. Fixed in 72fd496: the three limits above in the row; P-8 named; P-65 in batch 4; the run on B said not read; attempt 1's suite not read on its own; the PR's time 14:03:39Z; the headers kept named. Plus a trim to stay under budget
Carbon (gemini-4-carbon-high, medium fallback in one part) d2cc189 One major (the row named the run's number, not its id): fixed in 1f92c50
Muse (Wonder Full) d2cc189 One minor (the run on B in no exchange): fixed in 72fd496 and 42f1b33
Astra (gpt-6-astra) 42f1b33 A clean read

🤖 Generated with Claude Code

View guided diff Turn on auto-fix


CodeAnt-AI Description

Document how cancelled workflow runs appear in pull request checks

What Changed

  • Records that cancelling a workflow run cancels its suite and work check, while its judging check fails.
  • Confirms that pushes back to a head and pull request reopenings create separate workflow runs and check suites; reruns keep the original run and suite but create new checks.
  • Documents that one rollup read can associate each check with its suite’s conclusion and workflow run at a cost of 1.

Impact

✅ Clearer cancelled-run check results
✅ Distinguishable checks for separate workflow runs
✅ One-query check attribution

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

don-petry and others added 4 commits November 11, 2026 08:17
…check failure, and the rollup gives each check its suite's conclusion and its run (#681)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st weigh (a re-run's order, a timeout's cancel, the second before a suite concludes), P-8's measure, batch 4, and a trim to stay under the budget

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… its number

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he reads hold

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry
don-petry requested a review from a team as a code owner November 11, 2026 14:34
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-11T14:34:16.278109Z
We couldn’t complete this review. Comment @codex review to try again.
42f1b33 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@codeant-ai

codeant-ai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

CodeAnt  CodeAnt Review Status

✅  Review completed

Status Commit Started (UTC) Finished (UTC)
✅  #1f156 Incremental review completed 14ff054 Nov 11, 2026 · 22:59 23:00
✅  #a0da8 Reviewed your PR 42f1b33 Nov 11, 2026 · 14:34 14:42

@codeant-ai

codeant-ai Bot commented Nov 11, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@codeant-ai codeant-ai Bot added the size:XL This PR changes 500-999 lines, ignoring generated files label Nov 11, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new trial, P-65, to the software factory design documentation and adds its corresponding trial log file trials/P-65.json. Trial P-65 verifies and documents the behavior of GitHub Actions check suites and workflow runs when a workflow run is cancelled mid-way, showing that the check suite and workflow run conclude as cancelled while any judging job using if: always() concludes as a failure. I have no feedback to provide on these changes.

@codeant-ai

codeant-ai Bot commented Nov 11, 2026

Copy link
Copy Markdown

CodeAnt  CodeAnt Nitpicks

1 code suggestion

1. The pass condition covers every cancelled run, but the fixture says B's run was cancelled and not read, so the verdict lacks evidence for that condition.

Logic error · trials/P-65.json:5

@codeant-ai

codeant-ai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

CodeAnt  CodeAnt PR Risk

🟢  Low Risk

  • The PR appears safe to merge; no actionable issues were identified in the review.

Assessed commit: 14ff054c1eaf

… read on its own

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@don-petry

Copy link
Copy Markdown
Contributor Author

Factory Builder: renewing this item's Run ceiling. Its ten Runs since a writer last acted all ended in the runner outage of 2026-11-11 (14:00Z to 22:24Z, before the v0.1.8 cut), as environment faults that reviewed nothing. #757 tracks counting such Runs out of the ceiling. A push is no writer's action, so the new head did not renew the count; this comment does.

@combsmith-sf-pr-review combsmith-sf-pr-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings: the change records trial P-65 and its fixture, and adds no code and no rule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants