Skip to content

The simulator gives each check its run and suite, and starts another run of a Lens beside the one it leaves (#749) - #800

Merged
don-petry merged 2 commits into
mainfrom
claude/sim-cancelled-749
Nov 13, 2026
Merged

don-petry merged 2 commits into
mainfrom
claude/sim-cancelled-749

Conversation

@don-petry

@don-petry don-petry commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #749

The simulator now reaches #681 step 3's rule (record 0203, #783). That rule passes over a check from a run whose suite concluded cancelled or timed_out, beside another run of the same Lens on the head that did not stop. This change adds no new rule and no record, by the Factory Builder's ruling.

Protected paths: sim/world.py and tests/test_simulator.py, pre-approved by the Maintainer 2026-11-08, ruled by the Factory Builder 2026-11-11.

What

Run ids and suites. Each check carries run_id and suite_conclusion, as adapters/checks.py writes them:

  • A Lens that a workflow runs reports from a workflow run. Its suite is null while pending and concludes as its check does.
  • A Lens no workflow runs, CodeQL's, has run_id null and its App's suite.
  • Each push starts new runs. A person's re-run keeps its run's id and suite, as P-65 recorded.
  • A check a job creates names the Lens's own run. Another workflow's check has a run of its own; another App's has none.

The fault (another_run). On one seed in four (seed % 4 == 3), another run of a workflow Lens starts on the head. The causes are the three P-65 recorded:

  • a push back to the head, a new run;
  • a reopening, a new run;
  • a re-run of an earlier run there, which keeps that run's id and stands in the rollup for its earlier attempt.

The run it leaves stays beside it. If it was in progress, it is cancelled: suite cancelled, if: always() judging check failure. Otherwise it stays as it concluded. The fault prefers a Lens whose run just failed. A person's re-run result is that person's (the ruling on #38).

Grounding. sim/grounding.py adds a_check_names_its_run_and_suite and a_cancelled_run_stays_beside_a_later_run, both on trials/P-65.json. The port reads each of them from that fixture (tests/test_adapters.py).

Invariants, restated in the world's own words:

  • No verdict: a stopped run beside a run of the same check that did not stop is no check of the head.
  • I-27 and the queue: I-27 (no model review before the checks pass) and the queue's objections and merge gate read each Lens through passed() / failed(). A stopped run is passed over beside one that did not stop, and a failure an earlier run left counts.
  • New: a stopped run never lets a Lens pass that a completed failure fails. A failing check of a Lens's own App and file, whose suite concluded and did not stop, lets no review start, nothing be approved or enqueued, and nothing converge.
  • settle(): the Maintainer pushes past an earlier run's check that did not pass and did not stop, as for another App's check.

Held by

  • The seed test: tests/test_simulator.py::test_the_seeds_start_another_run_of_a_lens_on_the_head. It holds:
    • all three causes are reached, on seed % 4 == 3 alone, and a re-run keeps its run's id;
    • a run left cancelled never passes;
    • the Kernel's own lens_reading passes a Lens with a cancelled run beside a passing one;
    • a completed failure beside a passing run is never approved or enqueued.
  • The invariants: test_invariants_hold_under_faults, on the fixed seeds.
  • Mutants: all 67 of sim/world.py and sim/grounding.py, with the Kernel mutants this test names, are killed. Among them:
    • "another run of a Lens never starts" and "a cancelled run's judging check passes" (the world's);
    • "every run is passed over beside a run that did not stop" and "the simulator's stopped run is never passed over" (the Kernel's), each killed by the simulator's tests alone;
    • "the latest run alone is read".
  • Re-anchored: three older mutants, for the facts() line and the queue's merge gate.
  • Runs: the simulator suite passes on 300 seeds (CI's count), and the tests the mutants name pass on 40 (the mutants' count). make checks passes.

Seed review

I compared every seed's trace and transitions over 250 steps, before and after: 76 of 300 seeds change.

Seeds Why they change
56 (all on seed % 4 == 3) Another run of a Lens starts on the head. From that step on the world differs: the run left beside the new one, the Lens pending again, and what follows
18: 2, 21, 24, 26, 36, 58, 65, 102, 141, 165, 170, 216, 219, 220, 254, 255, 268, 279 A check concluded timed_out, its suite timed_out. Alone it is now every run stopped, so unreadable (verifying check_unreadable -> defer), where it used to fail (addressing check_failed -> admit), under ruling (b)
2: 16, 202 The same timed-out check, with no pass between it and a person's act. The trace is unchanged, but the State that act moves from is verifying, not addressing, so the transitions it makes differ

None is unexplained.

Not modelled

  • A push back to the head or a reopening is modelled as its run alone. The world adds no timeline event, and the item does not leave the queue. The facts it gives are still ones GitHub can give, for example a run another event starts. The Claude review named this.
  • A failing check whose suite has not yet concluded (concluding) is never served. In P-65 a judging check concluded about a second before its suite; the world concludes both at once.

Second readers

Reader Head Result
Carbon (Gemini 4 Carbon) 7183765 no findings
Muse 7183765 1 major, fixed in a66ba2b: the queue's objections and merge gate ignored earlier runs. 1 minor, fixed: created and posted checks named no run
Astra (gpt-6-astra) 7183765 2 majors, both fixed in a66ba2b: the test now asserts a re-run keeps its id, and that only seed % 4 == 3 gets the fault; sim/AGENTS.md names the test
Claude + security 7183765 no blocker. 2 majors, fixed in a66ba2b: the test now reads the Kernel's own lens_reading (it could pass vacuously), and the queue reads earlier runs. Minors: a re-run's result is the person's, which is fixed; the two not modelled above are named

The fixes in a66ba2b came after the readers; none of them re-read that head.

🤖 Generated with Claude Code

View guided diff Turn on auto-fix

don-petry and others added 2 commits November 12, 2026 20:27
…run of a Lens beside the one it leaves (#749)

Each Lens a workflow runs reports from a workflow run, its suite concluding as its check does; on one seed in four another run starts on the head, by a push back to it, a reopening or a re-run of an earlier run, and the run it leaves stays beside it, cancelled with its judging check failed if it was in progress (trials/P-65.json). The world's invariants read a stopped run passed over beside one that did not stop, and a new one holds that a stopped run never lets a Lens pass that a completed failure fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Kernel's own Lens, a re-run keeps its id and is the person's, and every check names its run

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry
don-petry requested a review from a team as a code owner November 13, 2026 02:57
@coderabbitai

coderabbitai Bot commented Nov 13, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-13T02:57:21.484379Z
We couldn’t complete this review. Comment @codex review to try again.
a66ba2b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@codeant-ai

codeant-ai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

CodeAnt  CodeAnt Review Status

✅  Review completed

Status Commit Started (UTC) Finished (UTC)
✅  #169fa Reviewed your PR a66ba2b Nov 13, 2026 · 02:57 03:05

@codeant-ai

codeant-ai Bot commented Nov 13, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@sonarqubecloud

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements simulation and validation logic for tracking multiple workflow runs of a Lens on a commit head, addressing issue 749 and trial P-65. It introduces tracking for workflow run IDs and suite conclusions, handles cancelled or stopped runs correctly when a new run is started (via push backs, reopenings, or re-runs), and updates the state evaluation logic to ensure stopped runs do not incorrectly cause a Lens to pass or fail. Corresponding tests have been added to verify these behaviors in both the adapter and simulator tests. I have no feedback to provide as there are no review comments.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="sim/grounding.py">

<violation number="1" location="sim/grounding.py:93">
P3: This comment cites issue 749 for trials/P-65.json, but the design doc, schema, Kernel and port cite issue 681 for the same trial. Cite one issue number for the trial's provenance so the grounding can be traced.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread sim/grounding.py Resolved
@codeant-ai

codeant-ai Bot commented Nov 13, 2026

Copy link
Copy Markdown

CodeAnt  CodeAnt Nitpicks

2 code suggestions

1. create() removes same-name checks from every run, so a later run deletes the earlier run's check even though only checks in the same suite are replaced.

Incomplete implementation · sim/world.py:1443-1444


2. Accepting None here lets this test pass when a job-created check loses its workflow run ID, despite the simulator contract requiring that ID.

Incorrect condition logic · tests/test_simulator.py:908

@codeant-ai

codeant-ai Bot commented Nov 13, 2026

Copy link
Copy Markdown

CodeAnt  CodeAnt PR Risk

🟢  Low Risk

  • The PR appears safe to merge; no actionable issues were identified in the review.

Assessed commit: a66ba2b1a4a5

@combsmith-sf-pr-review combsmith-sf-pr-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings against the rubric.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Simulator: a cancelled run of a Lens beside a later run on the same head, with run ids on its checks

2 participants