You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
P2 (#860 follow-up): runaway-PR detection in daily health check (commits/comments/cycles/age thresholds) #948
Part of the #860 runaway remediation (tracker: #926). The per-PR budget (#928) and the org self-trigger fix (combsmith/.github#538) prevent/bound runaways. This issue adds the detection net so the next novel runaway surface is caught on day one instead of running unseen — #860 ran for four days with no human noticing.
Problem
There is no proactive signal for a PR accumulating abnormal automated activity. Detection is pull (someone happens to look), not push.
Fix
Extend daily-pr-review-health / scripts/fleet_monitor.sh (and/or scripts/pr_review_health.sh) to flag any open PR exceeding soft thresholds as a runaway candidate, with a direct link and the triggering metric:
> 50 commits, or
> 200 comments, or
> N automated cycles (reuse compute_pr_automation_cycles from scripts/lib/pr-automation-budget.sh), or
open > 48h with active agent churn.
Thresholds env-overridable. Output via the existing health-report mechanism (issue/label health-check + automated-report, as in #916's report). Detection only — it should not itself mutate PRs.
Acceptance criteria
The daily health check lists any open PR over the thresholds, with link + metric.
Thresholds are env-overridable with sane defaults.
No false positive on a normal, converging PR (validate against current open PRs).
Thanks for the detailed write-up and for connecting this back to the #860 incident — the context around detection vs. prevention is really helpful for scoping the work. Before we dig in: for the automated-cycles threshold specifically, do you have a target value for N in mind (or a formula based on PR age/size), or should the implementer propose a default based on what would have flagged #860 on day one?
Issue:#948 — P2 (#860 follow-up): runaway-PR detection in daily health check (commits/comments/cycles/age thresholds)
Scope
Add a detection-only runaway-PR net to the daily health check: scan open PRs and flag any exceeding soft thresholds (commits / comments / automated cycles / age-with-churn) as a runaway candidate, surfaced via the existing health-check + automated-report issue mechanism. No PR mutation.
Design
Pure threshold logic in a new sourced lib (scripts/lib/pr-runaway-detect.sh), reusing compute_pr_automation_cycles from scripts/lib/pr-automation-budget.sh for the cycles metric — the unit under test.
gh-API gathering + report rendering in scripts/pr_runaway_scan.sh (no LLM; mirrors the pure-fn / gh-jq idiom of fleet_monitor.sh).
Thresholds env-overridable with sane defaults: RUNAWAY_MAX_COMMITS=50, RUNAWAY_MAX_COMMENTS=200, RUNAWAY_MAX_CYCLES=10, RUNAWAY_MIN_AGE_HOURS=48, RUNAWAY_CHURN_MIN_CYCLES=1 (age criterion only fires with active agent churn, so a normal/quiet aged PR does not false-positive).
scripts/pr_runaway_scan.sh — gh-API + jq scan of every open PR, detection only (no PR mutation); writes pr_runaway_report.md, sets HAS_RUNAWAY/RUNAWAY_COUNT
.github/workflows/daily-pr-review-health.yml — runs the scan and opens a health-check+automated-report issue when candidates are found
.github/workflows/lint.yml — registered the new bats file
ok 18 RUNAWAY_MIN_AGE_HOURS override is respected
ok 19 RUNAWAY_CHURN_MIN_CYCLES override gates the age criterion
ok 20 non-numeric inputs degrade to 0 (no crash, no false fire)
ok 21 missing arguments degrade to 0
ok 22 pr_age_hours computes whole hours against an injected now
ok 23 pr_age_hours degrades to 0 on unparseable input
ok 24 generate_runaway_report renders each candidate with its link and reason
ok 25 generate_runaway_report on an empty file prints an all-clear line, not a table
Full lint.yml bats list (1018 tests) passes; bash .dev-lead/scripts/dev-lead-lint.sh passes (shellcheck 87 files + agent frontmatter + CODEOWNERS); both modified workflows are valid YAML.
Files Changed
scripts/lib/pr-runaway-detect.sh (new): pure threshold/detection logic + report renderer
Activity
github-actions commentedon Jun 26, 2026
Thanks for the detailed write-up and for connecting this back to the #860 incident — the context around detection vs. prevention is really helpful for scoping the work. Before we dig in: for the automated-cycles threshold specifically, do you have a target value for N in mind (or a formula based on PR age/size), or should the implementer propose a default based on what would have flagged #860 on day one?
don-petry commentedon Jul 3, 2026
Dev-Lead Implementation Plan
Issue: #948 — P2 (#860 follow-up): runaway-PR detection in daily health check (commits/comments/cycles/age thresholds)
Scope
Add a detection-only runaway-PR net to the daily health check: scan open PRs and flag any exceeding soft thresholds (commits / comments / automated cycles / age-with-churn) as a runaway candidate, surfaced via the existing
health-check+automated-reportissue mechanism. No PR mutation.Design
scripts/lib/pr-runaway-detect.sh), reusingcompute_pr_automation_cyclesfromscripts/lib/pr-automation-budget.shfor the cycles metric — the unit under test.scripts/pr_runaway_scan.sh(no LLM; mirrors the pure-fn / gh-jq idiom offleet_monitor.sh).RUNAWAY_MAX_COMMITS=50,RUNAWAY_MAX_COMMENTS=200,RUNAWAY_MAX_CYCLES=10,RUNAWAY_MIN_AGE_HOURS=48,RUNAWAY_CHURN_MIN_CYCLES=1(age criterion only fires with active agent churn, so a normal/quiet aged PR does not false-positive).Implementation Checklist
scripts/lib/pr-runaway-detect.sh— thresholds +pr_runaway_reasons+is_pr_runaway+generate_runaway_reportscripts/pr_runaway_scan.sh— gather open-PR metrics, run detection, writepr_runaway_report.md, setHAS_RUNAWAY(detection only).github/workflows/daily-pr-review-health.yml— run scan + createhealth-check/automated-reportissue when candidates found.github/workflows/lint.yml— add new bats file to thebatsjob listTests to Write
tests/test_pr_runaway_detect.bats— each threshold fires at boundary (>50,>200,>N,>48h+churn)Test Command
scripts/run-bats.sh tests/test_pr_runaway_detect.bats(full suite vialint.ymlbats job); lint viabash .dev-lead/scripts/dev-lead-lint.shdon-petry commentedon Jul 3, 2026
Dev-Lead: Implementation Complete
Plan Execution
scripts/lib/pr-runaway-detect.sh— pure, env-overridable threshold logic (pr_runaway_reasons,is_pr_runaway,pr_age_hours,generate_runaway_report); reusescompute_pr_automation_cyclesfrom the Runaway PR #860: agents self-triggering — add per-PR churn breaker, human-gated cap resets, and self-trigger guards #926 budget lib for the cycles metricscripts/pr_runaway_scan.sh— gh-API + jq scan of every open PR, detection only (no PR mutation); writespr_runaway_report.md, setsHAS_RUNAWAY/RUNAWAY_COUNT.github/workflows/daily-pr-review-health.yml— runs the scan and opens ahealth-check+automated-reportissue when candidates are found.github/workflows/lint.yml— registered the new bats filetests/test_pr_runaway_detect.bats— 25 unit tests (threshold boundaries, env overrides, false-positive guard, multi-metric, report rendering)Acceptance Criteria
generate_runaway_report)RUNAWAY_MAX_COMMITS=50,RUNAWAY_MAX_COMMENTS=200,RUNAWAY_MAX_CYCLES=10,RUNAWAY_MIN_AGE_HOURS=48,RUNAWAY_CHURN_MIN_CYCLES=1Test Results
Full
lint.ymlbats list (1018 tests) passes;bash .dev-lead/scripts/dev-lead-lint.shpasses (shellcheck 87 files + agent frontmatter + CODEOWNERS); both modified workflows are valid YAML.Files Changed
scripts/lib/pr-runaway-detect.sh(new): pure threshold/detection logic + report rendererscripts/pr_runaway_scan.sh(new): open-PR scan orchestration (detection only)tests/test_pr_runaway_detect.bats(new): 25 unit tests.github/workflows/daily-pr-review-health.yml: scan + runaway-issue steps.github/workflows/lint.yml: registered the new test fileNotes
compute_pr_automation_cycles(Runaway PR #860: agents self-triggering — add per-PR churn breaker, human-gated cap resets, and self-trigger guards #926) so the detection net and the circuit breaker agree on what an automated cycle is.RUNAWAY_CHURN_MIN_CYCLES(default 1) so a benign PR left open past 48h with no agent activity is not flagged.daily-pr-review-healthworkflow.feat: implement issue #948 — P2 (#860 follow-up): runaway-PR detectio…