Skip to content

[Fleet Monitor] petry-projects/.github-private β€” issue-triage-runner.ymlΒ #316

Closed

Description

@github-actions

🟠 DEGRADED: petry-projects/.github-private / issue-triage-runner.yml

Metric Value
Failure Rate 25% (2 / 8 runs)
Status DEGRADED
p50 Duration 196s
p95 Duration 212s
Successful Runs 6
Cancelled Runs 0

Workflow: issue-triage-runner.yml

Last updated by Fleet Monitor run on 2026-06-11.
Threshold: failure rate > 10% over the monitored window.

Activity

  1. added
    dev-leadFor dev-lead agent pickup
    and removed
    dev-leadFor dev-lead agent pickup
    on Jun 9, 2026
  2. don-petry commented on Jun 9, 2026

    @don-petry
    Collaborator
    All engines are currently rate-limited. Please re-apply the `dev-lead` label when the rate limit clears to retry.
  3. added
    dev-leadFor dev-lead agent pickup
    and removed
    dev-leadFor dev-lead agent pickup
    on Jun 9, 2026
  4. don-petry commented on Jun 9, 2026

    @don-petry
    Collaborator

    Dev-Lead Implementation Plan

    Issue: #316 β€” [Fleet Monitor] petry-projects/.github-private β€” issue-triage-runner.yml

    Root Cause Analysis

    Two distinct failure modes were found across 4/12 failed runs:

    Bug 1 β€” Prompt/guard contradiction (3 failures: runs 27168683186, 27168684938, 27168687742)

    • issue-triage.md instructs Claude to return {"skip": true} when an issue has 2+ labels.
    • aw.sh already handles this as a pre-Claude label-count guard before calling the model.
    • aw.sh then rejects any {"skip": true} from the model as "possible prompt injection" and exits 1.
    • Net effect: Claude follows the prompt instruction, the script treats it as an attack, the run fails.

    Bug 2 β€” Non-existent label (1 failure: run 27172272227)

    • issue-triage.md allows and instructs Claude to use the needs-triage label.
    • needs-triage does not exist in the repo (repo has needs-human-review).
    • gh issue edit --add-label "needs-triage" fails with 'needs-triage' not found.

    Scope

    Fix .github/workflows/issue-triage.md to align the prompt with aw.sh's runtime contract:

    1. Remove the skip instruction (the pre-Claude guard in aw.sh already handles 2+ labels).
    2. Replace needs-triage β†’ needs-human-review (the label that actually exists).
    3. Replace good-first-issue β†’ good first issue (latent label-name mismatch, same failure mode as bug 2).

    Implementation Checklist

    • tests/aw/issue-triage/test_aw_run.sh β€” add regression tests (TDD, write first)
    • .github/workflows/issue-triage.md β€” remove skip instruction; fix label names

    Tests to Write

    • issue-triage.md allowed list does NOT contain needs-triage
    • issue-triage.md allowed list DOES contain needs-human-review
    • issue-triage.md allowed list does NOT contain good-first-issue
    • issue-triage.md prompt body does NOT contain the skip-for-labels instruction
    • safe-output apply accepts needs-human-review as a valid label

    Test Command

    bash tests/aw/issue-triage/test_aw_run.sh && bash tests/aw/issue-triage/test_runner_yml.sh

  5. don-petry commented on Jun 9, 2026

    @don-petry
    Collaborator

    Dev-Lead: Implementation Complete

    Root Causes Found

    Bug 1 β€” Prompt/guard contradiction (3/4 failures)
    issue-triage.md instructed Claude to return {"skip": true} for issues with 2+ labels. However, aw.sh already enforces this as a pre-Claude label-count guard before calling the model β€” and then explicitly rejects any skip flag returned by the model as "possible prompt injection", failing the run. Internal issues with the initiative label (1 label, below the threshold) still triggered Claude to return skip, likely because the prompt's skip instruction created ambiguity in the model's decision.

    Bug 2 β€” Non-existent label (1/4 failures)
    issue-triage.md included needs-triage in the allowed label set and instructed Claude to use it. The label needs-triage does not exist in the repo; gh issue edit --add-label fails with 'needs-triage' not found. Also fixed a latent version of the same bug: good-first-issue (hyphenated) was listed, but the repo label is good first issue (with space).

    Plan Execution

    • Post implementation plan as issue comment
    • Write 5 failing regression tests (TDD, confirmed red before implementation)
    • Remove skip instruction from issue-triage.md prompt
    • Replace needs-triage β†’ needs-human-review in allowed list and prompt
    • Replace good-first-issue β†’ good first issue in allowed list and prompt
    • Update test 6 and scenarios.md to use corrected label names
    • Run full test suite β€” all pass

    Test Results

    PASS  scenario-4: 2 existing labels β†’ skip=true
    PASS  safe-output: 4 labels exceeds max-labels:3 β†’ rejected
    PASS  safe-output: disallowed label β†’ rejected
    PASS  safe-output: skip flag β†’ no writes
    PASS  safe-output: empty comment β†’ rejected
    PASS  safe-output: valid allowed labels β†’ validation passed
    PASS  issue-triage.md: allowed list does not contain 'needs-triage'
    PASS  issue-triage.md: allowed list contains 'needs-human-review'
    PASS  issue-triage.md: allowed list does not contain 'good-first-issue'
    PASS  issue-triage.md: prompt does not contain skip instruction
    PASS  safe-output: 'needs-human-review' is a valid allowed label
    
    Results: 11 passed, 0 failed
    
    (runner yml: 7 passed, 0 failed β€” dev-lead-lint: all checks passed)
    

    Files Changed

    • .github/workflows/issue-triage.md: removed skip instruction; replaced needs-triage β†’ needs-human-review and good-first-issue β†’ good first issue throughout
    • tests/aw/issue-triage/test_aw_run.sh: updated test 6 to use needs-human-review; added tests 7–11 as regression guards
    • tests/aw/issue-triage/scenarios.md: updated label names to match

    Notes

    The skip instruction removal is safe: aw.sh's pre-Claude guard (skip-if-labels-ge: 2 default) handles skip for over-labelled issues before Claude is ever called. The apply job is already gated by if: needs.triage.outputs.skip != 'true', so the skip flow continues to work correctly.

  6. added 5 commits that reference this issue on Jun 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupfleet-trackerTracked by the Actions Fleet Monitorhealth-checkAutomated health check report

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions