Skip to content

[Fleet Monitor] petry-projects/.github β€” .github/workflows/pr-review-mention-tests.ymlΒ #739

Closed

Description

@don-petry

🟑 WARNING: petry-projects/.github / .github/workflows/pr-review-mention-tests.yml

Metric Value
Failure Rate 14.3% (1 / 7 runs)
Status WARNING
p50 Duration 20s
p95 Duration 36s
Successful Runs 6
Cancelled Runs 0

Workflow: .github/workflows/pr-review-mention-tests.yml

Last updated by Fleet Monitor run on 2026-07-15.
Threshold: failure rate > 10% over the monitored window.

Activity

  1. don-petry commented on Jul 15, 2026

    @don-petry
    ContributorAuthor

    Dev-Lead Implementation Plan

    Issue: #739 β€” [Fleet Monitor] petry-projects/.github β€” .github/workflows/pr-review-mention-tests.yml

    Scope

    The workflow's flakiness (14.3% failure rate) comes from unretried curl downloads of a pinned mikefarah/yq from GitHub Releases in the "Install bats, shellcheck, jq, and yq" step. GitHub Releases redirects to objects.githubusercontent.com, which intermittently returns transient network/5xx errors. AGENTS.md (unit tests "MUST be … deterministic, and not access external networks") pins the intent: make the one unavoidable network fetch resilient. Fix: add bounded curl retry/backoff to the three yq-related downloads.

    Implementation Checklist

    • .github/workflows/pr-review-mention-tests.yml β€” add --retry/--retry-delay/--retry-connrefused/--retry-all-errors to the three curl calls (yq binary, checksums, checksums_hashes_order)

    Tests to Write

    • test/workflows/pr-review-mention/install-resilience.bats β€” assert every GitHub-Releases curl in the tests workflow carries retry flags (regression guard against a bare, flake-prone download)

    Test Command

    bats --print-output-on-failure test/workflows/pr-review-mention/

  2. don-petry commented on Jul 15, 2026

    @don-petry
    ContributorAuthor

    Dev-Lead: Implementation Complete

    Root Cause

    The suite's ~14% flake came from the "Install bats, shellcheck, jq, and yq" step fetching a pinned mikefarah/yq from GitHub Releases with a bare curl. GitHub Releases 302-redirects to objects.githubusercontent.com, which intermittently returns transient network/5xx errors; a bare curl aborts on the first blip and fails the job. AGENTS.md requires unit tests be deterministic, so the one unavoidable network fetch was made resilient rather than removed.

    Plan Execution

    • Added a regression test asserting every GitHub-Releases curl carries bounded retries (confirmed red before the fix)
    • Added --retry 5 --retry-delay 2 --retry-connrefused --retry-all-errors to all three yq-related curl calls (binary + two checksum files)
    • Full bats suite green; shellcheck, dev-lead-lint, and yamllint (max 200) all pass

    Test Results

    1..13
    ok 1 ack: a human commenter is @-mentioned in the acknowledgement
    ok 2 ack: the acknowledgement always carries the pr-review-agent marker
    ok 3 ack: never emits a literal @donpetry-bot when the bot is the actor (comment path)
    ok 4 ack: never emits a literal @donpetry-bot when the bot self-assigns as reviewer (pull_request path)
    ok 5 ack: a human reviewer-assigner is still @-mentioned (pull_request path)
    ok 6 guard: excludes comments authored by donpetry-bot itself
    ok 7 guard: excludes any comment carrying an <!-- pr-review-agent marker
    ok 8 guard: no longer keys the exclusion on the narrow mention-ack string alone
    ok 9 guard: still fires on an @donpetry-bot mention
    ok 10 guard: still gates comment events on trusted author_association
    ok 11 guard: still fires when donpetry-bot is assigned as reviewer
    ok 12 install: the tests workflow exists
    ok 13 install: every curl download uses bounded retries with backoff
    

    dev-lead-lint.sh: all checks passed. yamllint (line-length max 200): OK.

    Files Changed

    • .github/workflows/pr-review-mention-tests.yml: added bounded curl retries (--retry 5 --retry-delay 2 --retry-connrefused --retry-all-errors) to the three yq download calls, with a comment explaining the [Fleet Monitor] petry-projects/.github β€” .github/workflows/pr-review-mention-tests.ymlΒ #739 rationale.
    • test/workflows/pr-review-mention/install-resilience.bats (new): asserts every curl download line in the tests workflow carries retry/backoff flags, guarding against reintroducing a bare, flake-prone fetch.

    Notes

    • Retry budget is intentionally bounded (5 attempts, 2s base delay + curl's exponential backoff) so a genuinely-down mirror still fails fast well within the job's 10-minute timeout rather than hanging.
    • --retry-connrefused/--retry-all-errors are supported by the curl shipped on ubuntu-latest (8.x); no version bump needed.
    • apt-get was left unchanged: it is served from cached Azure mirrors and is not the observed flake source; adding retries there would be scope creep beyond this issue.
  3. added a commit that references this issue on Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions