You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Test the budget poller's artifact trust filter: only default-branch artifacts may be believed (blocks the retry-cron consumer) #2148
Follow-up to #2029 (PR 2031, the dry-run budget poller). bp_download_latest_log in scripts/lib/budget-poller.sh decides which uploaded budget-poller-log artifact the poller, and soon the retry cron (#2139), will believe. It has no test at all (tests/dev-lead/unit/test_budget_poller.bats never calls it and never mocks gh api). That function is a trust boundary: if it ever selects an artifact produced by a pull-request branch run, a PR author could feed false usage numbers to the fleet. Review threads on PR 2031 asked for these tests twice; they were not delivered, so they are tracked here.
What the function must keep doing (read the merged code and confirm; do not trust this summary)
Select only artifacts whose workflow_run.head_branch equals the repo's default branch, not expired, newest first, across all pages (--paginate), because other branches' runs can fill page one and hide an older trusted log.
Allow-list the default-branch name's characters, falling back to main on anything else.
Return 2 when the listing call fails mid-pagination; return 1 when no artifact is usable; fall through to an older artifact when a newer one is unreadable or not valid JSON.
Scope
Add tests only, in tests/dev-lead/unit/test_budget_poller.bats or a new file next to it, calling bp_download_latest_log directly with gh and unzip replaced by a PATH shim. No live network. Do not change budget-poller.sh unless a test exposes a real bug; if one does, fix it minimally and say so.
Acceptance criteria
A newer artifact from a non-default branch (for example a PR branch) is ignored in favour of an older default-branch one. This is the security test and must fail if the head_branch filter is removed (prove it by temporarily removing the filter locally and showing the test fails; say so in the PR).
A single page with one matching artifact is selected.
The newest trusted artifact is on page 2 while page 1 holds only other branches' artifacts: it is still found.
A listing failure on a later page returns 2 and does not abort the caller.
No usable artifact returns 1; an expired artifact is skipped.
The newest artifact is unreadable or not valid JSON: the next older valid one is used.
A default-branch name with unsafe characters falls back to main and is never interpolated into the jq filter raw.
Shellcheck clean; existing tests pass unchanged; real counts quoted from the pushed head.
Blocks
#2139 (the retry cron that will act on this artifact) must not start until this has merged.
Thanks for filing this follow-up and for spelling out the trust-boundary reasoning so clearly. Since the artifact filter guards against PR-branch runs feeding false usage numbers, I'd like to confirm scope: should the tests also cover the case where the repo's default branch is renamed mid-run (for example, the API reports a different default branch than the artifacts' head_branch), or are the seven listed acceptance criteria the complete set?
Issue:#2148 — Test the budget poller's artifact trust filter: only default-branch artifacts may be believed
Scope
Add bats tests that call bp_download_latest_log directly with gh and unzip replaced by a PATH shim (no network). The gh shim evaluates the function's real --jq filter against per-page fixture JSON, so removing the head_branch filter is observable. No change to scripts/lib/budget-poller.sh unless a test exposes a real bug.
Implementation Checklist
tests/dev-lead/unit/test_budget_poller.bats: add a gh/unzip PATH-shim helper and a new bp_download_latest_log section (file is already in the lint.yml bats list)
Tests to Write
AC1: newer PR-branch artifact ignored in favour of older default-branch one (and proven to fail with the filter removed)
AC2: single page, one matching artifact selected
AC3: newest trusted artifact on page 2, page 1 only other branches → found
AC4: listing fails on page 2 → returns 2, dest untouched, set -e caller continues
AC5: no usable artifact → 1; expired artifact skipped
AC6: newest unreadable / not valid JSON → next older valid one used
AC7: unsafe default-branch name → falls back to main, never interpolated raw into the jq filter
Dev-Lead: Progress (in progress — not yet durable)
Provisional. Nothing is pushed yet; the automation posts the authoritative
completion record (with PR number + head SHA) after the push. Treat the notes
below as intent, not delivered results.
Plan Execution (intended)
Added a bp_download_latest_log section to tests/dev-lead/unit/test_budget_poller.bats (already in the lint.yml bats list). gh/unzip are PATH shims; the gh shim runs the function's own --jq filter over per-page fixture listings and serves page 2+ only under --paginate.
scripts/lib/budget-poller.sh is unchanged — no test exposed a bug.
Mutation check (local workspace, provisional)
Each mutation was applied to budget-poller.sh locally, the new tests run, and the file restored:
Mutation
New tests that fail
remove head_branch filter
AC1 security test + AC2 (non-main default), AC3, AC5, both AC7
drop --paginate
AC2, AC3, AC4
remove branch allow-list
both AC7
drop .expired == false
both AC5
|| return 2 → || true
AC4
drop jq -e validity check
AC6
stop after first unreadable artifact
AC6
Files touched (this workspace, not yet pushed)
tests/dev-lead/unit/test_budget_poller.bats: tests only
Notes
Real pass counts will be quoted from the pushed head in the completion record.
The implementation for issue #2148 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.
Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.
Activity
github-actions commentedon Oct 9, 2026
Thanks for filing this follow-up and for spelling out the trust-boundary reasoning so clearly. Since the artifact filter guards against PR-branch runs feeding false usage numbers, I'd like to confirm scope: should the tests also cover the case where the repo's default branch is renamed mid-run (for example, the API reports a different default branch than the artifacts' head_branch), or are the seven listed acceptance criteria the complete set?
don-petry commentedon Oct 9, 2026
Dev-Lead Implementation Plan
Issue: #2148 — Test the budget poller's artifact trust filter: only default-branch artifacts may be believed
Scope
Add bats tests that call
bp_download_latest_logdirectly withghandunzipreplaced by a PATH shim (no network). Theghshim evaluates the function's real--jqfilter against per-page fixture JSON, so removing thehead_branchfilter is observable. No change toscripts/lib/budget-poller.shunless a test exposes a real bug.Implementation Checklist
tests/dev-lead/unit/test_budget_poller.bats: add agh/unzipPATH-shim helper and a newbp_download_latest_logsection (file is already in thelint.ymlbats list)Tests to Write
set -ecaller continuesmain, never interpolated raw into the jq filterTest Command
bats tests/dev-lead/unit/test_budget_poller.bats+bash .dev-lead/scripts/dev-lead-lint.shdon-petry commentedon Oct 9, 2026
Dev-Lead: Progress (in progress — not yet durable)
Plan Execution (intended)
bp_download_latest_logsection totests/dev-lead/unit/test_budget_poller.bats(already in thelint.ymlbats list).gh/unzipare PATH shims; theghshim runs the function's own--jqfilter over per-page fixture listings and serves page 2+ only under--paginate.scripts/lib/budget-poller.shis unchanged — no test exposed a bug.Mutation check (local workspace, provisional)
Each mutation was applied to
budget-poller.shlocally, the new tests run, and the file restored:head_branchfilter--paginate.expired == false|| return 2→|| truejq -evalidity checkFiles touched (this workspace, not yet pushed)
tests/dev-lead/unit/test_budget_poller.bats: tests onlyNotes
Real pass counts will be quoted from the pushed head in the completion record.
don-petry commentedon Oct 9, 2026
Dev-Lead: Implementation Complete — PR #2155
The implementation for issue #2148 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.
9f50d34d9a790919f1f7929929b82fc94b87b4b7main: 1 file(s), +245/-0 linesReview happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.
feat: implement issue #2148 — Test the budget poller's artifact trust…