Skip to content

canary-rollout evaluate mis-reports all 6 cross-repo (#482) reusables as 'fully rolled out' β€” reads local tags, not .githubΒ #1049

Closed

Description

@don-petry

πŸ”΄ P1 β€” the gate cannot see cross-repo agents' channel tags. With the registry now populated (#1044) + all agents dispatchable (#1047), I ran evaluate for the 4 ring1 reusables. All four returned:

ring0 ->        (empty)
ring1 ->        (empty)
stable ->       (empty)
frontier: none β€” fully rolled out (all rings on candidate).

This is WRONG. Live channel state on petry-projects/.github:

agent ring1 stable
auto-rebase b7e6b670 acec8959 (old baseline)
dependabot-automerge b7e6b670 376a4fcb (old)
dependabot-rebase b7e6b670 376a4fcb (old)
pr-review-mention 93bfca45 376a4fcb (old)

All four are at ring1 with stable still on the OLD baseline β€” i.e. their real frontier is ring1 (ring1β†’stable pending), and they are READY to promote. The evaluate reports "fully rolled out" and would skip them.

Root cause

evaluate resolves channel/release tags from the local git checkout (this repo, .github-private) β€” the run log shows it only fetched dev-lead/* and pr-review/* tags (the two .github-private-hosted agents). The 6 #482 reusables are cross-repo (host: petry-projects/.github); their <name>/<channel> tags live on petry-projects/.github, not in this checkout β†’ they resolve empty β†’ "all rings equal (empty) β†’ fully rolled out."

Fix

For an agent whose host β‰  this repo (cross-repo), resolve its channel + release tag SHAs against host via gh api repos/<host>/git/refs/tags/<name>/<ch> (dereference annotated tags), exactly as cut-release.sh --promote does β€” do NOT read them from the local checkout. dev-lead/pr-review (host = .github-private) keep the local path. Add a bats case: a cross-repo agent with ring1=candidate, stable=old must report frontier=ring1, not "fully rolled out."

Until fixed, the automation cannot evaluate OR promote any of the 6 #482 reusables (it thinks they're done). This is now the top blocker to finishing the rollout.

Refs #1044 (registry), #1045/#1047 (dispatch), #1002/#1026, #548, #959 (cross-repo cut precedent).

Activity

  1. don-petry commented on Jul 3, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #1049 β€” canary-rollout evaluate mis-reports all 6 cross-repo (#482) reusables as 'fully rolled out' β€” reads local tags, not .github

    Scope

    Make canary-rollout.sh resolve channel + release tag SHAs against an agent's host repo (via gh api, dereferencing annotated tags) when the host is not this repo, so cross-repo (#482) reusables' real frontier is computed instead of collapsing to "fully rolled out". Agents hosted here (dev-lead/pr-review) keep the local git path.

    Root cause

    channel_commit and candidate_cut_date read <name>/<channel> and <name>/vX.Y.Z tags from the local checkout. Cross-repo agents' tags live on petry-projects/.github, so they resolve empty β†’ all rings equal (empty) β†’ "fully rolled out."

    Implementation Checklist

    • scripts/canary-rollout.sh: add THIS_REPO constant + _gh_tag_commit (gh-api channel/tag resolver, dereferences annotated tags, mirrors cut-release.sh)
    • scripts/canary-rollout.sh: make channel_commit <agent> <channel> host-aware β€” cross-repo β†’ _gh_tag_commit host <agent>/<channel>; else local git
    • scripts/canary-rollout.sh: add _gh_candidate_cut_date and make candidate_cut_date host-aware β€” cross-repo β†’ resolve <agent>/v* release tags on host, deref annotated, match candidate commit, return tagger date

    Tests to Write

    • tests/canary_rollout.bats: cross-repo agent (host=petry-projects/.github) with next/ring0/ring1=candidate, stable=old β€” evaluate must NOT report "fully rolled out"; must show the ring1->stable transition (frontier reached ring1, ready to promote). Stubs return tag SHAs only via gh api (git tag lookups return empty), proving the cross-repo path is used.

    Test Command

    bats tests/canary_rollout.bats

  2. don-petry commented on Jul 3, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • Added THIS_REPO constant + _gh_tag_commit (gh-api channel/tag resolver, dereferences annotated tags, mirrors cut-release.sh's gh_release_commit)
    • Made channel_commit <agent> <channel> host-aware β€” cross-repo agents resolve <name>/<channel> on host via the GitHub API; dev-lead/pr-review (host = this repo) keep the local git path
    • Added _gh_candidate_cut_date and made candidate_cut_date host-aware β€” cross-repo agents resolve their <name>/vX.Y.Z release tag (and its tagger date) on host
    • Added a bats regression case: a cross-repo agent with next/ring0/ring1=candidate, stable=old must report the ring1->stable pending transition, NOT "fully rolled out"

    Root cause (confirmed)

    channel_commit and candidate_cut_date read <name>/<channel> and <name>/vX.Y.Z tags from the local checkout. The six #482 reusables are cross-repo (host: petry-projects/.github); their tags live there, so they resolved empty β†’ all rings equal (empty) β†’ "fully rolled out". Fix routes cross-repo tag resolution through gh api repos/<host>/git/ref/tags/... (dereferencing annotated tags), exactly as cut-release.sh --promote does.

    Test Results

    1..52
    ...
    ok 49 orchestrator: a non-allowlisted failure (reusable unchanged) still BLOCKS as PRE_EXISTING
    ok 50 orchestrator: promote --allow-pre-existing advances a BLOCKED+PRE_EXISTING frontier (dry-run)
    ok 51 orchestrator: promote --allow-pre-existing REFUSES a BLOCKED+REGRESSION frontier
    ok 52 orchestrator: cross-repo agent resolves channel+release tags on host β†’ ring1->stable pending, NOT 'fully rolled out' (#1049)
    

    New test #52 was confirmed RED against the unfixed code (reported "fully rolled out") and GREEN after the fix. Full suite: 52/52 pass. .dev-lead/scripts/dev-lead-lint.sh (shellcheck + agent frontmatter): all checks passed.

    Files Changed

    • scripts/canary-rollout.sh: THIS_REPO constant; new _gh_tag_commit / _gh_candidate_cut_date gh-api resolvers; channel_commit and candidate_cut_date now branch to the host (gh-api) path for cross-repo agents and keep the local-git path for agents hosted in this repo.
    • tests/canary_rollout.bats: _crossrepo_stub helper + a regression test proving a cross-repo agent (auto-rebase, host petry-projects/.github) with stable on an old baseline reports ring1->stable (READY to promote), not "fully rolled out". The stub returns tag SHAs only via gh api; local git returns empty, so the test fails if the code regresses to the local path.

    Notes

    • Fail-safe behavior preserved: on a gh error the resolvers return empty, matching the existing local-path degradation semantics; evaluate remains read-only.
    • Scope limited to tag resolution as the issue specifies β€” the pure gate core (scripts/lib/canary-rollout.sh) and the ring registry are unchanged.
  3. added a commit that references this issue on Jul 3, 2026
  4. added a commit that references this issue on Jul 3, 2026
  5. added 8 commits that reference this issue on Aug 2, 2026
  6. 16 remaining items

  7. added 12 commits that reference this issue on Aug 7, 2026
  8. added 2 commits that reference this issue on Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupenhancementFeature requests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions