Skip to content

[Phase 3] Add an initiative-driver canary over the cross-repo release path #885

Closed

Description

@github-actions

Story

As a fleet maintainer,
I want have a post-merge canary that periodically dry-run-dispatches the central driver against a fixture target repo/epic and alerts on failure,
so that a silent break of the cross-repo release path surfaces immediately instead of when someone notices nothing got released.

Acceptance Criteria

  1. New .github/workflows/initiative-driver-canary.yml + scripts/initiative_driver_canary.sh mirror initiative-planner-canary.yml / scripts/initiative_canary.sh: a scheduled (off-peak, ≤1/day) + workflow_dispatch job fires a DRY-RUN dispatch (-f dry_run=true) of initiative-driver.yml with target_repo set to a fixture — creating/labeling nothing.
  2. The canary classifies the dispatched run and, on failure, finds-or-creates a single tracking issue (labels initiative-canary / dev-lead / health-check) and posts the report; a clean run annotates a notice.
  3. The fixture is a CONFIGURED target repo + epic (repo variables INITIATIVE_DRIVER_CANARY_TARGET / INITIATIVE_DRIVER_CANARY_EPIC, with workflow_dispatch overrides) where the epic carries initiative:auto and has ≥1 OPEN sub-issue whose blockers are all closed — so the dry-run actually resolves the cross-repo gate + blocked_by; the canary no-ops with a ::warning:: when unconfigured.
  4. The canary treats the run as FAILED (and alerts) not only when the dispatched run's conclusion != success, but also when — for a CONFIGURED fixture — the run log shows no READY (dry-run, not labeling) decision for the target repo (the cross-repo release path resolved to nothing: a hollow green).
  5. Logic (run classification incl. the no-READY hollow-green guard + dispatch-arg assembly) is unit-tested in tests/test_initiative_driver_canary.bats, mirroring tests/test_initiative_canary.bats.
  6. AGENTS.md records the canary as a documented repo-specific workflow exempt from template syncs (mirror the existing initiative-planner-canary.yml exception entry).

Tasks / Subtasks

Dev Notes

  • Mirror scripts/initiative_canary.sh closely: it already fires gh workflow run "$WORKFLOW_FILE" --repo "$REPO" -f target_repo="$REPO" -f dry_run=true, polls gh run list, and classifies the conclusion. For the driver, target the fixture and pass -f epic=<fixture-epic> and -f dry_run=true.
  • The driver has NO 'Unsupported event type: discussion' fingerprint (it is pure-bash, no claude-code-action). Use driver-relevant signals: (a) run conclusion != success, and (b) the HOLLOW-GREEN guard — for a configured fixture, the run log MUST contain the driver's dry-run decision line #<n> — READY (dry-run, not labeling). for the target; its absence means the cross-repo gate/blocked_by resolved to nothing and is a FAILURE. (That exact string is emitted by scripts/initiative-driver.sh in the DRY_RUN branch.)
  • Dry-run is mandatory for zero cost: with dry_run=true the driver logs the READY decision and applies no dev-lead label, so no dev-lead/LLM run is triggered.
  • The fixture epic must be armed (initiative:auto) with ≥1 ready sub-issue (open, blockers closed) so a dry-run produces a READY decision — otherwise the canary cannot distinguish 'path healthy' from 'path broken'. Document this in the fixture setup.
  • PAT required: the dispatch needs GH_PAT_WORKFLOWS (a workflow_dispatch with GITHUB_TOKEN never starts a run) — same constraint as the planner canary.
  • Cron must be offset off the top of the hour per the off-peak scheduling standard (Reliable scheduled Actions: off-peak cron offsets, scheduling standard, and schedule-reliability reporting #722/[Phase 1] Codify the off-peak scheduling standard + CI compliance check #724); the planner canary runs 06:30 just after Fleet Monitor (06:00) — pick a nearby non-colliding offset.

Project Structure Notes

All in .github-private — dev-lead-authorable. Register the new test in lint.yml's repo-specific bats list (AGENTS.md notes that list is extended, not reset, by template syncs).

References

  • .github/workflows/initiative-planner-canary.yml
  • scripts/initiative_canary.sh
  • tests/test_initiative_canary.bats
  • AGENTS.md#workflow-files (exception entries)
  • scripts/initiative-driver.sh#DRY_RUN

Likely target surface

  • new .github/workflows/initiative-driver-canary.yml
  • new scripts/initiative_driver_canary.sh
  • new tests/test_initiative_driver_canary.bats
  • AGENTS.md
  • .github/workflows/lint.yml

Story prepared by the BMAD Scrum Master (Bob) for epic #882. Status: ready-for-dev.

Activity

  1. don-petry commented on Jun 21, 2026

    @don-petry
    Collaborator

    Dev-Lead Implementation Plan

    Issue: #885 — [Phase 3] Add an initiative-driver canary over the cross-repo release path

    Scope

    Add a post-merge canary that dry-run-dispatches initiative-driver.yml against a configured fixture target repo + epic and alerts (find-or-create tracking issue) when the run fails OR when a configured fixture produces no READY (dry-run, not labeling) decision (a hollow-green cross-repo break).

    Implementation Checklist

    • scripts/initiative_driver_canary.sh — pure helpers (classify_canary_run w/ hollow-green guard, canary_is_failure, canary_report) + main() dry-run dispatch/poll, mirroring scripts/initiative_canary.sh
    • .github/workflows/initiative-driver-canary.yml — off-peak cron + workflow_dispatch; guard (target+epic configured) → run → alert → annotate; fixture via INITIATIVE_DRIVER_CANARY_TARGET / INITIATIVE_DRIVER_CANARY_EPIC repo vars w/ dispatch overrides; GH_PAT_WORKFLOWS
    • AGENTS.md — add exception entry mirroring the initiative-planner-canary.yml entry
    • .github/workflows/lint.yml — register tests/test_initiative_driver_canary.bats in the bats list

    Tests to Write

    • tests/test_initiative_driver_canary.bats — classify (success+READY→OK, success-no-READY→HOLLOW_GREEN, failure/timeout→FAILED, empty/null→NO_RUN), alert decision, report content, and main() with mocked gh (dry-run dispatch arg assembly, hollow-green failure, unconfigured no-op, env-var fallback)

    Test Command

    scripts/run-bats.sh tests/test_initiative_driver_canary.bats and bash .dev-lead/scripts/dev-lead-lint.sh

  2. don-petry commented on Jun 21, 2026

    @don-petry
    Collaborator

    Dev-Lead: Implementation Complete

    Plan Execution

    • scripts/initiative_driver_canary.sh — pure helpers (classify_canary_run with the hollow-green guard, canary_is_failure, canary_report) + main() dry-run dispatch/poll, mirroring scripts/initiative_canary.sh
    • .github/workflows/initiative-driver-canary.yml — off-peak cron (40 6 * * *, just after the planner canary's 06:30) + workflow_dispatch; guard → run → alert → annotate; fixture via INITIATIVE_DRIVER_CANARY_TARGET / INITIATIVE_DRIVER_CANARY_EPIC repo vars with dispatch overrides; GH_PAT_WORKFLOWS
    • AGENTS.md — added the template-sync exception entry mirroring the initiative-planner-canary.yml entry
    • .github/workflows/lint.yml — registered tests/test_initiative_driver_canary.bats in the bats list
    • tests/test_initiative_driver_canary.bats — 17 tests (TDD: written red first, then green)

    Acceptance Criteria Mapping

    • AC1 dry-run dispatch of initiative-driver.yml with -f target_repo=<fixture> -f epic=<fixture> -f dry_run=true — creates/labels nothing.
    • AC2 classifies the run; on failure find-or-creates a single tracking issue (initiative-canary / dev-lead / health-check) titled 🔴 Initiative-driver canary failing and posts the report; clean run emits a ::notice::.
    • AC3 fixture configured via repo variables with target_repo/epic dispatch overrides; the guard step no-ops with a ::warning:: when unconfigured.
    • AC4 treats the run as FAILED when conclusion != success and via the HOLLOW-GREEN guard — a green run whose log lacks READY (dry-run, not labeling) (cross-repo gate/blocked_by resolved to nothing).
    • AC5 classification (incl. hollow-green), dispatch-arg assembly, and the unconfigured no-op are unit-tested with a mocked gh.
    • AC6 AGENTS.md exception entry added; bats file registered in lint.yml.

    Test Results

    $ bats tests/test_initiative_driver_canary.bats
    1..17
    ok 1 classify_canary_run: success with a READY decision is OK
    ok 2 classify_canary_run: success WITHOUT a READY decision is HOLLOW_GREEN
    ok 3 classify_canary_run: a failing conclusion is FAILED (even with a READY line)
    ok 4 classify_canary_run: timed_out / cancelled are FAILED
    ok 5 classify_canary_run: an empty conclusion (no run found) is NO_RUN
    ok 6 classify_canary_run: a null conclusion (still running) is NO_RUN
    ok 7 canary_is_failure: OK does not alert
    ok 8 canary_is_failure: every non-OK status alerts
    ok 9 canary_report: OK report names the path, target and epic
    ok 10 canary_report: the hollow-green status surfaces the no-READY hint
    ok 11 main: dispatches the driver in DRY-RUN against the fixture and exits OK
    ok 12 main: a configured run with no READY decision is a HOLLOW_GREEN failure
    ok 13 main: a failing driver run sets CANARY_FAILED and exits non-zero
    ok 14 main: a non-numeric CANARY_EPIC is rejected before any dispatch
    ok 15 main: a malformed CANARY_TARGET is rejected before any dispatch
    ok 16 main: INITIATIVE_DRIVER_CANARY_* are used when CANARY_* are unset
    ok 17 main: exits with error when the fixture target/epic are unset
    
    Full bats suite (run-bats.sh, all repo test files): 672 tests, all passing.
    dev-lead-lint.sh: all checks passed (shellcheck on 73 files + agent profiles).
    actionlint .github/workflows/initiative-driver-canary.yml: exit 0.
    

    Files Changed

    • scripts/initiative_driver_canary.sh (new): canary logic — dry-run dispatch of the driver against the fixture + the hollow-green classifier.
    • .github/workflows/initiative-driver-canary.yml (new): scheduled/dispatch canary workflow (guard → run → alert → annotate).
    • tests/test_initiative_driver_canary.bats (new): 17 unit tests over the pure helpers + mocked main().
    • AGENTS.md: template-sync exception entry for the new workflow.
    • .github/workflows/lint.yml: registered the new bats file.

    Notes

    • Input validation rejects a malformed target_repo (must be owner/repo) and a non-numeric epic before anything reaches the gh command line (injection guard), mirroring the planner canary's discussion-number check.
    • A green run whose logs cannot be fetched defaults to HOLLOW_GREEN (alert) — conservative by design: better to alert than to silently miss a broken path.
    • Operational follow-up (out of scope, no code change): set the repo variables INITIATIVE_DRIVER_CANARY_TARGET / INITIATIVE_DRIVER_CANARY_EPIC to a fixture epic armed with initiative:auto and ≥1 ready open sub-issue; until then the canary no-ops with a ::warning::.
  3. added a commit that references this issue on Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupinitiativeEpic / initiative tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions