Skip to content

Live idea:approved canary + Fleet Monitor coverage #822

Closed

Description

@don-petry

Story

As a org owner,
I want add a post-merge canary on the idea:approved → dispatch path and extend Fleet Monitor coverage to the new stub,
so that a silent revert like #655 is caught on the first approval rather than when a human notices nothing planned.

Acceptance Criteria

  1. A scheduled smoke check asserts the stub dispatches the central planner (a dry-run dispatch against a fixture Discussion, or a synthetic idea:approved add/remove) and alerts on a failure or the 'Unsupported event type: discussion' error.
  2. The Fleet Monitor health-check ([Fleet Monitor] petry-projects/.github-private — .github/workflows/initiative-planner.yml #627) is extended to the new standards/workflows/initiative-planner.yml stub across enrolled repos.
  3. An alert fires if any enrolled repo's stub drifts from the canonical template SHA.

Tasks / Subtasks

Dev Notes

Project Structure Notes

Monitoring lives in .github-private alongside the planner; reuse the existing fleet-monitor/health-check tooling.

References

Likely target surface

  • .github/workflows/initiative-planner.yml
  • scripts/initiative-planner/

Story prepared by the BMAD Scrum Master (Bob) for epic #817. Status: ready-for-dev.

Activity

  1. github-actions commented on Jun 20, 2026

    @github-actions
    Contributor

    Thanks for the detailed story and acceptance criteria — this is well-scoped and easy to pick up! Before we start implementation: for the scheduled canary (AC #1), should the dry-run dispatch target a fixture Discussion in this repo, or would you prefer a fully synthetic event (e.g., mocking the discussion[labeled] webhook payload) so no real Discussion object is needed at all?

  2. don-petry commented on Jun 20, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #822 — Live idea:approved canary + Fleet Monitor coverage

    Scope

    Add a scheduled dry-run canary for the idea:approved → planner-dispatch path, and extend the Actions Fleet Monitor to track the per-repo initiative-planner.yml stub and alert on drift from the canonical template SHA.

    Implementation Checklist

    • AC#1 scripts/initiative_canary.sh — dry-run dispatch of initiative-planner.yml against a fixture Discussion, poll the resulting run, classify success / failure / Unsupported event type: discussion. Pure helpers + main() (mirrors auto_rebase_health.sh).
    • AC#1 .github/workflows/initiative-planner-canary.yml — scheduled (daily) + workflow_dispatch canary; alerts (find-or-create issue) on a non-OK result.
    • AC#2/Add @claude delegation, auto-merge, and rebase handling #3 scripts/fleet_stub_drift.sh — sourced lib of pure functions: classify a repo stub blob SHA vs. the canonical template SHA (ALIGNED / DRIFTED / MISSING), build report + alert JSON.
    • AC#2/Add @claude delegation, auto-merge, and rebase handling #3 Extend scripts/fleet_monitor.sh — new "Initiative-planner stub coverage & drift" section reusing the discovered repo list; fetch canonical + per-repo stub SHAs, emit fleet_stub_drift.json, set HAS_STUB_DRIFT.
    • AC#3 Extend .github/workflows/actions-fleet-monitor.yml — alert step gated on env.HAS_STUB_DRIFT == 'true'.
    • Register the two new test files in .github/workflows/lint.yml (per AGENTS.md bats-list rule).

    Tests to Write

    • tests/test_initiative_canary.bats — classify_canary_run (OK / FAILED / UNSUPPORTED_EVENT / NO_RUN), alert-decision helper, alert-body render, and a gh-mocked main asserting the dry-run dispatch (dry_run=true).
    • tests/fleet_stub_drift.bats — classify_stub_drift, row builder, report generator, alert-JSON, and count helpers over a fixture TSV.

    Test Command

    scripts/run-bats.sh tests/test_initiative_canary.bats tests/fleet_stub_drift.bats (full suite via lint.yml bats job); lint via bash .dev-lead/scripts/dev-lead-lint.sh

  3. don-petry commented on Jun 20, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • AC#1 — scripts/initiative_canary.sh + .github/workflows/initiative-planner-canary.yml: daily (06:30 UTC) + workflow_dispatch canary that fires a dry-run dispatch of initiative-planner.yml against a fixture Discussion, polls the resulting run, and alerts (find-or-create tracking issue + wakes dev-lead) on a failure or the Unsupported event type: discussion regression fingerprint. Dry-run ⇒ creates nothing.
    • AC#2 — Extended the Actions Fleet Monitor (scripts/fleet_monitor.sh §2a) to track each enrolled repo's initiative-planner.yml stub, reusing the already-discovered repo list (extends [Fleet Monitor] petry-projects/.github-private — .github/workflows/initiative-planner.yml #627 rather than building a parallel monitor). Pure classification logic in the new sourced lib scripts/fleet_stub_drift.sh.
    • AC#3 — Drift detection: per-repo stub blob SHA compared against the canonical org-template SHA (ALIGNED/DRIFTED/MISSING). Drifted enrolled repos are exported to fleet_stub_drift.json; actions-fleet-monitor.yml opens/updates a fleet-stub-drift tracking issue and fires repository_dispatch to wake dev-lead when HAS_STUB_DRIFT.
    • Registered both new test files in lint.yml; added an AGENTS.md "Workflow Files" exception entry for the repo-specific canary workflow.

    Test Results

    # New tests (TDD — written before implementation):
    tests/test_initiative_canary.bats ... 14 ok
    tests/fleet_stub_drift.bats ........ 13 ok
    
    # Full lint.yml bats suite:
    631 tests, 0 failures
    
    # Lint:
    bash .dev-lead/scripts/dev-lead-lint.sh  → all checks passed (shellcheck 69 files + agent profiles)
    shellcheck --severity=warning -x scripts/{fleet_monitor,fleet_stub_drift,initiative_canary}.sh → clean
    YAML parse (initiative-planner-canary.yml, actions-fleet-monitor.yml, lint.yml) → OK
    

    Files Changed

    • scripts/initiative_canary.sh (new): canary — pure classify_canary_run/canary_is_failure/canary_report + main() (dispatch+poll). Mirrors auto_rebase_health.sh layout.
    • .github/workflows/initiative-planner-canary.yml (new): scheduled canary workflow with failure alerting.
    • scripts/fleet_stub_drift.sh (new): pure stub-coverage/drift lib (classify, row, count, alert-JSON, report).
    • scripts/fleet_monitor.sh: §2a stub-SHA scan over discovered repos; appends a "stub coverage & drift" report section; exports fleet_stub_drift.json + HAS_STUB_DRIFT/STUB_DRIFT_COUNT.
    • .github/workflows/actions-fleet-monitor.yml: "Track initiative-planner stub drift" + "Dispatch dev-lead for stub drift" steps; tightened the clean-run annotation.
    • .github/workflows/lint.yml: registered the two new bats files.
    • tests/test_initiative_canary.bats, tests/fleet_stub_drift.bats (new): unit tests (gh mocked; no network).
    • AGENTS.md: documented the repo-specific canary workflow exception.

    Notes

    • Per Live idea:approved canary + Fleet Monitor coverage #822 Dev Notes, the canary uses a dry-run dispatch (dry_run=true) so apply-plan only logs intended mutations to an artifact — no epic/issues are created.
    • The canary requires GH_PAT_WORKFLOWS (a workflow_dispatch fired with GITHUB_TOKEN is accepted but never starts a run) and reads the fixture Discussion from repo variable INITIATIVE_CANARY_DISCUSSION; an unset fixture skips with a ::warning:: rather than failing.
    • "Enrolled" = a repo that has the .github/workflows/initiative-planner.yml stub. MISSING (no stub) is reported as informational/not-enrolled and is not alerted; only DRIFTED enrolled repos trigger the drift alert (AC#3).
    • Stub-drift compares GitHub contents-API blob SHAs (git blob object IDs), so equal SHAs ⇒ byte-identical stubs. Canonical source defaults to petry-projects/.github standards/workflows/initiative-planner.yml (overridable via CANONICAL_STUB_REPO/CANONICAL_STUB_PATH).
    • Per repo constraints, no commits/PRs were created — the CI workflow handles git operations.
  4. added a commit that references this issue on Jun 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupenhancementFeature requestsinitiativeEpic / initiative tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions