Skip to content

[Phase 2] Promote standards/ci-standards.md to the formalized ring/soak/promotion model (+ ADR) #869

Closed

Description

@don-petry

Parent: #495 · Phase 2 · Work type: Docs/Standard · Targets SC3, SC8.

The org standard standards/ci-standards.md (in the public petry-projects/.github repo) currently documents the rings model aspirationally and carries a "Rollout status / next phase" caveat (≈ lines 125–129) stating that next/ringn channels and automated soak-gated promotion are not yet built. Once #499/#500/#501/#502/#868 land, that note is stale and the standard must be promoted to the formalized, machine-backed model.

This is the org-standard counterpart to the automation issues — separate because the standard lives in the public .github repo, not this one.

Scope

  1. Replace the "Rollout status / next phase" note and the abstract rings table with the formalized model: host-relative next, ring0 = .github+.github-private, ring1 = TalkTerm+bmad-bgreat-suite, stable = rest (mirrors standards/canary-rings.json, the machine-readable source of truth from [Phase 2] Define concentric rings (0 self-host → 1 low-traffic → 2 rest) + keep production duty on stable #500).
  2. Document the soak / health gate rules: min_healthy_runs = ceil(baseline_runs / 7) over a trailing 7-day window, failure-rate ≤ baseline, low-traffic "we still wait / no floor", 4-hourly evaluator cadence, and the gate states PROMOTE/SOAKING/INVESTIGATE/RESET (from [Phase 2] Replace PUT-contents clobber deploy with versioned, ring-staged, health-gated promotion #501).
  3. Document promotion = single gated channel-tag move (innermost-out) and rollback = move the tag back, with channel tags as the rollout state (from [Phase 2] Replace PUT-contents clobber deploy with versioned, ring-staged, health-gated promotion #501/[Phase 2] One-action rollback + per-version observability report #502).
  4. Update the stub pin guidance (table row + check_centralized_workflow_stubs expectations) so inner-ring repos pin to <name>/ring0|ring1, not just <name>/stable.
  5. Land an ADR (docs/initiatives/canary-rollout-adr.md) recording these decisions, mirroring the prior initiative-doc precedent; link it from the standard.

Done when: standards/ci-standards.md describes the implemented (not aspirational) ring/soak/promotion model, the "next phase" caveat is removed, stub-pin guidance + the audit's expected pins match the ring map, and an ADR is linked.

Decisions context: combsmith/.github#516.

Activity

  1. github-actions commented on Jun 21, 2026

    @github-actions
    Contributor

    Thanks for filing this and tying it back to the parent initiative — having the org standard reflect the implemented model (rather than the aspirational one) is exactly the kind of hygiene that keeps contributors from building on stale guidance. Before we kick off the edits to standards/ci-standards.md and draft the ADR, could you confirm the current landing status of the prerequisite issues (#499, #500, #501, #502, #868)? Knowing which of those have merged will clarify how much of the formalized model is ready to document as fact versus still in-flight.

  2. don-petry commented on Jul 1, 2026

    @don-petry
    CollaboratorAuthor

    Feed the #548 graduated soak gate (dwell 4/8/12h + traffic-scaled sample + robust baseline + per-candidate window + ring0 waiver + failure-triage) into this as the formalized ring/soak/promotion standard. Document at org level in petry-projects/.github/standards/ (ci-standards.md section + ADR); the config (registry.yml/control.yml) stays centralized in .github-private, not per-repo. Cross-links: .github#548 (gate), #993/#994 (impl), the new gate-hardening item.

  3. don-petry commented on Jul 1, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #869 — [Phase 2] Promote standards/ci-standards.md to the formalized ring/soak/promotion model (+ ADR)

    Scope note — cross-repo split

    Scope items 1–4 edit standards/ci-standards.md and its check_centralized_workflow_stubs
    audit expectations, which live in the public petry-projects/.github repo — not in this
    .github-private repo, and not reachable from a dev-lead run scoped here (the issue itself notes
    the standard "lives in the public .github repo, not this one"). The deliverable that lands in
    this repo is scope item 5: the ADR at docs/initiatives/canary-rollout-adr.md, recording
    the ring/soak/promotion decisions so the public-repo standard edit can link to it.

    Implementation Checklist

    • Create docs/initiatives/canary-rollout-adr.md — ADR recording:
      • The formalized ring map (host-relative next, ring0 = org-infra, ring1 = TalkTerm +
        bmad-bgreat-suite, stable = rest), mirroring standards/canary-rings.json as source of truth.
      • The soak/health gate: min_healthy_runs = ceil(baseline_runs / 7) over a trailing 7-day window,
        failure-rate ≤ baseline, low-traffic "we still wait / no floor", 4-hourly evaluate-only cadence,
        and gate states PROMOTE / SOAKING / INVESTIGATE / RESET.
      • Promotion = single gated channel-tag move (innermost-out); rollback = move the tag back; channel
        tags as the rollout state.
      • Stub-pin guidance: inner-ring repos pin <name>/ring0|ring1, not just <name>/stable.
      • Header/format mirroring the existing initiative-doc precedent
        (docs/initiatives/auto-rebase-vs-merge-queue.md, agentic-release-strategy.md).

    Tests to Write

    • None. This is a pure documentation deliverable (Phase 2 TDD exception). No in-repo test is
      tied to the ADR path, doc links, or ci-standards content — standards/canary-rings.json (already
      covered by tests/canary_rollout.bats) remains the machine-readable source of truth and is
      unchanged.

    Test Command

    bats tests/canary_rollout.bats (full suite) + bash .dev-lead/scripts/dev-lead-lint.sh

  4. don-petry commented on Jul 1, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • Confirmed the cross-repo split: scope items 1–4 (standards/ci-standards.md + its
      check_centralized_workflow_stubs expected-pin map) live in the public
      petry-projects/.github
      repo and are not reachable from a .github-private dev-lead run.
      Scope item 5 — the ADR — is the in-repo deliverable.
    • Created docs/initiatives/canary-rollout-adr.md (scope item 5), recording all four decision
      areas so the public-repo standard edit can link here and drop its "next phase" caveat:
      • Formalized host-relative ring map, mirroring standards/canary-rings.json as source of truth
        (next = host, ring0 = other org-infra repo, ring1 = TalkTerm + bmad-bgreat-suite,
        stable = rest).
      • Soak/health gate: min_healthy_runs = ceil(baseline_runs / 7) over a trailing 7-day window,
        failure-rate ≤ baseline (quality checked first), no synthetic floor / "we still wait",
        4-hourly evaluate-only cadence, and states PROMOTE/SOAKING/INVESTIGATE/RESET.
      • Promotion = single gated channel-tag move (innermost-out, git tag -f only); rollback =
        move the tag back to a prior immutable vX.Y.Z; channel tags are the rollout state; mover =
        GH_PAT_WORKFLOWS ([Phase 2] Extend release-channel-tags to ring channels + codify in apply-rulesets.sh #868).
      • Stub-pin guidance: inner-ring repos pin <agent>/ring0|ring1, not <agent>/stable, so soak
        actually observes the candidate; check_centralized_workflow_stubs map must match the ring map.

    Test Results

    Phase 2 (TDD) skipped per the pure-documentation exception — the deliverable is a Markdown ADR
    with no observable behavioral change. standards/canary-rings.json (the machine-readable source of
    truth) is unchanged, so its guard suite is logically unaffected; ran it to confirm no regression, plus
    the required repo lint:

    $ bats tests/canary_rollout.bats
    ok 1 ceil_div: exact division
    ... (17 more) ...
    ok 19 orchestrator: evaluate prints a per-ring gate report and exits 0 (read-only)
    ok 20 orchestrator: promote --override --dry-run shows the move but never pushes
    20 tests, 0 failures
    
    $ bash .dev-lead/scripts/dev-lead-lint.sh
      [lint] shellcheck on 83 file(s)...
      [lint] validating 4 agent profile(s)...  OK
      [lint] validating .github/CODEOWNERS...  OK
      [lint] all checks passed
    

    All 8 relative links in the ADR were verified to resolve to existing files.

    Files Changed

    • docs/initiatives/canary-rollout-adr.md (new): ADR recording the implemented ring/soak/promotion/
      rollback + stub-pin decisions, with header/format mirroring the existing initiative-doc precedent
      (auto-rebase-vs-merge-queue.md, agentic-release-strategy.md).

    Notes

    • Cross-repo boundary (not a limitation of this change). The issue's "Done when" spans two repos.
      The standard-file edits (remove the "Rollout status / next phase" caveat, formalize the rings table,
      soak/promotion rules, and the audit's expected pins, then link this ADR) must land in
      petry-projects/.github → standards/ci-standards.md, which a dev-lead run scoped to
      .github-private cannot modify. This ADR is that edit's in-repo counterpart and prerequisite; a
      follow-up PR against the public .github repo is required to close the standard-side half. The
      issue itself flags this ("separate because the standard lives in the public .github repo").
    • No standards/ci-standards.md or check_centralized_workflow_stubs file exists in this repo, so
      none was modified.
  5. added a commit that references this issue on Jul 1, 2026
  6. added 18 commits that reference this issue on Aug 2, 2026
  7. added a commit that references this issue on Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions