Skip to content

[Phase 3] Add local canonical-AGENTS.md self-validation to CI (informational) #646

Closed

Description

@don-petry

Story

As a repo maintainer,
I want run the structural linter against this repo's own canonical AGENTS.md on pull requests,
so that structural regressions to the source-of-truth AGENTS.md are caught at PR time before they propagate downstream.

Acceptance Criteria

  1. lint.yml runs scripts/agents-md-lint.sh against the repo's AGENTS.md on pull_request in informational mode (annotates but does not fail the build in this phase).
  2. The check emits GitHub ::warning:: annotations for each structural finding.
  3. No new required status check is added while the check is informational; promotion to a blocking/required check is deferred to Phase 4.
  4. The change is limited to workflow wiring; shellcheck stays clean and the workflow remains valid YAML.

Tasks / Subtasks

Dev Notes

  • lint.yml already has shellcheck and bats jobs on pull_request; add a lightweight informational job/step that invokes the linter — do not make it a required check (that is Phase 4).
  • run-bats.sh demonstrates the ::warning:: annotation convention used in this repo for non-fatal signals; mirror it so findings surface without failing the build.
  • The linter's informational default mode (from Story 2) is what keeps this non-blocking; do not pass the failing-mode flag here.
  • This validates only THIS repo's AGENTS.md; org-wide validation is the audit integration (Story 3).

Project Structure Notes

Edit .github/workflows/lint.yml only. Any repo-specific-workflow exception should be noted per the AGENTS.md convention for documented repo-specific additions.

References

  • .github/workflows/lint.yml
  • scripts/run-bats.sh
  • AGENTS.md

Likely target surface

  • .github/workflows/lint.yml

Story prepared by the BMAD Scrum Master (Bob) for epic #642. Status: ready-for-dev.

Activity

  1. don-petry commented on Sep 1, 2026

    @don-petry
    ContributorAuthor

    Dev-Lead Implementation Plan

    Issue: #646 — [Phase 3] Add local canonical-AGENTS.md self-validation to CI (informational)

    Scope

    Wire the existing scripts/agents-md-lint.sh structural linter into CI so this repo's own canonical AGENTS.md is validated in informational mode on every pull request, surfacing each structural finding as a GitHub ::warning:: annotation without failing the build.

    Reconciliation note (repo reality vs. story references)

    The story references .github/workflows/lint.yml and scripts/run-bats.sh, neither of which exists in this repo. The actual equivalents are:

    • .github/workflows/ci.yml — the repo's canonical PR lint/test workflow (jobs: Lint, ShellCheck, Agent Security Scan, Secret scan); runs on every pull_request to main (not path-filtered), so it is the correct home for a self-check that must catch AGENTS.md regressions on any PR.
    • .dev-lead/scripts/run-bats.sh — the ::warning:: annotation convention referenced by the story.

    The pre-existing agents-md-rules-tests.yml is path-filtered to the linter's own files, so it would not run on a PR that only edits AGENTS.md — hence it is unsuitable and ci.yml is used instead.

    Implementation Checklist

    Tests to Write

    • test/workflows/ci/agents-md-selfcheck.bats — assert the job exists, targets AGENTS.md, runs the linter in informational mode (no --mode failing), and emits ::warning:: annotations (mirrors the existing secret-scan-config.bats static-wiring test pattern; auto-run by ci-tests.yml over test/workflows/ci/)

    Test Command

    bats --print-output-on-failure test/workflows/ci/ (plus shellcheck via .dev-lead/scripts/dev-lead-lint.sh and bash scripts/agents-md-lint.sh AGENTS.md sanity run)

  2. don-petry commented on Sep 1, 2026

    @don-petry
    ContributorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    Reconciliation note

    The story referenced .github/workflows/lint.yml and scripts/run-bats.sh, neither of which exists in this repo. The wiring was placed in the repo's actual canonical PR lint/test workflow .github/workflows/ci.yml (which already runs shellcheck + other lint jobs on every pull_request to main, so it catches AGENTS.md regressions on any PR). The pre-existing agents-md-rules-tests.yml is path-filtered to the linter's own files and would not fire on an AGENTS.md-only PR, so it was unsuitable. The ::warning:: convention is mirrored from .dev-lead/scripts/run-bats.sh.

    Test Results

    1..15
    ok 1 agents-md-selfcheck: the ci workflow exists
    ok 2 agents-md-selfcheck: the linter script it invokes exists and is executable
    ok 3 agents-md-selfcheck: ci runs the structural linter against this repo's AGENTS.md
    ok 4 agents-md-selfcheck: the self-check stays informational (no --mode failing)
    ok 5 agents-md-selfcheck: findings surface as ::warning:: annotations
    ok 6 agents-md-selfcheck: the self-check itself documents its informational, deferred-to-Phase-4 status
    ok 7 install: the ci workflow exists
    ok 8 install: every curl download in ci.yml uses bounded retries
    ok 9 install: the yamllint pip install carries a bounded retry budget
    ok 10 install: the AgentShield npx fetch retries transient registry errors
    ok 11 install: every job in ci.yml declares a bounded timeout-minutes
    ok 12 install: the gitleaks release download retries on transient failure
    ok 13 secret-scan: the ci workflow exists
    ok 14 secret-scan: gitleaks scan passes --config .gitleaks.toml
    ok 15 secret-scan: the .gitleaks.toml config ships at repo root
    

    Additional verification:

    • bash .dev-lead/scripts/dev-lead-lint.sh → all checks passed (shellcheck on 34 files + CODEOWNERS)
    • yamllint (CI config) on ci.yml → clean
    • actionlint 1.7.7 (runs shellcheck on the run: block) on ci.yml → clean
    • Runtime simulation under bash -eo pipefail: exits 0 against the clean repo AGENTS.md (no annotations) and against a fixture with a required-level finding (annotations emitted, still exit 0) — confirming the check is non-blocking.
    • tests/agents_md_lint.bats + tests/test_agents_md_rules_config.bats → 40/40 pass (unaffected).

    Files Changed

    • .github/workflows/ci.yml: new informational agents-md-selfcheck job (+ header documenting the Phase-4-deferred, non-required status).
    • test/workflows/ci/agents-md-selfcheck.bats: static-wiring regression test (job present, targets AGENTS.md, informational — no --mode failing, emits ::warning::, documents Phase 4 deferral).

    Notes

    • The job creates a new non-required status check named "AGENTS.md Self-Check (informational)"; it must not be added to branch protection until Phase 4. This is documented in the ci.yml job header.
    • If AGENTS.md were ever deleted, the linter returns exit 2 and the (non-required) job would go red — intentional fail-loud behavior that cannot block merge.
  3. added a commit that references this issue on Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions