Skip to content

feat: implement issue #507 — [Phase 2a] Encode the PR-limit config as a machine-readable single source of truth - #562

Merged
don-petry merged 3 commits into
mainfrom
dev-lead/issue-507-20260630-2153
Jun 30, 2026
Merged

don-petry merged 3 commits into
mainfrom
dev-lead/issue-507-20260630-2153

Conversation

@don-petry

@don-petry don-petry commented Jun 30, 2026 •

Copy link
Copy Markdown
Contributor

Closes #507

Implemented by dev-lead agent. Please review.

Summary by CodeRabbit

  • New Features
    • Added a shared configuration for pull-request limit rules, including overall caps, per-source limits, and exemption settings.
    • Introduced automated checks to validate that the configuration stays well-formed and complete.
  • Chores
    • Added a GitHub Actions workflow to run the PR-limits validation tests on relevant changes and on updates to the main branch.

… a machine-readable single source of truth
@don-petry
don-petry requested a review from a team as a code owner June 30, 2026 21:58
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jun 30, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@don-petry, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 49 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5954c768-de84-44bc-ba0d-ed0ac32a9249

📥 Commits

Reviewing files that changed from the base of the PR and between dbbf499 and f573aa1.

📒 Files selected for processing (2)
  • .github/workflows/pr-limits-tests.yml
  • test/scripts/pr-limits/pr-limits-config.bats
📝 Walkthrough

Walkthrough

This PR adds standards/pr-limits.json, a machine-readable configuration defining org-wide and per-source automation PR caps, exempt actors, and exempt labels with a provisional status. It adds a Bats test suite validating the config's schema and values, and a GitHub Actions workflow running these tests on relevant path changes.

Changes

PR-limits config and CI gate

Layer / File(s) Summary
PR-limits configuration data
standards/pr-limits.json
Defines provisional org-wide automation PR cap, per-source caps for dev-lead/claude/initiative-driver, exempt actors and labels, schema version, and explanatory notes.
Bats validation suite for config contract
test/scripts/pr-limits/pr-limits-config.bats
Validates file existence, JSON parseability, provisional status, non-empty notes, positive-integer caps (org-wide and per-source), required top-level keys, and exempt_actors contents including dependabot[bot].
CI workflow wiring
.github/workflows/pr-limits-tests.yml
Adds a path-filtered workflow triggered on pull_request/push to main, with read-only permissions, per-ref concurrency cancellation, dependency installation (bats, shellcheck, jq), and Bats test execution.

Sequence Diagram(s)

sequenceDiagram
  participant Developer
  participant Workflow as PR-limits Tests workflow
  participant Bats
  participant Config as pr-limits.json

  Developer->>Workflow: push/pull_request touching pr-limits files
  Workflow->>Bats: run pr-limits-config.bats
  Bats->>Config: parse and validate JSON contract
  Config-->>Bats: status, caps, exempt_actors
  Bats-->>Workflow: pass/fail result
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Possibly related issues

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly names the PR-limit config and matches the main change.
Linked Issues check ✅ Passed The config, test, and workflow cover the requested single-source JSON, provisional caps, exemptions, and PR-scoped validation.
Out of Scope Changes check ✅ Passed The changes stay focused on the PR-limit config, its test, and the dedicated workflow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev-lead/issue-507-20260630-2153

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #562
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-06-30T22:29:10Z

@don-petry

Copy link
Copy Markdown
Contributor Author

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-06-30T22:29:10Z

@don-petry
don-petry enabled auto-merge (squash) June 30, 2026 21:59

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new configuration file standards/pr-limits.json defining provisional pull request limits and exemptions, along with a BATS test suite to validate its structure and values. The feedback suggests expanding the test coverage to ensure that all top-level keys, including exempt_labels and _schema_version, are validated, and that the security label and schema version are explicitly tested.

@don-petry
don-petry disabled auto-merge June 30, 2026 21:59
@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
**Review State:**
The `gemini-code-assist[bot]` review is marked `COMMENTED` (not `CHANGES_REQUESTED`), offering suggestions to expand test coverage but not blocking.
---
## Summary
**Bot:** SonarCloud  
**Issues addressed:** 0  
**Tier 1 blockers:** 0  
**Status:** No action required
The SonarCloud Quality Gate has passed with zero issues. There are no failing checks, no security hotspots, and no mandated fixes. The PR is clear of code quality blockers.
The `gemini-code-assist[bot]` comment offers optional test coverage suggestions (validating `exempt_labels`, `_schema_version` keys, and the `security` label), but these are not blocking recommendations—the current test suite satisfies the quality gate.

@don-petry
don-petry enabled auto-merge (squash) June 30, 2026 22:00

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/pr-limits-tests.yml:
- Around line 51-58: The workflow currently installs shellcheck in the setup
step but never uses it, so either remove the unused shellcheck install from the
install step or add a dedicated shellcheck invocation for the bats scripts in
this job. Update the PR-limits workflow by editing the install-and-run sequence
around the Install bats, shellcheck, and jq and Run bats suite steps so the
installed tools match the actual commands executed.
- Around line 46-49: Update the Checkout repository step in the pr-limits-tests
workflow to disable credential persistence; the job only reads the repo, so add
persist-credentials: false to the actions/checkout usage. Locate the checkout
step in the workflow and keep the rest of the test job unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: e440ee47-577c-4346-83b5-53e51ac8a226

📥 Commits

Reviewing files that changed from the base of the PR and between c2beceb and dbbf499.

📒 Files selected for processing (3)
  • .github/workflows/pr-limits-tests.yml
  • standards/pr-limits.json
  • test/scripts/pr-limits/pr-limits-config.bats

Comment thread .github/workflows/pr-limits-tests.yml Resolved
Comment thread .github/workflows/pr-limits-tests.yml Resolved
@don-petry
don-petry disabled auto-merge June 30, 2026 22:03
Address advisory review on #562:
- Gemini: include _schema_version and exempt_labels in the required-keys
  test, and assert the 'security' exempt-label (ADR §6/§7.4 safety property
  that urgent security/hotfix PRs are never throttled) plus a positive
  _schema_version.
- CodeRabbit: set persist-credentials: false on the read-only test job's
  checkout (zizmor) — it never pushes, so the token need not persist.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HUhVsBbihbTjVvbcVf1WuU
@don-petry
don-petry enabled auto-merge (squash) June 30, 2026 22:05
@don-petry
don-petry disabled auto-merge June 30, 2026 22:06
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 30, 2026
@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — fix-reviews (applied)

Changes committed and pushed.

@sonarqubecloud

Copy link
Copy Markdown

@don-petry

Copy link
Copy Markdown
Contributor Author

Dev-Lead — review-changes (no-changes)

No changes were needed for this PR.

@don-petry
don-petry enabled auto-merge (squash) June 30, 2026 22:11
@don-petry

Copy link
Copy Markdown
Contributor Author

@donpetry-bot review

@donpetry-bot

Copy link
Copy Markdown
Contributor

@don-petry I'm on it — starting a fresh review now. Results will appear in a few minutes.

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: LOW
Reviewed commit: f573aa1d181d90bc9cefc14bb9ca5f2ac3d0583e
Review mode: triage-approved (single reviewer)

Summary

Phase 2a of the PR-limits initiative (#507): adds standards/pr-limits.json (a provisional, machine-readable single source of truth for org-wide/per-source automation PR caps and exempt actors/labels), a 13-case Bats suite validating the config contract, and a path-filtered CI workflow to run it. Additive only (3 new files, +189/-0); no executable/runtime code paths change. Triage cleared this as low-risk and this confirmation review agrees.

Linked issue analysis

Closes #507 (Phase 2a — encode the PR-limit config as a machine-readable single source of truth). The PR delivers exactly the data file the issue scopes, with consumers (admission gate #507b, apply path #508) explicitly deferred. Config is flagged status=provisional with inline ADR references, matching the issue's 'data only, pending human sign-off' intent. Substantively addressed.

Findings

No blocking findings.

  • Prior CodeRabbit CHANGES_REQUESTED (on stale SHA dbbf499) raised two items: unused shellcheck install and credential persistence. Both resolved on later commits — the final workflow installs only bats+jq and sets persist-credentials: false on the read-only checkout.
  • Gemini (low-priority) advisory to cover _schema_version, exempt_labels, and the 'security' exempt label is now satisfied: the final test suite asserts all three (required-keys loop + dedicated 'security' and positive _schema_version cases).
  • New GitHub Actions workflow is well-hardened: permissions contents:read, pinned checkout SHA (v7.0.0), persist-credentials:false, timeout, concurrency cancel-in-progress. No pull_request_target, no untrusted input interpolation, no secrets — no Actions security smell.
  • Secret-scanning MCP tool not available in this environment; gitleaks CI check passed.
  • No secrets, auth, crypto, DB migrations, or runtime-behavior changes. Risk: LOW.

CI status

All checks green or appropriately skipped. PR-limits Tests (Validate config) SUCCESS; CI Lint/ShellCheck/Secret scan (gitleaks)/Agent Security Scan SUCCESS; CodeQL SUCCESS; SonarCloud Quality Gate passed (0 new issues, 0 hotspots); AgentShield SUCCESS. mergeStateStatus is BLOCKED only because the org-leads team review is still required (human gate) — not due to any failing check.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry merged commit 3ca1f86 into main Jun 30, 2026
23 checks passed
@don-petry
don-petry deleted the dev-lead/issue-507-20260630-2153 branch June 30, 2026 22:14

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: LOW
Reviewed commit: f573aa1d181d90bc9cefc14bb9ca5f2ac3d0583e
Review mode: triage-approved (single reviewer)

Summary

Adds standards/pr-limits.json (provisional, data-only single source of truth for PR-limit caps + exempt actors/labels), a Bats suite validating its contract, and a path-filtered CI workflow to run it. 3 new files, +189/-0, no executable/logic changes. Triage-approved confirmation: assessment holds.

Linked issue analysis

Closes #507 (Phase 2a). AC#1 (single-source JSON with org-wide cap, per-source sub-caps for dev-lead/claude/initiative-driver, exempt-actor list) and AC#2 (all numerics flagged provisional pending human sign-off) are both met by standards/pr-limits.json and enforced by the Bats tests. Scope is data-only as specified; admission gate (#507b) and apply path (#508) are explicitly out of scope.

Findings

No blocking findings. Prior CodeRabbit CHANGES_REQUESTED was raised on a stale commit (dbbf499) and both its items are already resolved in the reviewed head f573aa1: (1) unused shellcheck install removed — install step now fetches only 'bats jq'; (2) checkout now sets persist-credentials: false. Workflow uses contents: read only and pins actions/checkout by SHA (v7.0.0). No secrets, no migrations, no auth/crypto surface. Secret-scanning MCP tool not exposed in this environment; relied on gitleaks CI (passed).

CI status

All required checks green: Validate config, Lint, ShellCheck, CodeQL, Analyze (actions), SonarCloud, Secret scan (gitleaks), Agent Security Scan, AgentShield, npm audit all SUCCESS. Remaining ecosystem audits (pnpm/pip/cargo/govulncheck) and dependabot-automerge SKIPPED (not applicable). reviewDecision: APPROVED.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Phase 2a] Encode the PR-limit config as a machine-readable single source of truth

2 participants