Skip to content

Fix PR branch rejection for Copilot-review-only rulesets - #75230

Merged
pelikhan merged 4 commits into
mainfrom
pelikhan/protected-branch-safe-outputs
Nov 6, 2026
Merged

pelikhan merged 4 commits into
mainfrom
pelikhan/protected-branch-safe-outputs

Conversation

@pelikhan

@pelikhan pelikhan commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

GitHub reports protected: true for branches covered only by automatic Copilot code-review rulesets, although those rules do not restrict pushes. This caused Design Decision Gate run 42076616520 to reject an ADR push for #75196 before any git operation. Current main retained the same broad flag check after #74946.

Approach

Permit a narrow exception only when branch metadata explicitly disables classic protection, every applicable rule is copilot_code_review, and the classic protection endpoint confirms absence with HTTP 404. Read rule pages directly: Octokit's paginator can swallow a later-page HTTP 409 and return incomplete results.

Default branches, classic protection, restrictive or unknown rules, mixed rulesets, malformed/empty initial responses, and failed confirmation remain blocked. Existing GitHub enforcement and push mechanics are unchanged. No repository rules, credentials, workflow prompts, or generated locks are modified.

Validation

  • 398 tests passed across pr_helpers.test.cjs and push_to_pull_request_branch.test.cjs, including later-page restrictive rules/409, classic protection, default branches, unavailable metadata, and handler-level no-git assertions.
  • TypeScript checks and git diff --check passed.
  • make agent-report-progress passed; a subsequent guarded error-status adjustment passed make agent-report-progress-no-test and the same focused tests without lint warnings.

Review notes and limitations

Classic-protection confirmation can require Administration read permission. If the existing runtime token cannot confirm absence, this exception intentionally remains blocked; this change does not request or grant broader credentials. The observed ruleset metadata is current, not a historical snapshot of the failed run.

The initial read-only security review identified pagination and classic-protection evidence issues, addressed in this candidate. Formal security-review clearance for the revised source remains unavailable because the synchronous reviewer invocation could not accept follow-up messages. This draft does not claim that clearance; the exception and permission behavior need review before merge.

Only reference-documentation prose changed in Markdown, documenting the narrower classification and fail-closed permission handling. No workflow frontmatter, executable snippets, prompts, imports, sandbox, or credential destinations changed. No live workflow was dispatched or rerun.

Relates to #75196.

Confirm classic protection is absent and read every applicable rules page before allowing the narrow advisory-only exception. Keep restrictive and uncertain protection blocked.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan marked this pull request as ready for review November 5, 2026 19:17
Copilot AI dynamic review requested due to automatic review settings November 5, 2026 19:17
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · routed: opus55 medium · ~11 AIC · ⌖ ~2 AIC · ⊞ 7.7K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

Generated by PR Specification Reviewer for #75230

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: gpt56 max · ~1 AIC · ⌖ ~1 AIC · ⊞ 6.3K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt56 max · ~3 AIC · ⌖ ~1 AIC · ⊞ 118 · ◷

Generated by Ponytail Reviewer for #75230

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head ef1b8f09d04c0b1c43b867b84c8ce806d81aa8a2 against the Safe Outputs MCP Gateway Specification (repository checkout-resolution requirements RCR1–RCR7) and Checkout Behavior Specification §6.3/T-CHK-015. The implementation preserves target/default-branch and checkout authorization boundaries, enumerates rules pages, and fails closed on incomplete protection evidence. I found no demonstrated normative contract violation requiring code changes.

One documentation correction is requested inline: the new claim that branch and rules reads use only contents: read is inaccurate for the rules/classic-protection endpoints, which may require Administration: read; this matters because the implementation intentionally blocks when those reads cannot be confirmed. No specification synchronization or branch push was necessary.

Generated by PR Specification Reviewer for #75230 · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review-specs to run again

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)
reviewed_at: 2026-11-05T19:21:22Z
review_event: REQUEST_CHANGES
top_themes: unbounded pagination in branch-rule verification; fail-closed protection checks
files_reviewed: actions/setup/js/pr_helpers.cjs, actions/setup/js/pr_helpers.test.cjs, actions/setup/js/push_to_pull_request_branch.test.cjs, docs/src/content/docs/reference/safe-outputs-pull-requests.md
comment_count: 1

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K · ◷
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The protection classification is narrowly scoped and otherwise fail-closed, but the new pagination loop has no termination bound. A malformed or non-paginating response can hang the safe-output job until runner timeout and consume API rate limits. Please bound pagination and add a regression test.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

Comment thread actions/setup/js/pr_helpers.cjs Resolved

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd. Requesting changes: the new exception can let a protected default branch through when repos.get fails.

📋 Key Findings

Blocking

  • Default-branch bypass. If repos.get fails, defaultBranch is null and the default-branch guard is skipped. With only Copilot review rules, the new exception then allows a protected default branch. I reproduced this with a standalone Node probe: base returns a protection-rules block, and this PR returns null. Guarding the exception with defaultBranch restores base behaviour and keeps the intended allow path. See the inline comment on pr_helpers.cjs.

Non-blocking

  • Add a regression test for the failing repos.get case (inline on the test file).
  • The docs claim the default branch is "always rejected"; that is not true when repos.get fails (inline on the doc).
  • The rules loop has no page cap (inline on pr_helpers.cjs).

Positive highlights

  • ✅ Fails closed on malformed, empty, mixed, and unknown rules, and on any rules-query error.
  • ✅ Reads rule pages directly to avoid the paginator swallowing a later-page 409.
  • ✅ Good test coverage of the allow and block matrix.

Limitations

I did not run the vitest suite here because it is not installed in this environment. The default-branch finding comes from a standalone Node probe that exercises checkBranchPushable directly.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~4 AIC · ⌖ ~1 AIC · ⊞ 5.4K
Comment /matt to run again

Comment thread actions/setup/js/pr_helpers.cjs Resolved
Comment thread actions/setup/js/pr_helpers.test.cjs Resolved
Comment thread actions/setup/js/pr_helpers.cjs Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The branch-protection exception can fail open when default-branch metadata is unavailable or a generic 404 is treated as absent protection, potentially allowing direct agent commits to default or protected branches.

3 open issues
What changed in this PR

This PR narrows gh-aw’s safe-output branch checks so Copilot-review-only rulesets can qualify for an exception instead of automatically blocking PR branch pushes.

Changes:

  • Adds an exception requiring explicit metadata, Copilot-only rules, and classic-protection absence confirmation.
  • Reads rules page by page and blocks the exception when rule evidence is incomplete or unavailable.
  • Adds helper and handler regression coverage and documents permission limitations.
File Description
docs/​src/​content/​docs/​reference/​safe-outputs-pull-requests.md Documents the exception and permission-dependent behavior.
actions/​setup/​js/​push_to_pull_request_branch.test.cjs Covers successful Copilot-only pushes and rejection before git operations.
actions/​setup/​js/​pr_helpers.test.cjs Covers pagination, malformed responses, classic protection, and API failures.
actions/​setup/​js/​pr_helpers.cjs Implements rule classification and classic-protection confirmation.

Review effort: Max (auto)

Comment thread actions/setup/js/pr_helpers.cjs Resolved
Comment thread actions/setup/js/pr_helpers.cjs Resolved
Comment thread actions/setup/js/pr_helpers.cjs Resolved
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (docs/src/content/docs/reference/safe-outputs-pull-requests.md:594): The new permission statement is inaccurate for the APIs this change introduces. GET /repos/{owner}/{repo}/branches/{branch}/rules and GET .../protection require Administration: read for fine-grained tokens, not merely Contents: read. Because the implementation intentionally fails closed when these reads are unavailable, please change this sentence to say that branch metadata may use Contents: read but rules/classic-protection confirmation may require Administration: read, consistent with the following sentence and the frontmatter reference. - Fix PR branch rejection for Copilot-review-only rulesets #75230 (comment)
  3. Review (actions/setup/js/pr_helpers.cjs:120): The unbounded pagination loop can hang the safe-output job indefinitely if the endpoint repeatedly returns full Copilot-only pages or ignores the page parameter, rather than failing closed. - Fix PR branch rejection for Copilot-review-only rulesets #75230 (comment)
  4. Review (actions/setup/js/pr_helpers.cjs:168): [/diagnosing-bugs] Regression: when repos.get fails, defaultBranch stays null and the default-branch guard is skipped. This exception then lets a protected default branch through, which the base code blocks. I reproduced it: main with only Copilot review rules and a failing repos.get returns null on this branch, while base returns a protection-rules block. - Fix PR branch rejection for Copilot-review-only rulesets #75230 (comment)
  5. Review (actions/setup/js/pr_helpers.test.cjs:529): [/tdd] This test only covers a successful repos.get, so it misses the bypass above. Add a red case where repos.get rejects and the push must still be blocked. - Fix PR branch rejection for Copilot-review-only rulesets #75230 (comment)
  6. Review (docs/src/content/docs/reference/safe-outputs-pull-requests.md:596): [/grill-with-docs] This sentence is inaccurate. If repos.get fails, the default-branch check is skipped, so "always rejected" does not hold. That gap predates this PR, but the new text repeats the claim. Reword it to say the default branch is rejected whenever it can be identified. - Fix PR branch rejection for Copilot-review-only rulesets #75230 (comment)
  7. Review (actions/setup/js/pr_helpers.cjs:119): Low: this loop has no upper bound. If a response ignores page and keeps returning 100 copilot_code_review rules, the check never returns. Add a page cap that fails closed. - Fix PR branch rejection for Copilot-review-only rulesets #75230 (comment)

Once you start work, post this work summary as a PR comment without the leading Copilot mention, including the Sous-chef head, work, and state lines below, to record that the assignment is running. Apply the necessary fixes on this PR's existing branch, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved; stop when only human review or CI remains. Do not merge or trigger CI.

Sous-chef requested-head: ef1b8f0
Sous-chef requested-work: 41415b66511e5c8954a3064a6500cd68e07142ec1f4d7b07547d4259e3380852 59b39395fc0b02510bec55d5237a5150926263df2c8453e4c51e20bef9f5efe7 9b0e52a9972c4d7b15ed2277c34f9ba0661d72c5ac446030f97d20d25bcbab24 cceff0e46903aa2afb230adc075737d57575e3b747f7a6681b0967b6119c173e df5eedee0b49abeb0e187b051c52461db672d8f03819d883b6b49d123306250d ee7da072a12d91fac357da1351f4f2b7f318f0dec96b552f6e47f5040e86c908
Sous-chef requested-state: 3328fd5da2bb7c0a7e83e6c933a7d454fd52088001258eb5d1b9d8e062af3c82

Generated by 👨🍳 PR Sous Chef · pi · haiku45 · ~12 AIC · ⌖ ~10 AIC · ⊞ 1.2K · ◷
Comment /souschef to run again

Copilot AI commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

@copilot

Copilot assignment requested for the following outstanding work:...

Sous-chef requested-head: ef1b8f0
Sous-chef requested-work: 41415b66511e5c8954a3064a6500cd68e07142ec1f4d7b07547d4259e3380852 59b39395fc0b02510bec55d5237a5150926263df2c8453e4c51e20bef9f5efe7 9b0e52a9972c4d7b15ed2277c34f9ba0661d72c5ac446030f97d20d25bcbab24 cceff0e46903aa2afb230adc075737d57575e3b747f7a6681b0967b6119c173e df5eedee0b49abeb0e187b051c52461db672d8f03819d883b6b49d123306250d ee7da072a12d91fac357da1351f4f2b7f318f0dec96b552f6e47f5040e86c908
Sous-chef requested-state: 3328fd5da2bb7c0a7e83e6c933a7d454fd52088001258eb5d1b9d8e062af3c82

Work is underway: I’m merging the latest main, tightening the branch/ruleset checks, bounding pagination, and correcting the documentation. I’ll validate the changes and reply to the review threads; I will not merge the PR or trigger CI.

Copilot AI and others added 2 commits November 5, 2026 20:06
…anch-safe-outputs

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Require the authoritative Branch not protected response, explain Administration permission failures, and extend metadata and no-git regression coverage. Correct endpoint-specific permission documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@pelikhan
pelikhan merged commit 4e5de8b into main Nov 6, 2026
112 checks passed
@pelikhan
pelikhan deleted the pelikhan/protected-branch-safe-outputs branch November 6, 2026 05:48
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants