Repository navigation
Description
Background
The current PR Review Agent (agents/pr-reviewer.md) provides a solid cascading-tier framework but lacks specificity in several high-value areas. The articles below identify common failure modes in AI-generated PRs that the agent should actively surface:
- https://github.blog/ai-and-ml/generative-ai/agent-pull-requests-are-everywhere-heres-how-to-review-them/
- https://www.workflow86.com/a-checklist-for-pull-requests/
Goals
1. Detect CI Weakening (Hard Stop)
Any change that reduces CI coverage or disables safety checks must be an automatic blocker, regardless of risk tier.
2. Flag Code Duplication
Agent-generated PRs frequently reimplement existing utilities. The reviewer should search for duplicate logic before approving.
3. Enforce Correctness Tracing
The agent should trace at least one critical path end-to-end and verify permission checks on all branches, not just summarize the diff.
4. Reject Structureless Large PRs
PRs touching 5+ unrelated files with no implementation plan should be escalated immediately rather than reviewed deeply.
5. Check for Prompt Injection in Workflows
CI workflow files that execute model output or pass untrusted user input to shell commands must be flagged as HIGH risk.
6. Score PR Description Quality
The review output should assess whether the PR description includes: problem statement, risk categorization, test plan, rollback procedure, and monitoring plan — and note what's missing.
7. Structured Review Timeline
Reviews should follow a time-bounded protocol aligned with risk tier to reduce token spend and improve consistency.
Acceptance Criteria
AC-1: CI Weakening Detection
- The agent detects and flags any of the following as a HIGH severity blocker that prevents approval:
- Reduced test coverage thresholds in CI config
- Skipped or deleted test cases
- Disabled workflow steps or conditional gates added to bypass checks
- Findings include the specific file path and line number of the weakening change
AC-2: Code Duplication Check
- For any new utility, helper, middleware, or validation function introduced in the PR, the agent performs a codebase search for similar existing implementations
- If a near-duplicate is found, the finding is reported as MEDIUM severity with the path to the existing implementation
- Approval is withheld until duplication is resolved or the author justifies the new implementation
AC-3: Critical Path Tracing
- For MEDIUM and HIGH risk PRs, the agent traces at least one critical path (input → transform → output) rather than only reviewing changed lines
- Permission checks on all branches of any auth-related code are verified
- Boundary conditions (empty, zero, max values) are noted in the findings if not covered by tests
AC-4: Large PR Gating
- PRs meeting any of the following criteria are immediately escalated (Tier 3) without deep review:
- Changes span 5+ unrelated files with no stated implementation plan in the PR body
- PR purpose cannot be summarized in one sentence
- Only test file changes while CI is failing
- Escalation comment requests a structured breakdown before further review investment
AC-5: Prompt Injection in Workflows
- The agent scans any modified
.github/workflows/*.ymlfiles for:- User-controlled input passed to
run:steps without sanitization - Model output executed directly as shell commands
- Tokens with write permissions used where read-only would suffice
- User-controlled input passed to
- Any match is reported as HIGH severity with a reference to the specific step
AC-6: PR Description Scoring
- The review output includes a PR Description Quality section that checks for the presence of:
- Problem statement (what and why)
- Risk category (high/medium/low with justification)
- Test plan (what was tested and how)
- Rollback procedure
- Monitoring/observability plan
- Missing required elements are listed explicitly; PRs missing 3+ elements receive a MEDIUM finding recommending description improvement before merge
AC-7: Tiered Review Protocol
- Tier 1 (triage) completes without tool calls and classifies: scope, CI weakening indicators, large-PR gate
- Tier 2 (deep review) follows a structured order: CI/workflow changes → duplication search → critical path trace → security boundaries
- Tier 3 (escalation) fires for: HIGH risk, CI failing, large structureless PR, prompt injection finding, or 3+ missing description elements
- Idempotency marker (
<!-- pr-review-agent v1 sha=<SHA> -->) is checked before any tool calls are made
Out of Scope
- Changes to the risk tier thresholds (LOW/MEDIUM/HIGH decision table)
- Human reviewer workflow or assignment logic
- Integration with external SAST tools (separate initiative)
Activity
don-petry commentedon May 11, 2026
Claude finished @don-petry's task in 4m 59s —— View job •
claude/issue-43-20260511-2113• [Create PR ➔](https://github.com/petry-projects/.github-private/compare/main...claude/issue-43-20260511-2113?quick_pull=1&title=Issue%20%2343%3A%20Changes%20from%20Claude&body=This%20PR%20addresses%20issue%20%2343%0A%0AGenerated%20with%20%5BClaude%20Code%5D(https%3A%2F%2Fclaude.ai%2Fcode)Implementing PR Review Agent Improvements
agents/pr-reviewer.md:@petry-projects/org-leads)Summary
All 7 acceptance criteria from the issue are implemented in a single file change to
agents/pr-reviewer.md. The key additions:if: false,continue-on-error: true, deleted tests — hard stop blocker with file:line reporting${{ github.event.* }}inrun:steps, model output as shell, write-scoped tokens,pull_request_targetwithout trust checks — HIGH severity hard stopView PR #131 ·
feat(agents): improve PR review agent with 7 new safety checks
feat(agents): improve PR review agent with 7 new safety checks
don-petry commentedon Oct 21, 2026
Closing after the 2026-10-21 backlog triage.
Why: May wishlist; CI-weakening/duplication/injection checks now in prompts; superseded by #1088.
Evidence: main:prompts/deep-review*.md, agents/pr-reviewer.md; PR #131 closed
Reopen if this still reproduces on main.