Skip to content

feat: implement issue #1805 — dev-lead PR bodies are just 'Closes #N' — triage escalates every dev-lead PR for missing 3+ of 5 required description sections - #1806

Merged
don-petry merged 3 commits into
mainfrom
dev-lead/issue-1805-20260913-1008
Sep 13, 2026
Merged

don-petry merged 3 commits into
mainfrom
dev-lead/issue-1805-20260913-1008

Conversation

@don-petry

@don-petry don-petry commented Sep 13, 2026 •

Copy link
Copy Markdown
Collaborator

User description

Closes #1805

Implemented by dev-lead agent. Please review.


CodeAnt-AI Description

Add complete PR descriptions for dev-lead changes and repair incomplete existing PRs

What Changed

  • New dev-lead PRs include Problem, Risk, Test plan, Rollback, and Monitoring sections alongside the issue-closing reference.
  • Existing dev-lead PRs missing at least three required sections are repaired during fix and review runs without replacing their original content.
  • Repairs run only when the required sections are still missing, preserve concurrent edits, and avoid repeated body changes.
  • Section content reflects the issue and changed files, including applicable risk, test, rollback, and monitoring guidance.
  • Added automated coverage and updated the repository PR template to require all five sections.

Impact

✅ Fewer dev-lead PRs escalated for incomplete descriptions
✅ Clearer test, risk, rollback, and monitoring guidance
✅ Existing PR content preserved during one-time repairs

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

… — triage escalates every dev-lead PR for missing 3+ of 5 required description sections
@qodo-code-review

Copy link
Copy Markdown

ⓘ Qodo reviews are paused because your trial has ended. Ask your workspace admin to add credits to resume reviews. Manage billing

@codeant-ai

codeant-ai Bot commented Sep 13, 2026 •

Copy link
Copy Markdown

🤖 CodeAnt AI — Review Status

Status Commit Started (UTC) Finished (UTC)
✅ Incremental review completed 7c32edb Sep 13, 2026 · 11:58 11:59
✅ Reviewed your PR 9346d9d Sep 13, 2026 · 10:18 10:20

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@codeant-ai

codeant-ai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@coderabbitai

coderabbitai Bot commented Sep 13, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 50 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 505117d9-7ce2-4e5e-a8f8-cf1afc32235d

📥 Commits

Reviewing files that changed from the base of the PR and between 675cb4a and 5b1fb16.

📒 Files selected for processing (7)
  • .github/pull_request_template.md
  • .github/workflows/lint.yml
  • scripts/dev-lead-fix-ci.sh
  • scripts/dev-lead-fix-issue.sh
  • scripts/dev-lead-fix-reviews.sh
  • scripts/lib/dev-lead-pr-body.sh
  • tests/test_dev_lead_pr_body.bats

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Sep 13, 2026
Comment thread .github/pull_request_template.md Resolved
Comment thread scripts/lib/dev-lead-pr-body.sh Resolved
Comment thread scripts/lib/dev-lead-pr-body.sh Resolved
@codeant-ai

codeant-ai Bot commented Sep 13, 2026

Copy link
Copy Markdown

CodeAnt Nitpicks

2 code suggestions

1. With set -e, a failed gh pr create exits before the cleanup command, leaving the PID-based temporary body file behind on every failed attempt.

Resource leak · scripts/dev-lead-fix-issue.sh:642-648


2. The generated PR body claims the lint script ran even when dev-lead-fix-issue.sh skipped it because LINT_SCRIPT was unavailable.

Docstring mismatch · scripts/lib/dev-lead-pr-body.sh:70-79

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces structured PR-body generation and backfilling for the dev-lead agent to prevent triage escalation due to missing description sections, updating the PR template, scripts, and adding unit tests. The review feedback highlights several key improvements: securing temporary file creation in dev-lead-fix-issue.sh using mktemp with error handling, making the HTML comment stripping regex in dev-lead-pr-body.sh non-greedy to prevent over-matching, and asserting specific non-zero exit codes in the BATS tests instead of using generic non-zero checks.

Comment thread scripts/dev-lead-fix-issue.sh Resolved
Comment thread scripts/lib/dev-lead-pr-body.sh Resolved
Comment thread tests/test_dev_lead_pr_body.bats Resolved
Comment thread tests/test_dev_lead_pr_body.bats Resolved
@don-petry

Copy link
Copy Markdown
Collaborator Author

template-drift (Tier-1 red) — root cause is out-of-scope external-repo drift, left red intentionally.

template-drift verifies that every file scripts/seed-repo-template.sh ships into petry-projects/repo-template stays byte-identical to its standards/-derived baseline, so the template stays a thin distribution layer and never becomes a competing fork (#969).

Running scripts/template_stub_drift.sh locally shows the two DRIFTED files are both in the external repo-template repo:

  • .github/workflows/sonarcloud.yml — committed 227464a1 != expected d7aa265f
  • .github/dependabot.yml — committed 23d611e8 != expected 3e2daee1

Neither is touched by this PR. This PR's diff is confined to the dev-lead PR-body sections work (scripts/lib/dev-lead-pr-body.sh, the three dev-lead-fix-*.sh callers, tests/test_dev_lead_pr_body.bats, .github/pull_request_template.md, and one line adding the new bats file to lint.yml's bats job). None of those files is in the template-drift covered-file manifest.

This is the pre-existing #1448 known-red state, which is exactly why lint.yml excludes template-drift from the main-push trigger (if: github.event_name != 'push', see the job's own comment). The correct fix is re-seeding repo-template via scripts/seed-repo-template.sh in that repository — not a change I can or should make from this PR. Making the check green here would require relaxing what it asserts, which #1468 prohibits. Leaving it red with this explanation.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #1806
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-09-13T10:51:05Z

@don-petry

Copy link
Copy Markdown
Collaborator Author

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-09-13T10:51:05Z

@don-petry
don-petry enabled auto-merge (squash) September 13, 2026 10:21
@donpetry-bot

donpetry-bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at 9346d9d3298546d37166a148267184733916df4b — click to expand prior review.

Review — fix requested (cycle 1/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: 9346d9d3298546d37166a148267184733916df4b
Cascade: triage → deep (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5)

Summary

Adds a pure, unit-tested lib (dev-lead-pr-body.sh) that generates the five required PR description sections and idempotently backfills existing dev-lead PRs so triage stops escalating them for a missing description; wiring into fix-issue/fix-ci/fix-reviews and the PR template is clean and well-commented. Triage's headline 'security-critical: writes $body instead of $pr_body' is REFUTED — the head commit writes $pr_body under set -euo pipefail (the bot reviewed a stale commit). Remaining findings are minor: a CONFIRMED greedy-regex bug that drops text between inline HTML comments in the Problem summary, an unconditional 'lint ran pre-commit' claim, and a predictable $$ temp file (inconsistent with the backfill path's mktemp). Not HIGH — no security/safety-critical path defect — so no security-audit tier; escalating to the author on these findings. Note: the only red CI check, template-drift, is pre-existing known-red under #1448 and does not cover pull_request_template.md, so it is unrelated to this PR; all PR-relevant checks (shellcheck, bats, unit-tests, lint, CodeQL) are green.

Findings

  • MINOR (correctness): Greedy HTML-comment strip in dlpb_problem_line: sed 's/<!--.*-->//g' matches from the first <!-- to the last --> on a line, deleting any prose between two inline comments. Reproduced: input first <!-- a --> KEEPTHIS <!-- b --> last yields first last (KEEPTHIS dropped). Only degrades the derived Problem summary text — the section still emits and the gate still passes — but it is a real defect. Fix: sed 's/<!--[^>]*-->//g' (sed has no non-greedy; a negated char class is the idiom). [auditable: repro confirmed] — scripts/lib/dev-lead-pr-body.sh:121
  • MINOR (correctness): dlpb_test_plan_line unconditionally states bash scripts/dev-lead-lint.sh ... ran pre-commit. The string is hardcoded regardless of whether lint actually ran in a given dev-lead pass, which risks a false test-plan claim — the same truthfulness bar the lib's own header sets ('never claims tests that were not run'). Consider deriving this from whether the lint step actually executed. [auditable: repro unverifiable] — scripts/lib/dev-lead-pr-body.sh:75
  • MINOR (security): Create path writes the body to a predictable, PID-named path /tmp/dev-lead-pr-body-$$.md (dev-lead-fix-issue.sh:97) and, under set -euo pipefail, a failed gh pr create exits before rm -f, leaking the file. Low real-world impact (ephemeral GitHub-hosted runner, /tmp), but the backfill path in this same PR already uses mktemp (dev-lead-pr-body.sh:240) — align the create path to mktemp + a trap ... EXIT cleanup for consistency and to avoid the predictable-path smell. — scripts/dev-lead-fix-issue.sh:97
  • MINOR (concurrency): dlpb_backfill_pr_body reads the PR body then gh pr edit --body-file writes it back with no optimistic-concurrency check, so a body edited between read and write is overwritten. Likelihood is low (dev-lead workflow concurrency control; errors are swallowed best-effort), but worth noting since backfill can run on any open PR. — scripts/lib/dev-lead-pr-body.sh:224
  • INFO (correctness): Backfill is skipped whenever the body contains the literal marker <!-- dev-lead-pr-body-backfill -->. A user typing that exact HTML comment would suppress backfill, but the string is specific enough that collision is effectively impossible; acceptable as-is. [auditable: repro unverifiable] — scripts/lib/dev-lead-pr-body.sh:193
  • INFO (ci): The only red check, template-drift, is documented known-red under [bug] seed-repo-template fetches standards unpinned — pin to published stable, compliance accepts N-1 (continues #1436) #1448 and excluded from the push trigger; its manifest (template_stub_drift.sh) covers repo-template distribution stubs, NOT .github/pull_request_template.md, so it is unrelated to this PR. All PR-relevant checks are green (shellcheck, ShellCheck, bats, unit, unit-tests, lint, CodeQL, SonarCloud). — .github/workflows/lint.yml:286
  • INFO (test-quality): Two backfill negative-path tests assert [ "$status" -ne 0 ] rather than the specific -eq 1 the repo convention prefers (Gemini low-priority). Non-blocking tightening. — tests/test_dev_lead_pr_body.bats:134

Reviewed by the PR-review cascade (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@donpetry-bot

donpetry-bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor
Superseded by automated re-review at 7c32edb3bdcd33a261bbdfb36e1f3d7198e4e48e — click to expand prior review.

Review — fix requested (cycle 2/3)

The automated review identified the following issues. Please address each one:

Findings to fix

Automated review — NEEDS HUMAN REVIEW

Risk: MEDIUM
Reviewed commit: 9346d9d3298546d37166a148267184733916df4b
Cascade: triage → deep (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5)

Summary

New dev-lead PR-body builder lib (#1805) makes dev-lead emit the five required description sections so triage no longer escalates on a description gap. Two real but minor correctness defects: a greedy HTML-comment strip in dlpb_problem_line that deletes issue text between two same-line comments (confirmed by repro), and a temp-file leak in dev-lead-fix-issue.sh when gh pr create fails under set -e. Risk is MEDIUM (automation shell logic, no auth/secrets/migrations); the failing template-drift check is the pre-existing known-red #1448 (excluded from push, unrelated to this diff), and all diff-relevant checks (shellcheck/bats/unit/actionlint/CodeQL/gitleaks/Sonar) are green. Escalating for author fixes rather than approving; no security tier needed.

Findings

  • minor [logic/correctness @ scripts/lib/dev-lead-pr-body.sh:121]: dlpb_problem_line strips HTML comments with a greedy sed 's/<!--.*-->//g'. When an issue body has two comments on one line, all text between them is deleted. Repro: input ' KEEP-THIS-TEXT ' yields an empty summary (KEEP-THIS-TEXT lost). Use a non-greedy class, e.g. sed 's/<!--[^>]*-->//g'. Only degrades the Problem section content; does not affect gate passing.
  • minor [resource-leak @ scripts/dev-lead-fix-issue.sh:640]: In dev-lead-fix-issue.sh the PR body is written to /tmp/dev-lead-pr-body-$$.md and cleaned up with rm -f AFTER gh pr create. Under set -euo pipefail, a failed gh pr create exits before the rm, leaking the temp file. The PID-based path in /tmp is also predictable. Prefer mktemp + trap 'rm -f "$pr_body_file"' RETURN/EXIT (the sibling dlpb_backfill_pr_body already uses mktemp).
  • minor [logic/correctness @ scripts/lib/dev-lead-pr-body.sh:90]: dlpb_test_plan_line hardcodes 'bash scripts/dev-lead-lint.sh (shellcheck) ran pre-commit'. If a dev-lead pass skips that lint step, the generated Test plan section makes an untrue claim, contradicting the lib's stated truthfulness goal. Consider deriving the claim from whether lint actually ran this pass.
  • info [logic/correctness @ .github/pull_request_template.md:8]: The PR template's leading 'Delete sections that do not apply to this PR.' instruction now also covers the five newly-required sections, so a human contributor can delete them and re-trigger the missing-description triage this PR aims to prevent. Not a code defect (dev-lead itself emits all five), but it undercuts the fix's durability for human-authored PRs. Matches codeant advisory @ pull_request_template.md:28.

Reviewed by the PR-review cascade (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5). Reply if you need a human review.

Additional tasks

  1. Resolve all unresolved review thread comments from other reviewers
  2. Ensure all CI checks pass after your changes
  3. Rebase on the target branch if behind
  4. Do NOT modify files unrelated to the findings above

The review cascade will automatically re-review after new commits are pushed.

@don-petry

Copy link
Copy Markdown
Collaborator Author

No description provided.

@don-petry
don-petry disabled auto-merge September 13, 2026 11:49
@don-petry

Copy link
Copy Markdown
Collaborator Author

template-drift check (failing) — assessed, not gamed.

template-drift verifies that files committed in petry-projects/repo-template are byte-identical to their standards-derived baseline (so the template stays a thin distribution layer of standards/ and never forks). It resolves the covered files from a fixed manifest in scripts/template_stub_drift.sh — workflow stubs plus .github/dependabot.yml, .github/CODEOWNERS, CLAUDE.md, .gitleaks.toml — and reads them cross-repo via gh api.

This branch touches none of those covered files (its diff is .github/pull_request_template.md, .github/workflows/lint.yml, and the dev-lead scripts/tests), so the failure is not introduced by this PR. lint.yml itself documents this job as "known-red under #1448" and it depends on live cross-repo state in repo-template. The root cause therefore lives in repo-template / #1448, and the only way to flip it green from here would be to relax what the check asserts — which is exactly the anti-gaming failure mode to avoid (#1468, agentic-interaction-model §13). Leaving it red and flagging it instead; it needs re-seeding repo-template (scripts/seed-repo-template.sh) or the #1448 resolution, both out of scope for this review-response.

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (partial)

A commit was pushed, but not every requested change was applied. Per requested item:

  • .github/pull_request_template.md:28 — not applied
  • scripts/lib/dev-lead-pr-body.sh:193 — applied
  • scripts/lib/dev-lead-pr-body.sh:224 — not applied
  • scripts/dev-lead-fix-issue.sh:640 — applied
  • scripts/lib/dev-lead-pr-body.sh:121 — applied
  • tests/test_dev_lead_pr_body.bats:134 — applied
  • tests/test_dev_lead_pr_body.bats:158 — applied

The unaddressed items above still need work.

@don-petry
don-petry enabled auto-merge (squash) September 13, 2026 11:59
@codeant-ai codeant-ai Bot added size:XL This PR changes 500-999 lines, ignoring generated files and removed size:L This PR changes 100-499 lines, ignoring generated files labels Sep 13, 2026
@don-petry
don-petry disabled auto-merge September 13, 2026 12:00
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) September 13, 2026 12:06

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: 7c32edb3bdcd33a261bbdfb36e1f3d7198e4e48e
Cascade: triage → deep (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5)

Summary

Implements structured dev-lead PR bodies (#1805) via a pure, unit-tested lib plus an idempotent marker-keyed backfill; the generated body passes sc_description_missing with 0|. On deep inspection the three escalated advisory signals are non-blocking: a minor /tmp file leak only on a terminal gh-create failure, a by-design template-guidance gap (enforcement is the triage gate, not the template), and an explicitly-mitigated best-effort TOCTOU in the backfill. The only red check (template-drift) is the pre-existing known-red failure under #1448/#1486 documented in the workflow and covers no file this PR touches; all PR-relevant checks (shellcheck, bats, unit-tests, CodeQL, SonarCloud, actionlint, AgentShield) are green. No auth/secrets/migrations/injection and no safety hard-stops.

Findings

  • minor: Under set -euo pipefail, a failed gh pr create exits before rm -f "$pr_body_file", leaking the PID-named temp body file in /tmp on the create-failure path. Low blast radius (one file per failed run on a terminal-abort path; OS reclaims /tmp), but a trap 'rm -f "$pr_body_file"' EXIT after mktemp would close it cleanly.
  • info: The template comment instructs contributors to keep the five required sections, but nothing in the template can force it. This is by design: the enforcing layer is the sc_description_missing triage gate that escalates PRs missing 3+ sections, plus the new backfill for existing PRs. Not a code defect.
  • info: dlpb_backfill_pr_body re-reads the PR body immediately before gh pr edit and skips when it changed since the snapshot, shrinking the TOCTOU window; the author documents it cannot be fully closed and the whole backfill is best-effort (all gh failures swallowed). Acceptable for a cosmetic body append that preserves original content.

Reviewed by the PR-review cascade (triage: haiku 4.5 [sonnet 5] → deep: opus 4.8 [sonnet 5] + duck: o4-mini → audit: fable 5). Reply if you need a human review.

@don-petry
don-petry merged commit 20e00f8 into main Sep 13, 2026
68 of 70 checks passed
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-human-review Flagged by automated PR review agent size:XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dev-lead PR bodies are just 'Closes #N' — triage escalates every dev-lead PR for missing 3+ of 5 required description sections

2 participants