Skip to content

feat: implement issue #614 — [Phase 2] Register plan_json artifact type (plan rubric + structured-findings channel) - #761

Merged
don-petry merged 8 commits into
mainfrom
dev-lead/issue-614-20260615-1249
Jun 17, 2026
Merged

don-petry merged 8 commits into
mainfrom
dev-lead/issue-614-20260615-1249

Conversation

@don-petry

@don-petry don-petry commented Jun 15, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #614

Implemented by dev-lead agent. Please review.

Summary by CodeRabbit

  • New Features

    • Added plan review capability to validate and analyze initiative plans using structured critique rules.
    • Plan findings are now output as machine-readable JSON and integrated with GitHub Actions summaries.
  • Tests

    • Added comprehensive test suite for plan review workflow verification.

…pe (plan rubric + structured-findings channel)
@don-petry
don-petry requested a review from a team as a code owner June 15, 2026 13:02
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jun 15, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@don-petry, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 48 minutes and 39 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 643dc421-9a25-47b2-bcb8-afc297148ea6

📥 Commits

Reviewing files that changed from the base of the PR and between cd321f7 and e396628.

📒 Files selected for processing (2)
  • scripts/initiative-planner/review-plan.sh
  • tests/test_plan_review.bats
📝 Walkthrough

Walkthrough

Adds a plan_json review artifact type end-to-end: a fixed-critic rubric prompt (prompts/plan-review.md), a review-plan.sh driver that resolves the rubric and output channel from the registry and invokes run_agentic at the deep tier, a post-plan-findings.sh structured-findings output channel, Bats test coverage, and CI wiring.

Changes

plan_json Review Artifact Type

Layer / File(s) Summary
Plan review rubric and registry contract
prompts/plan-review.md, scripts/lib/README.md
Defines the fixed-critic checklist, pass/revise verdict semantics, and required findings JSON schema. Updates the registry README to document plan_json as a second artifact type with its rubric and output_channel mappings.
review-plan.sh driver
scripts/initiative-planner/review-plan.sh
New Bash entrypoint: validates the input plan.json, resolves rubric and output_channel from the registry, invokes run_agentic at the deep tier, captures findings JSON (file-first, stdout fallback), and dispatches to the resolved output channel script.
post-plan-findings.sh output channel
scripts/post-plan-findings.sh
New script: validates findings JSON presence and parseability, enforces pass/revise verdict, optionally writes normalized JSON to PLAN_FINDINGS_OUTPUT (creating parent dirs), emits a human-readable stdout block and optional GITHUB_STEP_SUMMARY section.
Bats tests, registry guard, and CI wiring
tests/test_plan_review.bats, tests/test_review_registry.bats, .github/workflows/lint.yml
New Bats suite covers registry resolution, driver wiring assertions, and post-plan-findings.sh behavior (happy path, error cases, directory creation, non-truncation). Updates the existing registry test to allow additive artifact types. Wires the new test into the CI bats job.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested labels

needs-human-review

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: registering the plan_json artifact type with plan rubric and structured-findings channel, directly addressing issue #614 Phase 2.
Linked Issues check ✅ Passed All five acceptance criteria from issue #614 are met: plan_json registered with rubric and output channel (AC#1), content_ref adapter implemented (AC#2), structured findings output channel created (AC#3), pr_diff unchanged (AC#4), and bats test coverage added (AC#5).
Out of Scope Changes check ✅ Passed All changes are directly aligned with issue #614 requirements. The modifications to test_review_registry.bats and documentation are scoped to support the additive plan_json registration without affecting existing pr_diff functionality.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dev-lead/issue-614-20260615-1249

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Phase 2 of the review registry by adding support for the plan_json artifact type. It registers a new adversarial plan rubric (prompts/plan-review.md), implements a content-ref adapter (review-plan.sh) and a structured-findings output channel (post-plan-findings.sh), and adds comprehensive BATS tests. Feedback on the changes highlights the need to print error logs to stderr when run_agentic fails to prevent debugging logs from being lost in the temporary directory, and to ensure the parent directory is created and file identity is checked before writing findings to avoid truncation issues.

Comment thread scripts/post-plan-findings.sh Resolved
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — review-changes (applied)

Changes committed and pushed.

@don-petry
don-petry enabled auto-merge (squash) June 15, 2026 13:11
@don-petry
don-petry disabled auto-merge June 15, 2026 13:13
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
- Quality Gate passed (0 new issues, 0 hotspots)
Files changed: None
Skipped: 0 issues flagged, all passing
```
No code changes are required—the PR has no new quality issues per SonarCloud, and all CI checks are passing or intentionally skipped. The PR is ready to merge from a code quality perspective.

@don-petry
don-petry enabled auto-merge (squash) June 15, 2026 13:13
donpetry-bot
donpetry-bot previously approved these changes Jun 15, 2026

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: LOW
Reviewed commit: 2004b137d1e3cef0ad8fe74e3f3362cd7e6ff53c
Review mode: triage-approved (single reviewer)

Summary

Phase 2 of the review-registry epic: registers the plan_json artifact type additively beside pr_diff. Adds a fixed adversarial plan rubric (prompts/plan-review.md), a content_ref adapter (review-plan.sh) reusing engine.sh deep-tier routing, and a structured-findings output channel (post-plan-findings.sh) that emits machine-readable findings for the planner and never posts a GitHub review. Clean, well-tested, additive change with all substantive CI green.

Linked issue analysis

Closes #614. All 5 acceptance criteria are met: (1) plan_json registry row maps to the plan rubric + structured-findings channel; (2) review-plan.sh feeds a plan.json into the rubric, reusing scripts/engine.sh run_agentic at the deep tier with no separate model selector; (3) post-plan-findings.sh delivers machine-readable findings JSON the planner consumes pre-apply-plan.sh, never calling post-pr-review.sh; (4) pr_diff is untouched — its cascade/channel guard stays green and the prior "sole type" test was correctly relaxed to a "never removed/renamed" guard; (5) tests/test_plan_review.bats (226 lines) exercises registry resolution, driver wiring, and output-channel behavior, and is registered in lint.yml.

Findings

No blocking issues. The two advisory findings from gemini-code-assist on the prior commit (3ab7bf7) were both resolved in the head commit (2004b13): (a) HIGH — error logs lost in the trap-deleted temp dir are now cat'd to stderr on failure (review-plan.sh, run_agentic failure + invalid-JSON fallback branches); (b) MEDIUM — the findings write now does mkdir -p on the parent and an -ef same-file identity check to avoid truncation (post-plan-findings.sh). Both fixes have dedicated regression tests. SonarCloud quality gate passed (0 new issues, 0 hotspots); CodeRabbit and Codex comments are rate-limit/usage notices, not findings.

CI status

All substantive checks green: ShellCheck, bats, CodeQL (actions + python), SonarCloud, gitleaks secret scan, agent-shield, dependency-audit, holdout-guard, and the dev-lead test suite. The only non-success entries are two CANCELLED dev-lead relay jobs (dev-lead / dispatch, dev-lead / ci-relay) — concurrency self-cancellations triggered by the [skip ci-relay] head commit, not real failures. SKIPPED entries are inapplicable language-specific audit stubs and dependabot-automerge. mergeStateStatus is BEHIND (base moved ahead); a rebase/merge of main resolves it and does not affect this review.


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry disabled auto-merge June 15, 2026 19:19
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
Issues addressed: 0
- No issues reported; quality gate passed with 0 new issues, 0 security hotspots
Files changed: None
Skipped (informational): 0
Status: No action required — PR is in good standing with all checks passing
```

@don-petry
don-petry enabled auto-merge (squash) June 15, 2026 19:20
@don-petry
don-petry disabled auto-merge June 16, 2026 21:28
@don-petry
don-petry enabled auto-merge (squash) June 16, 2026 21:32
@don-petry
don-petry disabled auto-merge June 16, 2026 21:33
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #761
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-06-16T22:05:49Z

@don-petry

Copy link
Copy Markdown
Collaborator Author

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-06-16T22:05:49Z

@don-petry
don-petry enabled auto-merge (squash) June 16, 2026 21:35
@don-petry
don-petry disabled auto-merge June 16, 2026 23:59
@don-petry
don-petry enabled auto-merge (squash) June 17, 2026 00:01
@don-petry
don-petry disabled auto-merge June 17, 2026 00:02
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — fix-bot-comment (no-changes)

Agent reasoning
- ✅ No failed/timed-out/cancelled checks with substantive work (dev-lead relay cancellations are expected self-cancellations)
- ✅ No CHANGES_REQUESTED reviews
- ✅ No blocking issues from SonarCloud, CodeQL, gitleaks, or ShellCheck
**Review state:**
- ✅ PR is APPROVED by donpetry-bot with detailed analysis
- ✅ gemini-code-assist[bot] provided feedback (resolved in prior commits per the approval review)
---
## Result
**No actionable issues to address.**
The SonarCloud comment is purely informational—it reports that the quality gate passed with zero new issues, hotspots, or code smells. The PR has no Tier 1 blockers and is already approved. No changes required.

@don-petry
don-petry enabled auto-merge (squash) June 17, 2026 00:02
@don-petry
don-petry disabled auto-merge June 17, 2026 16:55
@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — waiting on PR blockers (intent: review-changes)

PR: #761
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-06-17T17:29:17Z

@don-petry

Copy link
Copy Markdown
Collaborator Author

Note

@don-petry I reviewed this PR and no code changes were needed, but it still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews), so I cannot mark it done yet. I'll re-check automatically.
Next attempt after: 2026-06-17T17:29:17Z

@don-petry
don-petry enabled auto-merge (squash) June 17, 2026 16:59

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/initiative-planner/review-plan.sh`:
- Around line 74-75: The EXIT trap removes the entire WORK_DIR directory, but
PLAN_FINDINGS_OUTPUT defaults to a path within WORK_DIR (plan-findings.json).
This means the findings file is deleted before callers can consume it when
PLAN_FINDINGS_OUTPUT is not explicitly provided. Modify the trap cleanup logic
to preserve the findings file by either excluding it from the rm command,
copying it to a location outside WORK_DIR before cleanup, or by only deleting
specific subdirectories within WORK_DIR while preserving the findings output
file. Apply this fix to all instances where this pattern occurs (trap statement
around lines 74-75 and other occurrences referenced at lines 100-103).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 14b610e4-c0a3-4a2c-ada8-8778804a4a16

📥 Commits

Reviewing files that changed from the base of the PR and between a945977 and cd321f7.

⛔ Files ignored due to path filters (1)
  • scripts/lib/review-registry.tsv is excluded by !**/*.tsv
📒 Files selected for processing (7)
  • .github/workflows/lint.yml
  • prompts/plan-review.md
  • scripts/initiative-planner/review-plan.sh
  • scripts/lib/README.md
  • scripts/post-plan-findings.sh
  • tests/test_plan_review.bats
  • tests/test_review_registry.bats

@don-petry
don-petry disabled auto-merge June 17, 2026 17:01
coderabbitai[bot]
coderabbitai Bot previously approved these changes Jun 17, 2026
@don-petry
don-petry dismissed stale reviews from coderabbitai[bot] and donpetry-bot via e396628 June 17, 2026 17:06
@don-petry
don-petry enabled auto-merge (squash) June 17, 2026 17:06
@don-petry
don-petry disabled auto-merge June 17, 2026 17:07
@sonarqubecloud

Copy link
Copy Markdown

@don-petry

Copy link
Copy Markdown
Collaborator Author

Dev-Lead — waiting on PR blockers (intent: fix-reviews)

PR: #761
No changes were committed, but the PR still has blocking checks or reviews (failing or cancelled checks, or changes-requested reviews). The retry cron will re-attempt automatically. Next attempt after: 2026-06-17T17:38:01Z

@don-petry
don-petry enabled auto-merge (squash) June 17, 2026 17:08
@donpetry-bot

Copy link
Copy Markdown
Contributor

Advisory bots were rate-limited; auto-approval is withheld until they recover. pr-review-sweep will re-review this PR after 2026-06-17T18:08:10Z.

@donpetry-bot donpetry-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review — APPROVED ✓

Risk: LOW
Reviewed commit: e3966286106f740f589d3ea1f8e5e28f6493fc90
Review mode: triage-approved (single reviewer)

Summary

Phase 2 of epic #610 additively registers a new plan_json review artifact type: a fixed adversarial/semantic plan rubric (prompts/plan-review.md) bound to a machine-readable structured-findings output channel (post-plan-findings.sh), with a content_ref adapter (review-plan.sh) that reuses engine.sh deep-tier routing. pr_diff behavior is untouched and the change is fully covered by new bats tests registered in lint.yml. Triage cleared this as low-risk; confirmation review agrees.

Linked issue analysis

Closes #614. All five ACs are satisfied: (1) plan_json registry row maps to the plan rubric + structured-findings channel (review-registry.tsv, README); (2) review-plan.sh adapter feeds a plan.json into the rubric via run_agentic deep tier, introducing no separate model selector; (3) post-plan-findings.sh emits machine-readable findings JSON for the planner and never posts a GitHub review; (4) pr_diff is untouched — the 'sole type' guard was correctly relaxed to a 'never removed/renamed' guard and additive guards were added; (5) bats coverage (test_plan_review.bats) exercises registry resolution, driver wiring, and output-channel behavior, and is registered in lint.yml's bats list.

Findings

No blocking findings. Observations: (a) Shell scripts use set -euo pipefail with careful quoting and sensible edge-case handling — the EXIT-trap WORK_DIR removal is guarded so the default PLAN_FINDINGS_OUTPUT lands outside WORK_DIR, and the same-file (-ef) case avoids truncation; both are explicitly tested. (b) No security surface: no auth/secrets/credentials, no migrations, no network mutations; the output channel is a local artifact write and statically guarded against calling post-pr-review.sh or the GitHub review API. (c) Static-guard tests rely on grep patterns against the driver; acceptable for wiring locks. CodeRabbit's earlier CHANGES_REQUESTED was resolved and dismissed, ending in APPROVED.

CI status

All required checks green (shellcheck, bats, unit-tests, CodeQL, SonarCloud, gitleaks, AgentShield, validate-fixtures, prompt-coverage, and more). The CANCELLED dev-lead/review dispatch jobs are superseded re-runs; the final review/review check is SUCCESS. dependency-audit ecosystems SKIPPED as expected (no matching manifests).


Reviewed automatically by the PR-review agent (single-reviewer mode: fable 5). Reply if you need a human review.

@don-petry
don-petry merged commit 39f3ead into main Jun 17, 2026
36 of 48 checks passed
@don-petry
don-petry deleted the dev-lead/issue-614-20260615-1249 branch June 17, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Phase 2] Register plan_json artifact type (plan rubric + structured-findings channel)

2 participants