Repository navigation
fix(pr-review): detect provider usage-cap and fall back to next engine - #107
Conversation
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request enhances the rate-limit and usage-cap detection logic by expanding the regex patterns in is_rate_limited to cover subscription limits, billing caps, and specific Claude usage messages. It also updates the triage process to check both stdout and stderr for these patterns and improves error logging. A review comment correctly identifies a risk of false positives due to the broader regex patterns and suggests guarding the detection logic with a check for a non-zero exit code to ensure it only triggers when the command actually fails.
There was a problem hiding this comment.
Pull request overview
Improves the PR-review engine cascade by detecting provider “usage cap”/billing-limit style failures (not just classic API rate limits) so the workflow can exit with code 2 and fall back to the next engine instead of hard-failing the session.
Changes:
- Expand
is_rate_limitedmatching to include subscription/billing-cap phrasing plus HTTP 402 and 529 signals. - In triage, detect limit messages from both stdout and stderr, and print both channels when triage hard-fails for better CI diagnostics.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| scripts/review-one-pr.sh | Adds dual-channel (stdout+stderr) limit detection and prints both streams on triage hard-fail. |
| scripts/engine.sh | Broadens is_rate_limited regex to detect usage-cap/billing-limit/service-overload blocks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
2e5a348 to
1c49038
Compare
When Claude Code hits a subscription/token usage cap it exits with code 1 and writes a usage-limit message to stdout. The previous is_rate_limited regex only covered API-level rate-limit phrasing (429, "too many requests", etc.) and missed Claude Code's billing/plan-cap messages — so the cascade hard-failed instead of falling back through the existing gemini → copilot chain in review-batch.sh. Changes: - engine.sh: extend is_rate_limited to cover subscription/billing caps (out of tokens, claude.*usage, plan/billing/subscription limit, HTTP 402 payment-required, 529 overloaded). - review-one-pr.sh: check $TRIAGE_LOG (stderr) as well as $TRIAGE_RESULT (stdout) — some providers write their limit message to stderr. - review-one-pr.sh: surface both stdout and stderr in the hard-fail path so the actual error message is always visible in CI logs. Fixes #106 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove claude.*usage / usage.*claude (already covered by 'usage limit' substring match) and daily/monthly.*limit (too broad, false-positive risk on PR diffs). Condense 6-line comment to 3 lines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR description listed these patterns but they were absent from the regex. Adds claude.*usage, usage.*claude, daily.*limit, monthly.*limit to ensure Claude Code billing-cap messages are actually matched. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
dac63ef to
80d184d
Compare
The single-review prompt documents 'skip' as a valid return value for draft PRs and head-SHA mismatches. post-pr-review.sh only handled 'approve' and 'escalate', causing the review session to hard-fail (exit 1) rather than gracefully no-op (exit 100) when the reviewer returned skip. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
donpetry-bot
left a comment
There was a problem hiding this comment.
Automated review — APPROVED ✓
Risk: LOW
Reviewed commit: b5eedbdca7a116e1f4bb37174277468170464497
Review mode: triage-approved (single reviewer)
Summary
Extends is_rate_limited() in engine.sh to detect provider subscription/billing caps (plan limits, out-of-tokens, HTTP 402/529) so the engine fallback chain fires instead of hard-failing the session. Also adds a missing skip decision handler in post-pr-review.sh (exit 100 for draft PRs / SHA mismatches). Both changes are additive, safe, and well-scoped.
Linked issue analysis
Closes #106. The primary root cause (incomplete is_rate_limited regex missing Claude Code billing-cap phrasing) is directly addressed by the regex expansion. The skip handler fixes a secondary issue discovered during the same investigation.
Note: The PR description references changes to review-one-pr.sh (dual-channel stderr detection, improved diagnostic output) that are not present in the final diff — those changes were in intermediate commits but appear to have been reverted. Root causes #2 and #3 from the issue (stderr not checked, stdout swallowed on hard-fail) remain open. Consider a follow-up if those enhancements are still desired.
Findings
- No security concerns. Changes are limited to regex pattern matching and a conditional branch — no auth, secrets, eval, or injection surface.
- Regex patterns are reasonable. The new patterns (
out of.*token,plan.*limit,billing.*limit, HTTP402/529, etc.) are specific enough to avoid false positives in the rate-limit detection context, especially since the function is called on provider output text after a command failure. skiphandler is correct. Placed before the existing unknown-decision validation, extracts the reason from JSON, and exits with the expected code 100.- All prior review comments resolved. Gemini and Copilot both flagged the false-positive risk from broad patterns on successful triage output — the author confirmed this was gated on non-zero exit codes (though that specific change isn't in the final diff since
review-one-pr.shchanges were reverted).
CI status
| Check | Status |
|---|---|
| CodeQL (actions) | ✅ SUCCESS |
| SonarCloud | ✅ SUCCESS (0 new issues, 0 security hotspots) |
| AgentShield | ✅ SUCCESS |
| Dependency audit | ✅ SUCCESS/SKIPPED (no applicable ecosystems) |
| CodeRabbit | ✅ SUCCESS |
All checks passed.
Reviewed automatically by the PR-review agent (single-reviewer). Reply if you need a human review.
|
@claude[agent] - update base branch |



Problem
Closes #106
When Claude hits its subscription/token usage cap,
claude --printexits with code 1 and writes a usage-cap message to stdout. Theis_rate_limitedfunction only matched API-level rate-limit phrasing (429,too many requests, etc.) — it missed Claude Code's billing/plan-cap messages entirely.Result: the cascade hard-failed (exit 1) and aborted the session, instead of exiting 2 and triggering the existing gemini → copilot fallback chain in
review-batch.sh.Observed failure: https://github.com/petry-projects/.github-private/actions/runs/25648497571/job/75282153268
Changes
scripts/engine.sh— extendis_rate_limitedregexAdded three new failure-mode categories to the pattern:
out of.*token,token.*exhaust,claude.*usage,usage.*claude,plan.*limit,subscription.*limit,billing.*limit,daily.*limit,monthly.*limit402529scripts/review-one-pr.sh— dual-channel rate-limit detection + diagnosticsCheck stderr too:
$TRIAGE_LOG(stderr) is now read into$TRIAGE_STDERRand passed throughis_rate_limitedalongside$TRIAGE_RESULT(stdout). Some providers write their limit message to stderr.Surface the error on hard-fail: the previous failure path only printed stderr via
cat $TRIAGE_LOG, silently discarding stdout — where Claude Code actually writes its error. Both channels are now printed so the failure reason is always visible in CI logs.Behaviour after this fix