Skip to content

fix: Gemini fallback unavailable when Claude is rate-limited (no CLI / no GOOGLE_API_KEY) #146

Closed

Description

@don-petry

Parent issue

#145

Problem

When Claude hits a rate limit during a review run, the workflow is designed to fall back to the Gemini engine. In run #715 this fallback silently failed:

##[warning]Claude rate limit hit but Gemini fallback unavailable
(CLI not installed or GOOGLE_API_KEY missing) — falling through to Copilot

The warning is emitted and the session continues to the Copilot engine, but:

  • There is no validation step at workflow startup confirming Gemini is reachable.
  • The GOOGLE_API_KEY secret is not set (or not forwarded) in the runner environment, so every Claude exhaustion event silently skips Gemini and goes straight to Copilot — which may also be rate-limited (as happened here).
  • Operators have no early signal that the Gemini buffer is non-functional.

Acceptance criteria

  1. Pre-flight check: at workflow startup (before processing any PR), the script validates that the Gemini fallback is configured (CLI present + GOOGLE_API_KEY non-empty). If not, emit a ##[warning] and continue, but record the degraded state.
  2. Job summary: the run summary clearly states whether each engine (Claude / Gemini / Copilot) was available at startup.
  3. Secret documented: GOOGLE_API_KEY is added to the workflow's env block (or secrets) with a comment explaining its role. Repo/org secret must be configured.
  4. No silent skip: if Gemini CLI is missing, the warning must include the exact install command so operators can fix it without reading docs.
  5. Existing behaviour when Gemini IS available is unchanged.

Expected tests

  • Unit (bash/bats or shellcheck): validate_engines() helper returns correct degraded-state flag when GOOGLE_API_KEY is unset.
  • Unit: validate_engines() returns healthy when CLI is present and key is set.
  • Integration (dry-run): workflow invoked with GOOGLE_API_KEY unset emits the expected ##[warning] line in job output.
  • Integration (dry-run): job summary lists gemini: unavailable when key is absent and gemini: ok when present.

Activity

  1. don-petry commented on May 12, 2026

    @don-petry
    CollaboratorAuthor

    Claude finished @don-petry's task in 9m 24s —— View job • claude/issue-146-20260512-1802 • Create PR ➔


    Implementation complete

    • Read CLAUDE.md and explore repository structure
    • Find the relevant workflow and scripts
    • Create scripts/validate-engines.sh with validate_engines() helper
    • Modify scripts/review-batch.sh — pre-flight call + use GEMINI_AVAILABLE in fallback
    • Write 8 unit tests in tests/test-validate-engines.sh
    • Commit and push changes
    • Create PR, self-review, and tag code owners — CI is green ✅

    What was done

    Root cause: review-batch.sh only checked Gemini availability at the rate-limit fallback point (too late), with no install hint in the warning.

    New scripts/validate-engines.sh — validate_engines() runs before any PR is processed:

    • Checks Claude, Gemini, Copilot availability; exports CLAUDE_AVAILABLE / GEMINI_AVAILABLE / COPILOT_AVAILABLE
    • Emits ::warning:: for each unavailable engine with the exact fix command (e.g. npm install -g @google/gemini-cli)
    • Appends an engine-availability Markdown table to $GITHUB_STEP_SUMMARY

    scripts/review-batch.sh — pre-flight wired in at the top; the Claude→Gemini fallback now uses $GEMINI_AVAILABLE (set at startup) so the availability logic is evaluated once and reused throughout the run.

    tests/test-validate-engines.sh — 8 unit tests covering GEMINI_AVAILABLE=false when key unset, GEMINI_AVAILABLE=false when CLI absent, GEMINI_AVAILABLE=true when both present, warning annotation content (install command), and job-summary format (Gemini | ok / Gemini | unavailable).

    Note: GOOGLE_API_KEY was already documented in pr-review.yml:62 with a comment — no workflow file change required. The repo secret still needs to be configured in org/repo settings for the Gemini fallback to become functional.

    View PR #150 |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reports

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions