Skip to content

Pre-agentic context prefetch for the PR-review cascade: eliminate redundant in-loop gh fetches across the agentic tiers, flag-gated and ET/eval-verified #1101

Open
5 of 8 issues completed

Description

@github-actions

Goal

Restructure the cascading PR-review pipeline so the deterministic PR context (metadata + diff) is gathered ONCE in a shell preflight step and passed to the agentic tiers as pre-populated context, instead of each tier re-fetching it via gh pr view / gh pr diff Bash tool calls inside its own reasoning loop. This eliminates redundant tool round-trips and the reasoning tokens spent deciding to issue them, reducing Effective Tokens (ET) on the highest-volume agent workflow. Motivated by GitHub's May-2026 result (62% ET reduction on comparable agentic CI workflows) and by the Token Cost Observatory (#464).

Grounding correction vs the idea

The triage tier already prefetches metadata + diff at the shell level and runs with NO tools (review-one-pr.sh tier 1, --disallowed-tools). The real remaining opportunity is the four agentic tiers — deep review, rubber-duck, security-audit, single-review — which the tier prompts explicitly instruct to run gh pr view --json <20+ fields> and gh pr diff in-loop (prompts/deep-review.md steps 3-4 and peers). The proven in-repo template is that same triage prefetch (not dev-lead-preflight.sh, which is only a secrets check). Note also: pre-feeding does NOT let us drop the Bash/Read/Grep/Glob tools — those tiers still need them for dynamic cross-file exploration and gh api standards lookups; the saving is the eliminated redundant fetch, not tool-schema removal (MCP is already opt-in / default-off).

Success metric (initiative-level)

Across a fixed A/B sample of real reviews, the deep + security-audit tiers show a measurable, sustained ET reduction vs a frozen pre-change baseline (directional target >=30% on those tiers' input+ET, reported as an exact delta — not a hard gate), with zero regression on the held-out deep-review eval suite (evals/deep-review/holdout/). ET is read from the existing per-call JSONL (scripts/lib/token-metrics.sh, ET = m*(1.0I + 0.1C + 4.0*O)) and aggregated by tier via the Token Cost Observatory (scripts/token_report.sh, #464).

Cost cap

The measurement + validation work is explicitly bounded: the A/B run covers a fixed sample of at most ~30 PRs per arm (on/off), the review-quality check reuses the EXISTING held-out deep-review eval suite (bounded case count, no new eval generation), and no story introduces an unbounded or self-tuning loop. No production default is flipped until the bounded A/B + eval gate pass and a human records a go/no-go.

Shape

Phase 1 audits + freezes a baseline. Phase 2 builds the shared prefetch plumbing (flag-gated, byte-identical when off) and rewires the tier prompts to consume it. Phase 3 adds a HEAD-SHA freshness safeguard against force-push staleness. Phase 4 runs the bounded A/B + eval regression gate and gates the default-on flip on a human decision.

Activation is a human decision — this epic is created inert.


Planned from idea discussion #563 by the BMAD Scrum Master initiative-planner. Inert until a maintainer adds initiative:auto.

Activity

  1. added
    initiative:autoDriver may auto-release this epic's ready sub-issues to dev-lead
    on Jul 4, 2026
  2. don-petry commented on Aug 20, 2026

    @don-petry
    Collaborator

    Initiative driver — stale in-flight slot(s) detected

    The following open sub-issue(s) still carry the dev-lead label but have
    had no dev-lead activity for ≥ 72h, so they are occupying an in-flight
    slot without progressing: #1199, #1200.

    No labels were changed. A human should decide whether to split-and-close, retry,
    or park each one so the slot is freed for the rest of the epic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    initiativeEpic / initiative tracking issueinitiative:autoDriver may auto-release this epic's ready sub-issues to dev-lead

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions