Repository navigation
Route Contribution Check through model routing (cost/economy) - #73837
Conversation
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
There was a problem hiding this comment.
Review summary
No additional actionable issues found in the reviewed source hunk. The generated lock-file changes were not present in the pre-fetched patch, so this review is limited to the workflow frontmatter addition; existing automated comments already cover the routing prompt-injection concern.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~2 AIC · ⌖ ~1 AIC · ⊞ 1.7K
Comment /review to run again
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the orchestrator’s cost-focused economy routing change.
No open issues 🎉
What changed in this PR
Enables cost-focused economy model routing for the Contribution Check orchestrator, addressing #73824 while leaving sub-agent model selections unchanged.
Changes:
- Adds
goal: costandmode: economyto the workflow’s engine configuration. - Regenerates the lock file with a pinned router image, routing prompt preparation, and runtime configuration.
- Exposes the selected model, reasoning effort, and routing status to downstream reporting.
| File | Description |
|---|---|
.github/workflows/contribution-check.md |
Enables cost/economy routing for the orchestrator. |
.github/workflows/contribution-check.lock.yml |
Adds generated routing infrastructure and reporting metadata. |
Review effort: Balanced (auto)
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /codebase-design — this is a scoped, low-risk config change (3-line frontmatter addition + regenerated lock file), so no blocking issues. Approving with context on the pre-existing security-bot flag.
📋 Key Themes & Highlights
Key Themes
- GHAS prompt-injection flag is a known, accepted model-routing limitation, not new risk: the docs (docs/src/content/docs/reference/model-routing.md, "How routing works") explicitly state the classifier "reads the workflow's prompt... and treats that text as data, not instructions." This is the same design already shipped on 20+ routed workflows in this repo (avenger, duplicate-code-detector, design-decision-gate, several smoke tests). Contribution Check is not a special case needing a different mitigation before merge.
- The routed surface here is narrower than the automated comment implies: per the PR description, only the orchestrator is routed — it reads pr-filter-results.json (PR numbers/counts) and contributing-guidelines-truncated.md, not raw PR diffs/bodies. The contribution-checker subagent that actually processes untrusted PR patches stays pinned to claude-haiku-4.5 and is not routed. That mitigation is real, but it isn't spelled out next to the GHAS finding, so a reviewer skimming the diff could assume untrusted PR content flows straight into the classifier. Worth a one-line reply on that thread pointing to the "Unchanged" section of the PR description.
- Reversion of the unrelated SKILL.md recompile diff is confirmed correct — pr-meta.json shows only the two intended files changed (contribution-check.md +3/-0, contribution-check.lock.yml +38/-13), matching the PR description's claim.
Positive Highlights
- ✅ Clear, auditable rollback path stated in the PR description (gh aw audit, compare AIC, revert model-routing block if reports regress).
- ✅ Consistent with the existing repo-wide pattern for goal: cost / mode: economy on low-risk, repetitive triage-style workflows.
- ✅ No fixed-model conflicts introduced — model: claude-haiku-4.5 for the per-PR subagent dispatch prompt is correctly left untouched, since only the orchestrator engine is routed.
Note: sandboxed git remote access failed with a certificate error, so I could not independently re-diff the lock file against the prior committed revision; this review relies on the pre-fetched diff/PR description for the lock-file summary.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · ~115 AIC · ⌖ ~20 AIC · ⊞ 10.7K
Comment /matt to run again
|
🎉 This pull request is included in a new release. Release: |
Contribution Check runs every 4 hours (about 42 runs a week) with no model pin, so Copilot auto-selects
claude-sonnet-5.5at about 33 AIC a run. The routed orchestrator only fans out PRs and compiles verdicts. Economy routing should cut this by up to about 870 AIC a week.Change
.github/workflows/contribution-check.md: addsmodel-routingto theengineblock..github/workflows/contribution-check.lock.yml: regenerated withmake recompile. The diff:GH_AW_MODEL_ROUTING,experimental.modelRoutingin the AWF config,routing.enabled: trueinGH_AW_INFO_MODEL_CONFIG)ghcr.io/github/gh-aw-firewall/router:0.29.0model_effortandmodel_routing_statusfrom the agent job to downstream jobsUnchanged
claude-haiku-4.5, the prompt'smodel: claude-haiku-4.5instruction is kept, andreport-formatterandcomment-dispatcherstay ongpt-6-luna.allowed-models, blocked models ormodel-provider.make recompilealso rewrote.github/skills/agentic-workflows/SKILL.md, dropping its.github/aw/formal-methods.mdentry. That's outside this change, so I reverted it.After merge
gh aw audit <run>(Model Routing section) and compare AIC per run with about 33.mode: balanced. To roll back, removemodel-routing.