Repository navigation
Route nine pinned Codex workflows through GitHub Copilot model routing - #76323
Conversation
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ PR Specification Reviewer completed successfully! PR Specification ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review. Caution agentic threat detected DetailsPotential security threats were detected in the agent output. Detected threat types: prompt injection. Review the workflow run logs for details.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
There was a problem hiding this comment.
Caution
agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.
Details
Potential security threats were detected in the agent output. Detected threat types: prompt injection.
Review the workflow run logs for details.
Reviewed the nine workflow source changes and the available diff. I found no actionable correctness or maintainability issues in the changed lines. The grumpy-coder specialist pass was unavailable because its read attempt was denied by workflow permissions, so this assessment is based on my independent static review; no tests or workflow dispatches were run.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review to run again
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /grill-with-docs, /codebase-design, and /diagnosing-bugs to the nine routing migrations. Commenting, not blocking.
📋 Verification and findings
What I verified
- HEAD
a4f57f5matches the PR head. Rebuilt the compiler from HEAD and recompiled all nine workflows. Each lock matches committed output except the cron scatter line. model-routingvalues (goal,mode) match the schema. All nine frontmatters parse.hosted-webremoval is required:codex_engine.gorejectshostedWebfor non-OpenAI providers, so the workflows would not compile otherwise.- Locks contain the AWF router URL and image, and no
api.openai.comreference.OPENAI_API_KEYappears only in the--exclude-envlist.
Open items
daily-arxiv-researcher: hosted search is off, buttools.web-searchis still declared. Confirm output parity on the PR-branch run.- Inconsistent
mode:spec-enforcerandab-testing-advisorusebalanced; the others useeconomy. Confirm this is intentional. - Inconsistent
copilot-requests: write: some migrated workflows add it and others do not. Document the rule. - No regression test: the PR changes compiler output for nine workflows but adds no golden or compile test.
Limits
I could not diff against the pre-PR base, because base blob fetches fail with an SSL error. Findings rely on HEAD plus the diff.
@copilot please address the review comments above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 6.3K
Comment /matt to run again
There was a problem hiding this comment.
🔵 Needs a closer look
Required runtime routing/provider audits, per-run AIC comparisons, and output-parity checks remain unverified across all nine workflow migrations, particularly for arXiv after hosted web search removal.
No open issues 🎉
What changed in this PR
Migrates nine existing gh-aw Codex workflows from fixed OpenAI model pins to GitHub Copilot cost-based routing, addressing the routing coverage gap described in #76320.
Changes:
- Uses
economyrouting for seven workflows andbalancedforspec-enforcerandab-testing-advisor, without settingallowed-models. - Adds missing Copilot permissions and regenerates locks with router setup, Copilot gateway wiring, and runtime model metadata.
- Removes unsupported hosted-web configuration from
daily-arxiv-researcher, while retaining direct arXiv network access.
PR-branch run audits, cost comparisons, and output-parity verification remain pending, as noted in the description.
| File | Description |
|---|---|
.github/workflows/windows-grower.md |
Replaces the fixed model with Copilot economy routing. |
.github/workflows/windows-grower.lock.yml |
Generates Copilot routing, credentials, and runtime metadata wiring. |
.github/workflows/spec-enforcer.md |
Enables balanced routing and adds Copilot permission. |
.github/workflows/issue-arborist.md |
Replaces the fixed model with Copilot economy routing. |
.github/workflows/gpclean.md |
Enables economy routing and adds Copilot permission. |
.github/workflows/feature-grower.md |
Replaces the fixed model with Copilot economy routing. |
.github/workflows/feature-grower.lock.yml |
Generates economy routing and removes OpenAI credential wiring. |
.github/workflows/daily-storify.md |
Replaces the fixed model with Copilot economy routing. |
.github/workflows/daily-fact.md |
Enables economy routing and adds Copilot permission. |
.github/workflows/daily-arxiv-researcher.md |
Enables economy routing and removes unsupported hosted-web configuration. |
.github/workflows/ab-testing-advisor.md |
Replaces the fixed model with Copilot balanced routing. |
.github/workflows/ab-testing-advisor.lock.yml |
Generates balanced routing and Copilot-based threat detection. |
Review effort: Max (auto)
|
🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "b150cbceab155a445b01348ff9dd7851fef5f8a4",
"kind": "requested",
"schema_version": "1",
"state": "393f104d43bbc85a3385eb423d097bf2d49d9e4e2887bb7760d1019cba06d39c",
"work": [],
"workflow": "pr-sous-chef"
}
|
…rkflows-to-copilot # Conflicts: # .github/workflows/ab-testing-advisor.lock.yml # .github/workflows/daily-arxiv-researcher.lock.yml # .github/workflows/daily-storify.lock.yml # .github/workflows/feature-grower.lock.yml # .github/workflows/gpclean.lock.yml # .github/workflows/issue-arborist.lock.yml # .github/workflows/spec-enforcer.lock.yml # .github/workflows/windows-grower.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Merged the latest |
|
🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "1308aeee7da95a9cec3986fa4eb0d3eaff34d259",
"kind": "requested",
"schema_version": "1",
"state": "dee35d9940944cb1e9fe77374612f80a75899a148bb2bb38c82ccfe2dc3cca79",
"work": [],
"workflow": "pr-sous-chef"
}
|
…rkflows-to-copilot # Conflicts: # .github/workflows/ab-testing-advisor.lock.yml # .github/workflows/ab-testing-advisor.md Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Merged the latest |
|
🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "4248307cb1bf7b0da1dc6079fb4a3d3e7c95b3ee",
"kind": "requested",
"schema_version": "1",
"state": "1e91ea311322b70e78f7872dd8d9d8aa1b11e4c31f178c8081833c9054f6735b",
"work": [],
"workflow": "pr-sous-chef"
}
|
…rkflows-to-copilot # Conflicts: # .github/workflows/spec-enforcer.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Merged the latest |
|
🎉 This pull request is included in a new release. Release: |
Nine Codex workflows are pinned to fixed OpenAI API models. AWF's native OpenAI routing has routing metadata only for the
gpt-5.4andgpt-5.4-minifamilies, so these pins can't be routed natively. This PR moves them to Copilot-provider routing withgoal: cost.Source changes
copilot-requests: writedaily-arxiv-researcheropenai/gpt-5.6-soldaily-storifyopenai/gpt-6-solspec-enforceropenai/gpt-5.6-solgpcleanopenai/gpt-6.1-solissue-arboristopenai/gpt-6-solab-testing-advisoropenai/gpt-6.1-soldaily-factopenai/gpt-5.6-terrawindows-groweropenai/gpt-5.6-terrafeature-groweropenai/gpt-5.6-terraallowed-modelsis not set. Prompts, triggers, tools, safe outputs and evals are unchanged. Only these nine sources and their nine regenerated locks are in the diff.Deviation from the issue:
daily-arxiv-researcherlosesnetwork.hosted-webnetwork.hosted-webfor Codex on Copilot inference (pkg/workflow/codex_engine.go: "not functional with Codex using Copilot inference"). The block (an allowlist forarxiv.organdexport.arxiv.org) had to be removed for the workflow to compile.hostedWeb.codex.enabled: false).curlthroughnetwork.allowed, which still lists both domains.tools.web-searchandtools.web-fetchare unchanged.Lock changes (same in all nine)
apiProxy.routing.objectiveset to{goal: cost, mode}andcandidateModels: ["github-copilot/gpt-*"].gh-aw-firewall/router:0.29.10image.openai-proxybase_urlnow points at the Copilot gateway172.30.0.30:10002instead of OpenAI's:10000.--model "$GH_AW_MODEL_AGENT_CODEX"agent pin.OPENAI_API_KEYreferences (secret validation, agent and detection env, redaction lists).api.openai.comallow entries.CODEX_API_KEYwiring.--exclude-env OPENAI_API_KEY, which keeps any OpenAI key out of the agent container.CODEX_API_KEY="$COPILOT_DUMMY_BYOK", a placeholder; the AWF api-proxy adds the Copilot token.openai/*patterns in the sharedapiProxy.modelsalias table. Every workflow carries this table; it doesn't select a model.Notes for reviewers
main:ab-testing-advisorhas failed in activation every day since at least 2 Nov, at the daily-AIC quota check (quota_exhausted, e.g. run 42600855561). It is unrelated to this PR.make agent-finish: twopkg/clilog-download timing tests,TestDownloadWorkflowLogsForTargetsConcurrentAndResilientandTestDownloadWorkflowLogsForTargetsUsesOneWallClockTimeout, timed out under full-suite load. Both pass alone onmainand on this branch.