Skip to content

Route nine pinned Codex workflows through GitHub Copilot model routing - #76323

Merged
SivaKesava1 merged 12 commits into
mainfrom
copilot/move-codex-workflows-to-copilot
Nov 10, 2026
Merged

SivaKesava1 merged 12 commits into
mainfrom
copilot/move-codex-workflows-to-copilot

Conversation

Copilot AI commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Nine Codex workflows are pinned to fixed OpenAI API models. AWF's native OpenAI routing has routing metadata only for the gpt-5.4 and gpt-5.4-mini families, so these pins can't be routed natively. This PR moves them to Copilot-provider routing with goal: cost.

Source changes

Workflow Removed pin Mode Added copilot-requests: write
daily-arxiv-researcher openai/gpt-5.6-sol economy
daily-storify openai/gpt-6-sol economy
spec-enforcer openai/gpt-5.6-sol balanced ✓
gpclean openai/gpt-6.1-sol economy ✓
issue-arborist openai/gpt-6-sol economy
ab-testing-advisor openai/gpt-6.1-sol balanced
daily-fact openai/gpt-5.6-terra economy ✓
windows-grower openai/gpt-5.6-terra economy
feature-grower openai/gpt-5.6-terra economy
engine:
  id: codex
  model-provider: github
  model-routing:
    goal: cost
    mode: economy   # balanced for spec-enforcer / ab-testing-advisor

allowed-models is not set. Prompts, triggers, tools, safe outputs and evals are unchanged. Only these nine sources and their nine regenerated locks are in the diff.

Deviation from the issue: daily-arxiv-researcher loses network.hosted-web

  • The compiler rejects network.hosted-web for Codex on Copilot inference (pkg/workflow/codex_engine.go: "not functional with Codex using Copilot inference"). The block (an allowlist for arxiv.org and export.arxiv.org) had to be removed for the workflow to compile.
  • Effect: provider-hosted web search is off for this workflow (hostedWeb.codex.enabled: false).
  • The prompt never asks for hosted search. Papers come from the pre-agent fetch step, and PDFs from curl through network.allowed, which still lists both domains. tools.web-search and tools.web-fetch are unchanged.
  • Please confirm output parity in this workflow's PR-branch run.

Lock changes (same in all nine)

  • Added:
    • apiProxy.routing.objective set to {goal: cost, mode} and candidateModels: ["github-copilot/gpt-*"].
    • The Prepare model-routing conversation step and the gh-aw-firewall/router:0.29.10 image.
  • Changed: Codex's openai-proxy base_url now points at the Copilot gateway 172.30.0.30:10002 instead of OpenAI's :10000.
  • Removed:
    • The --model "$GH_AW_MODEL_AGENT_CODEX" agent pin.
    • 16 of 17 OPENAI_API_KEY references (secret validation, agent and detection env, redaction lists).
    • Both api.openai.com allow entries.
    • The real-key CODEX_API_KEY wiring.
  • Remaining, not used for agent inference:
    • --exclude-env OPENAI_API_KEY, which keeps any OpenAI key out of the agent container.
    • CODEX_API_KEY="$COPILOT_DUMMY_BYOK", a placeholder; the AWF api-proxy adds the Copilot token.
    • openai/* patterns in the shared apiProxy.models alias table. Every workflow carries this table; it doesn't select a model.

Notes for reviewers

  • Not yet done: PR-branch dispatch runs and audits (issue cases B and C) are left to maintainers.
  • Known failure on main: ab-testing-advisor has failed in activation every day since at least 2 Nov, at the daily-AIC quota check (quota_exhausted, e.g. run 42600855561). It is unrelated to this PR.
  • Flaky tests in make agent-finish: two pkg/cli log-download timing tests, TestDownloadWorkflowLogsForTargetsConcurrentAndResilient and TestDownloadWorkflowLogsForTargetsUsesOneWallClockTimeout, timed out under full-suite load. Both pass alone on main and on this branch.

Copilot AI and others added 2 commits November 9, 2026 22:34
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
Copilot AI changed the title [WIP] Move Codex workflows to GitHub Copilot model routing Route nine pinned Codex workflows through GitHub Copilot model routing Nov 9, 2026
Copilot AI requested a review from SivaKesava1 November 9, 2026 23:05
@SivaKesava1
SivaKesava1 marked this pull request as ready for review November 9, 2026 23:06
Copilot AI dynamic review requested due to automatic review settings November 9, 2026 23:06
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

PR Specification Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from PR Specification Reviewer · gpt56 · ~4 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 1.6K · ◷

Generated by PR Specification Reviewer for #76323

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output. Detected threat types: prompt injection.

Review the workflow run logs for details.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt61 medium · ~5 AIC · ⌖ ~1 AIC · ◇ ~1 AIC · ⊞ 181 · ◷

Generated by Ponytail Reviewer for #76323

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · routed: haiku55 xhigh · ~1 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 8.5K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: haiku55 xhigh · ~1 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 7.2K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.

Details

Potential security threats were detected in the agent output. Detected threat types: prompt injection.

Review the workflow run logs for details.

Reviewed the nine workflow source changes and the available diff. I found no actionable correctness or maintainability issues in the changed lines. The grumpy-coder specialist pass was unavailable because its read attempt was denied by workflow permissions, so this assessment is based on my independent static review; no tests or workflow dispatches were run.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /grill-with-docs, /codebase-design, and /diagnosing-bugs to the nine routing migrations. Commenting, not blocking.

📋 Verification and findings

What I verified

  • HEAD a4f57f5 matches the PR head. Rebuilt the compiler from HEAD and recompiled all nine workflows. Each lock matches committed output except the cron scatter line.
  • model-routing values (goal, mode) match the schema. All nine frontmatters parse.
  • hosted-web removal is required: codex_engine.go rejects hostedWeb for non-OpenAI providers, so the workflows would not compile otherwise.
  • Locks contain the AWF router URL and image, and no api.openai.com reference. OPENAI_API_KEY appears only in the --exclude-env list.

Open items

  1. daily-arxiv-researcher: hosted search is off, but tools.web-search is still declared. Confirm output parity on the PR-branch run.
  2. Inconsistent mode: spec-enforcer and ab-testing-advisor use balanced; the others use economy. Confirm this is intentional.
  3. Inconsistent copilot-requests: write: some migrated workflows add it and others do not. Document the rule.
  4. No regression test: the PR changes compiler output for nine workflows but adds no golden or compile test.

Limits

I could not diff against the pre-PR base, because base blob fetches fail with an SSL error. Findings rely on HEAD plus the diff.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~8 AIC · ⌖ ~2 AIC · ⊞ 6.3K
Comment /matt to run again

Comment thread .github/workflows/spec-enforcer.md Resolved
Comment thread .github/workflows/daily-fact.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Required runtime routing/provider audits, per-run AIC comparisons, and output-parity checks remain unverified across all nine workflow migrations, particularly for arXiv after hosted web search removal.

No open issues 🎉

What changed in this PR

Migrates nine existing gh-aw Codex workflows from fixed OpenAI model pins to GitHub Copilot cost-based routing, addressing the routing coverage gap described in #76320.

Changes:

  • Uses economy routing for seven workflows and balanced for spec-enforcer and ab-testing-advisor, without setting allowed-models.
  • Adds missing Copilot permissions and regenerates locks with router setup, Copilot gateway wiring, and runtime model metadata.
  • Removes unsupported hosted-web configuration from daily-arxiv-researcher, while retaining direct arXiv network access.

PR-branch run audits, cost comparisons, and output-parity verification remain pending, as noted in the description.

File Description
.github/​workflows/​windows-grower.md Replaces the fixed model with Copilot economy routing.
.github/​workflows/​windows-grower.lock.yml Generates Copilot routing, credentials, and runtime metadata wiring.
.github/​workflows/​spec-enforcer.md Enables balanced routing and adds Copilot permission.
.github/​workflows/​issue-arborist.md Replaces the fixed model with Copilot economy routing.
.github/​workflows/​gpclean.md Enables economy routing and adds Copilot permission.
.github/​workflows/​feature-grower.md Replaces the fixed model with Copilot economy routing.
.github/​workflows/​feature-grower.lock.yml Generates economy routing and removes OpenAI credential wiring.
.github/​workflows/​daily-storify.md Replaces the fixed model with Copilot economy routing.
.github/​workflows/​daily-fact.md Enables economy routing and adds Copilot permission.
.github/​workflows/​daily-arxiv-researcher.md Enables economy routing and removes unsupported hosted-web configuration.
.github/​workflows/​ab-testing-advisor.md Replaces the fixed model with Copilot balanced routing.
.github/​workflows/​ab-testing-advisor.lock.yml Generates balanced routing and Copilot-based threat detection.

Review effort: Max (auto)

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "b150cbceab155a445b01348ff9dd7851fef5f8a4",
  "kind": "requested",
  "schema_version": "1",
  "state": "393f104d43bbc85a3385eb423d097bf2d49d9e4e2887bb7760d1019cba06d39c",
  "work": [],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~2 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

…rkflows-to-copilot

# Conflicts:
#	.github/workflows/ab-testing-advisor.lock.yml
#	.github/workflows/daily-arxiv-researcher.lock.yml
#	.github/workflows/daily-storify.lock.yml
#	.github/workflows/feature-grower.lock.yml
#	.github/workflows/gpclean.lock.yml
#	.github/workflows/issue-arborist.lock.yml
#	.github/workflows/spec-enforcer.lock.yml
#	.github/workflows/windows-grower.lock.yml

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:...

Merged the latest main and regenerated the eight conflicted workflow lockfiles. Merge commit: 1308aee. Build and recompilation passed; no CI was triggered.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "1308aeee7da95a9cec3986fa4eb0d3eaff34d259",
  "kind": "requested",
  "schema_version": "1",
  "state": "dee35d9940944cb1e9fe77374612f80a75899a148bb2bb38c82ccfe2dc3cca79",
  "work": [],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~2 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

…rkflows-to-copilot

# Conflicts:
#	.github/workflows/ab-testing-advisor.lock.yml
#	.github/workflows/ab-testing-advisor.md

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:...

Merged the latest main and regenerated the conflicted workflow lock. Merge commit: 4248307c. Build, workflow recompilation, and impacted validation passed; I did not trigger CI. CodeQL timed out and could not be retried.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "4248307cb1bf7b0da1dc6079fb4a3d3e7c95b3ee",
  "kind": "requested",
  "schema_version": "1",
  "state": "1e91ea311322b70e78f7872dd8d9d8aa1b11e4c31f178c8081833c9054f6735b",
  "work": [],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~3 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

…rkflows-to-copilot

# Conflicts:
#	.github/workflows/spec-enforcer.lock.yml

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Nov 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:...

Merged the latest main and regenerated the conflicted spec-enforcer lock from its merged source. Merge commit: 50af9d82.

@SivaKesava1
SivaKesava1 merged commit dd77f72 into main Nov 10, 2026
24 checks passed
@SivaKesava1
SivaKesava1 deleted the copilot/move-codex-workflows-to-copilot branch November 10, 2026 21:24
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Route nine pinned Codex workflows through GitHub Copilot model routing

5 participants