Skip to content

Model routing: route Designer Drift Audit, Daily Documentation Diagram and Chaos PR Bundle Fuzzer on OpenAI instead of pinning GPT-6 Astra, GPT-5.6 Sol and GPT-5.6 Terra #74866

Closed

Description

@rabeckett

Estimated saving: about 750-850 AIC/week. These three Codex workflows run on the OpenAI key (model-provider: openai) and pin a model. AWF also routes natively on OpenAI (smoke-codex-routed-openai), so route them with economy and keep the OpenAI provider and key.

Workflow Output Now Mode Agent runs/wk AIC/run now Est. saving/wk
designer-drift-audit issue (drift between docs and the designer skill) openai/gpt-6-astra economy 5 ~122 440-490
daily-documentation-diagram PR (one docs diagram) openai/gpt-5.6-sol economy 7 ~66 300-340
chaos-pr-bundle-fuzzer draft test PRs (expire in 4h) openai/gpt-5.6-terra economy ~1 ~32 20-35

AIC/run includes threat detection.

Why

  • OpenAI routing is ready. smoke-codex-routed-openai has passed every run since 25 October. It limits candidates to gpt-5.4, but in its runs:

    • AWF's OpenAI catalogue lists all ten OpenAI models the router ranks (GPT-5.4, 5.4-mini and 5.5; GPT-5.6 Luna, Sol and Terra; GPT-6 Astra, Luna and Sol; and GPT-6.1 Sol).
    • Classification wasn't degraded.
    • The detection job resolved detection to gpt-5-mini.

    These three are the first real workflows to route on OpenAI. None has sub-agents, and each runs at most once a day, so a failed routed run costs one report, not a burst of runs.

  • Routed runs cost much less. On Copilot, routing has cut cost per run for the same pins. GPT-6 Astra to routed: Auto-Triage Issues 79 to 20 AIC. GPT-5.6 Sol: Terminal Stylist 106 to 35. Terra: Daily Go Test Parallelizer 55 to 11-33. The router uses the same per-model costs on OpenAI.

  • economy for all three. Designer Drift Audit files an issue, Daily Documentation Diagram opens a PR with one diagram that a maintainer reviews, and Chaos PR Bundle Fuzzer opens draft test PRs confined to its chaos folders. For this work, economy picks GPT-5.6 Luna (xhigh or max). A hard cross-system label picks gpt-5.6-sol:max, which still costs less than GPT-6 Astra.

  • Detection gets cheaper. It inherits today's pin: about 29 AIC per run for Designer Drift Audit, 11 for Daily Documentation Diagram and 6 for Chaos PR Bundle Fuzzer. Routed, it uses gpt-5-mini, about 0.5.

  • Not tied to one model. Copilot's 16 November retirement of GPT-5.6 Terra doesn't cover openai/ pins, but routing removes the dependency on a single model name anyway. No model restrictions.

Change

In .github/workflows/designer-drift-audit.md, daily-documentation-diagram.md and chaos-pr-bundle-fuzzer.md, delete the top-level model: line and add model-routing to the existing engine block, keeping model-provider: openai:

engine:
  id: codex
  model-provider: openai
  model-routing:
    goal: cost
    mode: economy

No permission changes: inference stays on the OpenAI key. Keep the prompts, steps and safe outputs. Don't add model restrictions.

Run gh aw compile designer-drift-audit daily-documentation-diagram chaos-pr-bundle-fuzzer. On main (fdfc5d5a0d) these compile with no new warnings: routing is enabled with "provider":"openai" and gpt-* candidates, and detection uses the detection model.

Check

  • Audit the first routed run of each with gh aw audit <run>. The selection in usage/aw_session.jsonl should be an OpenAI Luna model. If routing fails on OpenAI (the run fails at AWF startup, or the selection is rejected), restore the three pins and file the failure in gh-aw-firewall.
  • Compare AIC per run with the table. If Daily Documentation Diagram keeps getting gpt-5.6-sol:max and costs more than about 66 AIC per run, restore its pin.
  • Review Daily Documentation Diagram's next PRs.

Estimates use gh-aw usage from 19 October to 3 November and AWF v0.29.6's routing tables; the routed cost per run is estimated from routed Copilot workflows on the same pins.

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions