Skip to content

Refresh squad.agent.md model catalog + dynamic catalog discovery #1183

Closed

Description

@ischrei

Problem

The squad.agent.md "Valid models (current platform catalog)" list and the Premium/Standard/Fast fallback chains are hardcoded and have drifted from what GitHub Copilot platforms actually expose. As of 2026-05:

  • Missing from catalog: the entire claude-opus-4.7* family (claude-opus-4.7, claude-opus-4.7-1m-internal, claude-opus-4.7-high, claude-opus-4.7-xhigh) and gpt-5.5.
  • Stale entries: claude-sonnet-4, gpt-5.1, gpt-5.1-codex, gpt-5.1-codex-max, gpt-5.1-codex-mini, gemini-3-pro-preview no longer appear in the CLI task tool model enum.
  • Premium fallback chain head is still claude-opus-4.6, even though claude-opus-4.7 has shipped and is generally stronger.

Result: users who set agentModelOverrides to a 4.7 model in .squad/config.json work fine (free-form passthrough), but the coordinator's own bump-up logic and the default fallback chains route to older models than necessary.

Suggested catalog update (May 2026 snapshot)

Premium:  claude-opus-4.7-1m-internal (Internal), claude-opus-4.7,
          claude-opus-4.7-high (Internal), claude-opus-4.7-xhigh (Internal),
          claude-opus-4.6, claude-opus-4.6-1m (Internal), claude-opus-4.5
Standard: gpt-5.5, claude-sonnet-4.6, claude-sonnet-4.5, gpt-5.4,
          gpt-5.3-codex, gpt-5.2-codex, gpt-5.2
Fast:     claude-haiku-4.5, gpt-5.4-mini, gpt-5-mini, gpt-4.1
Excluded: lark-picker-secondary (router, not a generator)

Suggested fallback chain update

Premium:  claude-opus-4.7 -> claude-opus-4.6 -> claude-opus-4.5 -> claude-sonnet-4.6 -> (omit)
Standard: claude-sonnet-4.6 -> claude-sonnet-4.5 -> gpt-5.5 -> gpt-5.4 -> gpt-5.3-codex -> (omit)
Fast:     claude-haiku-4.5 -> gpt-5.4-mini -> gpt-5-mini -> gpt-4.1 -> (omit)

Notes:

  • Internal-only variants (*-1m-internal, *-high, *-xhigh) should stay in the catalog (so charters can reference them) but should NOT be the chain head — they fail for non-internal accounts and would burn a retry slot.
  • claude-opus-4.5 should drop out of the Designer/Visual default in the role-to-model mapping once 4.7 vision parity is confirmed.

Bigger ask: dynamic catalog discovery (optional follow-up)

The deeper problem is that the catalog will drift again as soon as it ships. Two ideas:

  1. Probe-once-per-session. On first session start after an install, the coordinator probes a small set of candidate model names with a 1-token spawn. Live models populate .squad/config.json under a new discoveredModels field (with a TTL — e.g., 30 days). The hardcoded list in squad.agent.md becomes the seed, not the truth.
  2. Read the platform tool schema. On CLI, the task tool exposes a model enum with the live list. If the SDK can read that enum at session start, we get a free, accurate snapshot with zero spawns. Same applies to runSubagent on VS Code if it ever exposes the equivalent.

Either approach lets users benefit from newly-released models the day they ship, without waiting for a Squad upgrade. It would also auto-prune deprecated entries — today they sit in the fallback chain consuming retries.

A minimal first step: a squad models refresh CLI command that probes and writes discoveredModels to config. Coordinator then unions hardcoded + discovered when picking models, and prefers discovered for the fallback chain head when one is present.

Why this matters

  • Cheaper retries (no fallback attempts on dead models).
  • Stronger defaults (4.7 > 4.6 for premium reasoning, vision, and 1M context use cases).
  • Forward-compatible — new models work without a Squad bump.

Filed from Squad v0.9.6-insider.2 — discovered while reviewing model selection layers in a downstream consumer repo.

Activity

  1. tamirdresher commented on May 27, 2026

    @tamirdresher
    Collaborator

    hey @bradygaster - what do you think on this plan? the catalog drift thing is biting me too on a downstream repo so i'd love to see the dynamic discovery idea move forward.

    between the two options i lean toward #2 (read the task tool's model enum) for the CLI path - its basically free and accurate by definiton, no probing spawns to pay for. option #1 (probe-once) makes sense as the fallback for VS Code / runSubagent where the enum isnt exposed yet.

    one thing not in the issue: if we do go the squad models refresh route, can we also surface what got pruned vs added in the CLI output? would help debugging when a charter suddenly stops resolving a model it used to get.

    happy to take a stab at the seed-vs-truth split in squad.agent.md if nobody else has cycles. lmk.

  2. diberry commented on May 30, 2026

    @diberry
    Contributor

    cc @ischrei @tamirdresher @bradygaster

    A different angle on the ownership question

    The catalog drift problem keeps coming back because Squad is holding model names on behalf of users. A TTL-refresh or dynamic catalog helps, but it's still Squad in the maintenance seat.

    One alternative worth considering: Squad expresses intent (tier + task type) and Copilot CLI resolves that to an actual model at dispatch time. Squad wouldn't need to hold model names at all — not a seed list, not a cache. The mechanism is already noted in this issue (task tool's model enum); the question is just who owns the resolution step.

    For users who want explicit control — pinning a specific model, locking a version for reproducibility — a config override (Layer 0) would still take precedence. That escape hatch keeps power users unaffected while removing the maintenance burden for everyone else.

    This would also quietly resolve the VS Code gap: if Squad doesn't carry model names, there's nothing to lose when runSubagent drops per-spawn model selection.

    The squad models refresh command proposed here would still be worth building, but as a diagnostic tool — "show what Copilot would resolve each tier to right now" — rather than the source of truth.

    Happy to be wrong on any of this; just wanted to raise the ownership framing since it seemed underrepresented in the current options.

  3. primetimetank21 commented on Jul 13, 2026

    @primetimetank21
    Contributor

    From Earl's Squad:

    Tracking what the category-based cost policy + squad models refresh work (currently open for review as PR #1445 against bradygaster/squad) will deliver toward this issue once merged, and what dynamic-discovery scope will remain open.

    PR #1445 (open) proposes to deliver:

    • The squad models refresh command (packages/squad-cli/src/cli/commands/models.ts) — the "minimal first step" the issue requested: reconciles the committed seed catalog against live sources (canonical Copilot models API → public github/docs models-and-pricing.yml fallback → seed-only), with pricing enrichment, algorithmic display-name normalization, loud unpriced-model warnings, and Default-tier price selection (Long context rows skipped).
    • 27 tests in test/models-refresh.test.ts covering all sourcing arms and enrichment edge cases.

    Remaining after #1445 merges:

    • Dynamic-discovery integration — squad models refresh will write only the gitignored diagnostic cache (.squad/.cache/models.json); it will not write discovered models to config with a TTL, and runtime model selection will not union hardcoded + discovered models or prefer discovered models as fallback-chain heads.
    • The static model catalog / fallback-chain refresh (Opus 4.8 + GPT-5.6 IDs) lives in packages/squad-sdk/src/config/models.ts and .squad-templates/model-selection-reference.md (which squad.agent.md defers to) — it was handled by the separately-merged catalog follow-up PR (docs/models: prefer newest per series (Opus 4.8), add GPT-5.6 IDs, fix Ralph free-model wording #1469, merged into dev) and is considered done.

    Leaving this issue open to track the dynamic-discovery integration path.

  4. bradygaster commented on Aug 23, 2026

    @bradygaster
    Collaborator

    Closing — shipped.

    The catalog and fallback chains were refreshed, and have since moved well past the May-2026 snapshot proposed here:

    Current state on dev (packages/squad-sdk/src/config/models.ts):

    premium:  gpt-5.6-sol, claude-opus-5, claude-opus-4.8, claude-opus-4.7, claude-opus-4.6, claude-sonnet-4.6
    standard: gpt-5.6-terra, claude-sonnet-5, claude-sonnet-4.6, gpt-5.5, gpt-5.4, gpt-5.3-codex, claude-sonnet-4.5, gemini-3.1-pro
    fast:     gpt-5.6-luna, claude-haiku-4.5, gpt-5.4-mini, gpt-5-mini
    

    Every specific defect reported here is addressed: the claude-opus-4.7 family and gpt-5.5 are present, the stale entries are pruned, and the premium chain head is no longer claude-opus-4.6.

    Related work that is still open, since this issue is referenced by both:

    Triage: 2026-08-23.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    triagedIssue has been triagedwave:2-soonWave 2 — next 2-4 weeks

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions