Skip to content

Enable new Claude models (Fable 5 / Opus 4.8) in dev-lead + pr-review engines with capability-aware selection #553

Closed

Description

@don-petry

Summary

Wire the new Claude models — Fable 5 (claude-fable-5, new tier above Opus) and Opus 4.8 (claude-opus-4-8) — into the dev-lead and pr-review engines via scripts/engine.sh, with capability-aware, cost-conscious model selection and per-tier fallback. This is the Claude-engine counterpart to #382 (Gemini 3.5) and the implementation of discussion #413.

Why

The Claude engine's highest-capability slots are still pinned to Opus 4.7, with neither Opus 4.8 nor Fable 5 wired in anywhere:

ENGINE_TRIAGE_MODEL = claude-haiku-4-5-20251001
ENGINE_DEEP_MODEL   = claude-sonnet-4-6
ENGINE_AUDIT_MODEL  = claude-opus-4-7      ← top tier
ENGINE_ACTION_MODEL = claude-sonnet-4-6
ENGINE_SINGLE_MODEL = claude-opus-4-7      ← single-reviewer

Opus 4.8 is a drop-in over 4.7 (same request surface, no new breaking changes — model-ID swap + prompt re-tune). Fable 5 raises the capability ceiling for the audit/single-reviewer tier where correctness matters most. We should evaluate the capability/cost tradeoff and enable each where it pays off.

Model reference (verified via the claude-api skill catalog)

Model ID Context Max out $/1M in $/1M out
Claude Fable 5 claude-fable-5 1M 128K $10 $50
Claude Opus 4.8 claude-opus-4-8 1M 128K $5 $25
Claude Sonnet 4.6 claude-sonnet-4-6 1M 64K $3 $15
Claude Haiku 4.5 claude-haiku-4-5 200K 64K $1 $5

API surface notes for Fable 5 / Opus 4.8 (both engines must honor these):

  • Adaptive thinking only — thinking:{type:"adaptive"}; budget_tokens and temperature/top_p/top_k are removed (400 if sent).
  • Thinking depth/spend via output_config.effort (low|medium|high|xhigh|max).
  • Fable 5 only: an explicit thinking:{type:"disabled"} returns 400 — omit the thinking param entirely instead.
  • Min cacheable prefix: Fable 5 = 2048 tok; Opus 4.8 = 4096 tok.

Dependency (blocking)

Proposed approach

  1. Model registry / config — introduce claude-opus-4-8 and claude-fable-5 as configurable per-tier defaults; keep the existing CLAUDE_*_MODEL_CHAIN fallback variables as the safe path (e.g. audit/single → claude-fable-5,claude-opus-4-8,claude-opus-4-7).
  2. Capability-aware selection — route by task type: triage→Haiku, deep/action→Sonnet 4.6/Opus 4.8, audit/single-reviewer→Fable 5 (or Opus 4.8) where correctness > cost. Fail closed (no silent downgrade unless the fallback chain is explicitly enabled).
  3. Request-surface compatibility — ensure both engines emit adaptive thinking + effort, never budget_tokens/sampling params, and omit thinking rather than disabling it on Fable 5.
  4. Observability — log per-run model ID, tier, token usage, and fallback hops (ties into the Token Cost Observatory, 📊 Token Cost Observatory — Weekly Report #464).
  5. Staged rollout — ride the Initiative: Safe Release Strategy for Agentic Workflows (versioning · rings · canary) #495 ring strategy: opt-in → one agent → both. Do not blind-flip the self-hosting agents' top tier.

Deliverables

  • engine.sh per-tier config + fallback chains updated for Fable 5 / Opus 4.8.
  • Capability/cost routing with explicit fail-closed behavior.
  • Adaptive-thinking + effort emitted correctly for the new models (no removed params).
  • shellcheck-clean; smoke test each tier resolves and falls back.
  • Rollout notes + env-var examples.

Acceptance criteria

Related

Per the Release Strategy initiative, Phase 1 is human-driven: the model bump should be cut and promoted by a human through the rings, not auto-rolled.

Activity

  1. github-actions commented on Jun 10, 2026

    @github-actions
    Contributor

    Thanks for the detailed write-up — the tier-by-tier breakdown, API surface notes, and rollout plan make the scope very clear! This is queued for review. One question before scoping the work: since #195 (broken tier routing + rate-limit failover) is listed as a hard prerequisite — without it the new top-tier slots are unreachable dead code — do you intend to land #195 and this issue together in a single PR, or should this issue stay blocked until #195 merges independently?

  2. don-petry commented on Jun 11, 2026

    @don-petry
    CollaboratorAuthor

    Cross-link — ideation output. The feature-ideation workflow (now running in this repo, #555) generated two proposals that align with this issue's scope:

    Canonical discussion remains #413. Both proposals reinforce the capability/cost-aware selection this issue tracks (still gated on #195).

  3. don-petry commented on Jun 11, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #553 — Enable new Claude models (Fable 5 / Opus 4.8) in dev-lead + pr-review engines with capability-aware selection

    Scope

    Wire claude-fable-5 and claude-opus-4-8 into scripts/engine.sh per-tier config and fallback chains, add pricing to model-pricing.tsv, and update the test suite to cover the new model routing.

    Implementation Checklist

    • scripts/engine.sh — update ENGINE_DEEP_MODEL → claude-opus-4-8, ENGINE_AUDIT_MODEL and ENGINE_SINGLE_MODEL → claude-fable-5; update fallback chains; update labels
    • scripts/lib/model-pricing.tsv — add claude-fable-5 pricing row ($10/$1/$12.50/$50 per MTok)
    • tests/dev-lead/unit/test_engine_new_models.bats — new test file (TDD): verify new tier defaults, new chain order, fable-5 fallback behaviour
    • tests/dev-lead/unit/test_engine_chain.bats — update stale tests that hardcode claude-opus-4-7 in chain positions that now use claude-opus-4-8 / claude-fable-5

    Tests to Write (tests/dev-lead/unit/test_engine_new_models.bats)

    • ENGINE_AUDIT_MODEL = claude-fable-5 on claude engine
    • ENGINE_SINGLE_MODEL = claude-fable-5 on claude engine
    • ENGINE_DEEP_MODEL = claude-opus-4-8 on claude engine
    • ENGINE_ACTION_MODEL unchanged (still claude-sonnet-4-6)
    • ENGINE_TRIAGE_MODEL unchanged (still claude-haiku-4-5-20251001)
    • CLAUDE_AUDIT_MODEL_CHAIN starts with claude-fable-5
    • CLAUDE_SINGLE_MODEL_CHAIN starts with claude-fable-5
    • CLAUDE_DEEP_MODEL_CHAIN starts with claude-opus-4-8
    • CLAUDE_ACTION_MODEL_CHAIN second element is claude-opus-4-8 (not opus-4-7)
    • Fallback: fable-5 rate-limited in audit tier → opus-4-8 tried
    • Fallback: audit chain fully exhausted → returns exit 2
    • Fallback: deep tier opus-4-8 rate-limited → sonnet tried
    • model_for_intent("fix-issue") returns claude-opus-4-8
    • model_for_intent("fix-reviews") still returns claude-sonnet-4-6

    Existing Tests to Update

    • test_engine_chain.bats:123 — CLAUDE_ACTION_MODEL_CHAIN fallback now opus-4-8 (not opus-4-7)
    • test_engine_chain.bats:136 — both-rate-limited test references opus-4-7; update to opus-4-8
    • test_engine_chain.bats:160 — deep tier default now opus-4-8; update stub map and assertion
    • test_engine_chain.bats:171 — audit tier default now fable-5; update stub map and assertion
    • test_engine_chain.bats:262 — writer non-RL propagation test; update to opus-4-8 in chain
    • test_engine_chain.bats:313 — "passing tier default model expands to chain" — update to opus-4-8 deep default

    Test Command

    bats tests/dev-lead/unit/test_engine_chain.bats tests/dev-lead/unit/test_engine_new_models.bats tests/dev-lead/unit/test_model_dispatch.bats tests/model_pricing.bats

  4. don-petry commented on Jun 11, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • scripts/engine.sh — ENGINE_DEEP_MODEL → claude-opus-4-8; ENGINE_AUDIT_MODEL / ENGINE_SINGLE_MODEL → claude-fable-5; fallback chains updated; labels updated
    • scripts/lib/model-pricing.tsv — added claude-fable-5* at $10 / $1 / $12.50 / $50 per MTok (effective 2026-06-11); claude-opus-4-8 is covered by existing claude-opus-4-* glob at $5/$25
    • tests/dev-lead/unit/test_engine_new_models.bats — 22 new tests: tier defaults, chain order, fallback behavior, dispatch routing, non-claude engines unchanged
    • tests/dev-lead/unit/test_engine_chain.bats — updated 8 stale tests that hardcoded pre-existing model IDs in chain positions now occupied by opus-4-8 / fable-5; added unset of pre-set chain vars to setup() in both test files

    Tier mapping after this change

    Tier Model Chain
    triage claude-haiku-4-5-20251001 haiku,sonnet-4-6 (unchanged)
    action claude-sonnet-4-6 sonnet-4-6,opus-4-8
    deep claude-opus-4-8 opus-4-8,sonnet-4-6
    audit claude-fable-5 fable-5,opus-4-8,opus-4-7
    single claude-fable-5 fable-5,opus-4-8,opus-4-7

    Env-var override examples

    # Opt individual tier into a specific model (e.g. test on opus-4-8 only for audit):
    CLAUDE_AUDIT_MODEL_CHAIN=claude-opus-4-8 ...
    
    # Full fallback chain for audit, skipping fable-5 for cost control:
    CLAUDE_AUDIT_MODEL_CHAIN=claude-opus-4-8,claude-opus-4-7 ...
    
    # Pin single-reviewer to opus-4-8 for a lower-cost single-reviewer run:
    CLAUDE_SINGLE_MODEL_CHAIN=claude-opus-4-8,claude-opus-4-7 ...

    API surface notes (Fable 5 / Opus 4.8)

    No params were removed — the engine.sh CLI invocations (claude --print --model) have never passed budget_tokens, temperature, top_p, or top_k. The Claude CLI handles adaptive thinking natively for new models. No --thinking disable flag is passed, so Fable 5's 400-on-disabled constraint is already satisfied.

    Test Results

    430 tests, 0 failures  (full dev-lead unit suite)
    13 tests, 0 failures   (model_pricing.bats)
    62 tests, 0 failures   (rate_limit + token_report + batch_fallback)
    shellcheck + agent profile lint: all checks passed
    

    Files Changed

    • scripts/engine.sh: tier models promoted to fable-5 / opus-4-8; fallback chains extended; Fable 5 API constraint comment added
    • scripts/lib/model-pricing.tsv: claude-fable-5* pricing row added
    • tests/dev-lead/unit/test_engine_new_models.bats: new test file (22 tests)
    • tests/dev-lead/unit/test_engine_chain.bats: 8 tests updated to reflect new chain positions; setup() clears pre-set chain env vars

    Notes

  5. added a commit that references this issue on Jun 11, 2026
  6. added 5 commits that reference this issue on Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupenhancementFeature requests

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions