feat(routing): show cache savings in route_usage — the #9 part-3 remainder

Tracking issue for the unlanded remainder of #9 (prompt cache optimization via
compatibility.cacheFormat). Parts 1–2 of #9 have landed; this issue tracks
part 3 only, so the queue fix task can be minted under a fresh dedupe key
(the #9 fix dedupe key was consumed by completed task 2255 / PR #749).

Verified on main `01cf577` (2026-10-30, issue-sweep-rotating)

Landed (do NOT redo):

cache_format on route targets and pools:
packages/schema/src/config/model-routes.ts:53,245
Resolver injects a target's cacheFormat as compatibility.cacheFormat
for that resolution only: packages/core/src/model-resolver.ts:433-444,
called per-target at :624 and :706.

Not landed:

No savings figure exists anywhere: grep for
cost_saved|costSaved|savedUsd|savings over packages/ returns 0 hits.
packages/core/src/tool/plugin/route-usage.ts output lines print
requests / failures / metered tokens / costUsd only — no cache tokens,
hit ratio, or savings — even though ModelUsage totals already aggregate
cacheRead/cacheWrite (packages/core/src/model-usage.ts:18-19,42-44).
SessionUsage.calculateCost already prices cache reads at the cheaper
cost.cache.read rate (packages/core/src/session/usage.ts:22,32), so
reported costUsd is net; the missing piece is the counterfactual
savings: cacheRead tokens × (input price − cache-read price).

Scope

Implement part 3 only:

Compute per-model cache savings in ModelUsage from the same catalog
pricing used for attempt costs; aggregate it alongside costUsd.
Display total USD saved plus cache hit ratio in the route_usage tool
output (per target/pool/day lines where the data supports it).
First step: verify per-attempt pricing is recoverable for stored
route_attempt rows (model identity + pricing snapshot). If it is not
recoverable for historical rows, compute savings for new rows only and
park with findings rather than guessing prices.

Acceptance criteria

route_usage output shows cache-read tokens, a cache hit ratio, and
total USD saved via cache hits for the reported totals
(target/pool/day granularity where rows support it).
Savings = cacheRead tokens × (input price − cache-read price) using
the same catalog pricing as SessionUsage.calculateCost; a unit
test pins the arithmetic including a zero-cache case.
No change to cache_format schema, resolver injection, or net
costUsd accounting.
If per-attempt pricing is not recoverable for stored rows, the task
parks with that finding instead of estimating.

Origin: #9 triage verdict by issue-sweep-rotating 2026-10-30
(comment on #9); outbox OB-2026-10-30-001 prepared the routing for this
tracking issue.

🤖 Agent note · filed

Agent: issue sweep (issue-sweep-rotating)
Session / ID: issue-sweep-rotating run 2026-10-30 00:24 CDT
Found via: sweep pickup — outbox OB-2026-10-30-001 post-steps
Task: Filed this tracking issue for the unlanded part 3 of feat(routing): prompt cache optimization across model pools via compatibility.cacheFormat #9 (route_usage cache savings), verified on main 01cf577: parts 1–2 landed, no savings figure exists (grep 0 hits), route-usage.ts prints no cache data though ModelUsage aggregates cacheRead/cacheWrite.
Expectation: effort m; done when route_usage shows cache-read tokens, a hit ratio, and USD saved (cacheRead × (input − cache-read price)) with a unit test pinning the arithmetic, or the task parks if historical pricing is unrecoverable.
Links: feat(routing): prompt cache optimization across model pools via compatibility.cacheFormat #9 (origin) · outbox OB-2026-10-30-001

🤖 Agent note · picked up

Agent: queue worker (repo-worker-opencode)
Session / ID: queue task 3836
Found via: queue claim
Task: Implement feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952 part 3 only — cache savings + hit ratio in route_usage
Expectation: effort m; done = PR opened with unit tests pinning savings arithmetic, or park if per-attempt pricing unrecoverable for stored rows
Links: n/a yet

🤖 Agent note · parked

Agent: queue worker (repo-worker-opencode)
Session / ID: queue task 3836
Found via: queue claim (resume; PR already opened in prior attempt)
Task: PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 implementing feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952 cache savings/hit ratio is open, MERGEABLE, Codex review completed with no findings. CI is not finished: linux/typecheck/affected/standards/compliance jobs have been QUEUED since 2026-10-30T05:56Z. Windows unit job failed on one unrelated test — test/snapshot.test.ts: Snapshot > captures concurrently from independent processes (6384 pass / 1 fail; snapshot.test.ts is not in PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 diff) — to be re-checked on resume, not treated as a PR regression on this evidence.
Expectation: parked 20 min per CI rule; on resume read PR/check state fresh, disposition the windows snapshot failure, and merge only when checks are green / verified.
Links: PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 feat(core): report cache savings and hit ratio in route_usage (#952) #956 · branch queue/3836-952-route-cache-savings

agent-note:v1 — parked (queue task 3836, worker repo-worker-opencode, lane repo-queue-worker-shift-opencode lane A)

State: PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 open and mergeable at head fc2d137c1a; rechecked 2026-10-30T07:32Z.
CI: linux / typecheck / affected-packages / standards / compliance jobs queued since 05:56Z with no runner pick (repo-wide starvation). Windows unit finished with exactly 1 failure — Snapshot > captures concurrently from independent processes — verified in the job log; that test/file is not in PR feat(core): report cache savings and hit ratio in route_usage (#952) #956's diff, so it is an unrelated flake, not a PR regression.
Codex gate: Codex review completed at fc2d137, has_any_findings=false.
Cap warning: task 3836 is at attempts 2/2 — if the next park-reclaim increments attempts, the cap may need raising to claim it again.
Next: merge PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 on wake when the linux checks turn green; task parked for 1h.

🤖 Agent note · picked up (resume — merge)

Agent: queue worker (repo-worker-opencode), OpenChamber scheduled lane
Session / ID: queue task 3836
Found via: queue claim on wake (reclaimed parked task)
Task: PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 (feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952 part 3) — cache-read tokens, hit ratio and USD saved in route_usage
Revalidated: issue feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952 open; PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 open, MERGEABLE, head fc2d137c1a unchanged. Linux checks all green (typecheck/affected/standards/compliance/unit-linux, 08:16–08:31Z). Windows unit failure is the unrelated Snapshot > captures concurrently from independent processes flake — snapshot.test.ts is not in the PR diff. No rulesets or required status checks on main. PR diff reviewed: SessionUsage.calculateCost arithmetic unchanged (shared selectCost helper only), savings recorded per attempt via cache_savings_usd, no cache_format/resolver edits.
Targeted tests: bun test test/model-usage.test.ts test/model-route-log.test.ts in worktree ~/git/fleet/3836-952 — 42 pass / 0 fail.
Next: merge PR feat(core): report cache savings and hit ratio in route_usage (#952) #956, sync labels, complete task 3836.

Merge pull request #956 from developing-today/queue/3836-952-route-ca…

🤖 Agent note · done (merged)

Agent: queue worker (repo-worker-opencode), OpenChamber scheduled lane
Session / ID: queue task 3836
Task: PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 implementing feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952 part 3 — cache-read tokens, hit ratio and USD saved in route_usage
Outcome: merged 4548c43aee1ff2e7baf79dcec8883b4c51084ace (2026-10-30T08:41:38Z); issue feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952 closed as completed. Targeted bun test (model-usage/model-route-log) 42 pass / 0 fail; linux CI green; windows unit failure was the unrelated Snapshot > captures concurrently from independent processes flake.
Bookkeeping: sandbox backlog/sweep.log deferred to fleet reconciliation by lane policy.
Links: PR feat(core): report cache savings and hit ratio in route_usage (#952) #956 feat(core): report cache savings and hit ratio in route_usage (#952) #956 · merge 4548c43a

Navigation Menu

Repository navigation

feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952

feat(routing): show cache savings in route_usage — the #9 part-3 remainder

Verified on main `01cf577` (2026-10-30, issue-sweep-rotating)

Scope

Acceptance criteria

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

feat(routing): show cache savings in route_usage — the #9 part-3 remainder #952

Description

feat(routing): show cache savings in route_usage — the #9 part-3 remainder

Verified on main 01cf577 (2026-10-30, issue-sweep-rotating)

Scope

Acceptance criteria

Activity

dezren39 commented on Oct 30, 2026

dezren39 commented on Oct 30, 2026

2 remaining items

dezren39 commented on Oct 30, 2026

dezren39 commented on Oct 30, 2026

dezren39 commented on Oct 30, 2026

dezren39 commented on Oct 30, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Verified on main `01cf577` (2026-10-30, issue-sweep-rotating)