Repository navigation
feat(runtime): trace per-request provider usage and cache attribution #1268
Description
Problem
Maka cannot explain a provider cache miss at request level.
request-shape.ts computes hashes for the system prompt, provider options, tool schema, and history. The trace mainly persists a turn-start snapshot, while same-turn tool-loop hashes are computed but not retained as request records.
Usage has a similar gap. recordLlmCall records the aggregate after the send and tool loop, not one row per provider request or retry. A trace cannot reliably connect cache usage to the request that produced it.
In the Terminal-Bench v11 comparison in #1217, Kimi Code had provider-proxy telemetry for each request. Maka exposed cumulative cell usage. The Maka trace shows a stable turn-level prefix, but it cannot identify which request missed or which request segment changed.
#615 already proposes capturing the exact prepared request. This issue should add usage and cache telemetry to that capture path, not create another request snapshot.
Desired outcome
Record one telemetry row for every physical provider request attempt.
The row needs:
- turn, step, attempt, and request-capture reference;
- input, cache-read, cache-miss, cache-write, output, and reasoning tokens;
- whether cache fields came from the provider or were derived;
- status, finish reason, latency, and time to first token when available;
- hashes and sizes for the prepared request segments.
Segment comparison can derive the first changed cacheable segment. It does not need another persisted classification field.
Acceptance criteria
- Tool-loop requests, retries, failures, and interruptions have separate records.
- Each usage record links to the request captured by fix(runtime): RuntimeEvent ledger is recovery-grade (WAL), not source-of-truth — write-path truncation blocks counterfactual replay #615.
- Provider-reported and derived cache values remain distinguishable.
- Missing provider usage remains missing rather than being estimated from partial data.
- Complete per-request usage reconciles with the existing aggregate.
- Request detail lives in RunTrace or referenced artifacts; RuntimeEvent keeps only the aggregate and reference needed for replay.
- Tests cover Anthropic-compatible and OpenAI-compatible multi-step requests.
- Harbor exports can identify the first changed request segment without a provider proxy.
Activity
Nyvo-io commentedon Jul 20, 2026
I would like to claim #1268: per-request provider usage and cache attribution.
Planned scope:
I will first audit the existing request-shape, RunTrace, usage, retry, and provider adapter contracts and confirm the #615 boundary. The implementation will preserve current aggregate/public behavior, avoid provider-proxy assumptions, and keep protocol-specific normalization behind existing provider boundaries. I will wait for maintainer assignment before creating the implementation branch.
Astro-Han commentedon Jul 20, 2026
Thanks for offering to take this. I had already started working on #1268 and have now opened #1277 with the implementation. Sorry I didn’t leave a note on the issue earlier to avoid the overlap.
The PR covers the request-capture artifact, per-attempt telemetry for tool loops, retries, failures, interruptions, and cancellation, cache provenance, aggregate trace references, and Harbor segment analysis. If you haven’t started yet, #1277 should cover the issue, so a separate implementation shouldn’t be necessary.