design(atlas): embedding cost ledger — per-batch token/$ accounting with a delta reuse-ratio alert #6058

Activity

1. dezren39

   added

   area/infraInfrastructure, deployment, and operationsInfrastructure, deployment, and operations

   kind/designA design proposal (design-architect): multi-step, phased, decision-bearingA design proposal (design-architect): multi-step, phased, decision-bearing

   on Nov 5, 2026
2. developing-today commented on Nov 5, 2026

   

   developing-todayBot

   on Nov 5, 2026 – with developing.today

   Contributor

   More actions

   CODE-1723
3. dezren39

   added

   queue/queuedHas a queued task in the Supabase agent queueHas a queued task in the Supabase agent queue

   task/triageQueue packet kind: verify, reproduce, and route this itemQueue packet kind: verify, reproduce, and route this item

   work/assignedQueue worker lifecycle stateQueue worker lifecycle state

   queue/claimedQueue worker lifecycle stateQueue worker lifecycle state

   status/in-progressQueue worker lifecycle stateQueue worker lifecycle state

   and removed

   queue/queuedHas a queued task in the Supabase agent queueHas a queued task in the Supabase agent queue

   on Nov 5, 2026
4. dezren39 commented on Nov 5, 2026

   

   dezren39

   on Nov 5, 2026

   MemberAuthor

   More actions

   🤖 Agent note · done (triaged)

   • Agent: queue worker (repo-worker-code), lane code-r
   • Session / ID: task 10424
   • Found via: queue claim
   • Task: Triage of this embedding cost ledger design at current main.
   • Expectation: triaged — fix path scoped and ordering recorded below; the design itself stays open.
   • Links: parent issue design(atlas): semi-automated ingestion operations — source set, usefulness filter, disk/IO + plan-spend guardrails #6019; delta protocol issue design(atlas): delta-only ingestion + offset embeds — land only changes; span-anchored chunks with non-doubling joins #6028

   Triage findings (task 10424, 2026-11-05)

   Still live, not already handled. Issue #6058 is OPEN (fresh gh check this run) with no hard-stop labels. Code search against current main finds no ledger implementation: reuse_ratio, chunks_reused, and embedding_cost all return zero hits in developing-today/code; the only "cost ledger" hits are unrelated spend-rollup docs. Nothing to close or dedupe — the sibling search shows no other issue covering a per-batch embedding ledger (nearest neighbours are issue #6061 storage growth model and issue #6069 Modal credits for embedding batch work, both OPEN and distinct).

   Dependencies and ordering (recorded per the packet acceptance criteria):

   • Parent issue design(atlas): semi-automated ingestion operations — source set, usefulness filter, disk/IO + plan-spend guardrails #6019 (ingestion operations / plan-headroom accounting) is OPEN — this ledger is its embedding-side complement, per the issue body.
   • Issue design(atlas): delta-only ingestion + offset embeds — land only changes; span-anchored chunks with non-doubling joins #6028 (delta-only ingestion + offset embeds) is OPEN and defines the delta protocol this ledger measures: batch identity and chunk identity (new vs reused) must come from issue design(atlas): delta-only ingestion + offset embeds — land only changes; span-anchored chunks with non-doubling joins #6028's semantics first — the reuse ratio is undefined until "reused chunk" is. Issue design(atlas): delta-only ingestion + offset embeds — land only changes; span-anchored chunks with non-doubling joins #6028's own triage earlier today already records issue design(atlas): embedding cost ledger — per-batch token/$ accounting with a delta reuse-ratio alert #6058 as one of its dependencies, so the pair should be designed as a set.
   • Issue design(atlas): ingest attestation record — in-toto-shaped per-batch statement binding chunks to sources #6045 (ingest attestation record, per-batch statement) is OPEN — the ledger row and the attestation statement describe the same batch, so their batch keys must agree; either record can carry the other’s fields rather than inventing a second batch identity.
   • Baseline calibration is blocked on real batches: per-source-class reuse baselines need observed ingest runs (first-pass ingestion issue task(atlas): first-pass corpus ingestion — all code repos, all findable docs, dezren39/cairn #6025 is itself gated behind issues design(atlas): semi-automated ingestion operations — source set, usefulness filter, disk/IO + plan-spend guardrails #6019/task(atlas): classification revalidation scan — verify nothing sensitive sits at the wrong tier #6021/design(atlas): delta-only ingestion + offset embeds — land only changes; span-anchored chunks with non-doubling joins #6028). The alert rule can be specified now — reuse ratio = chunks_reused / (chunks_new + chunks_reused), alert when <90% on an append-mostly source class, read as "delta protocol broken, not budget" — but baseline numbers cannot be invented before those runs exist.
   • Pricing input: the per-model $/token table (hosted vs self-hosted routes in the issue body) interacts with issue design(atlas): secret-tier local embedder — model selection and fleet-hardware evaluation (secret text never leaves the fleet) #6059 (secret-tier local embedder / model selection) and issue design(modal): best use of Modal credits — $30/mo recurring + $100 one-time grant, for Atlas embedding and GPU batch #6069 (Modal credits for embedding batch) — the ledger schema should store model + computed $ per batch so route changes don't rewrite history.

   Scoped next step (when worked): a design pass producing (1) the ledger schema (batch_id, source_class, chunks_new, chunks_reused, tokens_embedded, model, usd), (2) the reuse-ratio alert rule above, (3) per-source-class baseline slots to be filled from the first real batches. No decision from Drew is needed for that pass; implementation waits on issue #6028's batch semantics.
5. dezren39

   added

   status/triagedTriaged and routed (see agent note)Triaged and routed (see agent note)

   and removed

   status/in-progressQueue worker lifecycle stateQueue worker lifecycle state

   queue/claimedQueue worker lifecycle stateQueue worker lifecycle state

   work/assignedQueue worker lifecycle stateQueue worker lifecycle state

   on Nov 5, 2026
6. github-actions

   assigned

   dezren39

   on Nov 5, 2026