Repository navigation
feat(cost-accounting): per-tenant + per-device usage ledger (§5 Next) - #749
qingyun-wu wants to merge 3 commits into
Conversation
sonichi
left a comment
There was a problem hiding this comment.
LGTM — append-only JSONL with single-writeSync discipline is exactly the right shape for a usage ledger that has to survive crashes + concurrent writers without an in-process buffer.
Confirms my #751 review concern (concurrent-safety of record()):
openSync(path, 'a')+ singlewriteSync(line)+closeSync— POSIX guarantees atomicity forO_APPENDwrites ≤ PIPE_BUF (4096 Linux / 512 macOS). JSON line at 100-300 bytes is well under. Multiple async IIFE calls from voice-agent session-end can interleave safely; lines stay intact. The comment makes the discipline explicit (mirrorslearn-collector.py).
Things I like:
- Per-event tenant_id in the JSON line even though the path already routes by tenant. Defensive — multi-machine sync via append-concat preserves filtering correctness in
rollup(). tenantPathextension is backward-compatible — optionalforTenant?: string4th arg, existing callers untouched. The "cross-tenant read without invalidating current cache" comment explains the rationale.rollup()zero-default empty when no ledger — first-call/no-data case returns valid empty rollup, not undefined/throw. Audit UI consumer doesn't need a null-check.__internaltest helpers (__ledgerPathForTests,__resetLedgerForTests) — matches the__resetForTestspattern from #739 / #746. Consistent across the §5 scaffold series.- Truncate-by-reopen in
__resetLedgerForTestswith the right comment: avoidsfs.rmSyncbecause the ledger path may be under a private dir not owned outright in tests.
Three non-blocking observations:
-
rollup()reads the whole file into memory.readFileSync+split('\n'). Fine for the small near-term ledger sizes; will scale linearly when long-running tenants accumulate weeks/months of events. Worth a TODO comment for "when ledger > N MB, stream via createReadStream + readline." Not a blocker, but downstream Audit UI's first owner-runs-rollup-on-a-2-month-ledger will be the moment. -
record()doesmkdirSyncon every call. Cheap (recursive=true, idempotent), but called per voice-session-end + per tool-call (per #751). CachingensuredDirs: Set<string>at module scope skips redundant syscalls — pure perf, no correctness risk. -
by_modeldoesn't break outmsdurations. Currently onlyin_tokens,out_tokens, andrequestsper model. If the Audit UI ever wants "average latency by model," you'd needif (event.model && event.unit === 'ms')added to the by_model branch. Not in this PR's contract; flag for if/when that view becomes relevant.
Tests: 8 cases, includes the 14-test regression check against tenant.ts (backward-compat verification). Solid coverage.
Ready to merge. Unblocks #750 (audit-summary) + #751 (voice-agent emit).
…ale TODO Three fixups per sonichi's #749 review: 1. mkdirSync(target, recursive=true) is cheap but non-zero; record() fires per tool-call + per session-end, so an _ensuredDirs Set saves N-1 syscalls per ledger over the process lifetime. Pure perf, no correctness risk. 2. Add 'ms' to by_model. Previously by_model only carried in_tokens, out_tokens, requests — when audit-summary (#750) eventually adds a per-model average-latency view, it needs ms broken out by model. Free roll-up: emit shape unchanged; rollup() picks up ms when the event has both .model and unit='ms'. New test verifies. 3. TODO(scale) comment in rollup() docstring: 'reads whole file into memory' — fine near-term, but flagged to switch to createReadStream + readline when long-running tenants hit ~50MB+ ledgers. Also pulled in the time-range test's setTimeout(2) fix from #750 so #749's CI stays deterministic in isolation (was flaky due to ms- precision adjacent record() calls colliding on the same timestamp). Tests: 9/9 pass (was 8 — added 'by_model now breaks out ms' test). tsc clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…profile (#751 nit) Per sonichi's #751 review: 3 record() calls per session-end raised a question about fsync-per-write cost. Reality (per #749's implementation): record() does open+write+close with no explicit fsync — kernel batches via the page cache, so syscalls amortize cleanly. The _ensuredDirs cache added in #749's fixup commit further reduces the mkdir cost. Doc-comment explains the cost profile + flags the revisit trigger (hundreds-of-events-per-session, e.g. when bodhi exposes per-utterance token counts) so the next reader doesn't re-debate batching at the current scale. No code change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (§5 Next) Implements §5 Next "Audit UI built on existing audit.jsonl files. 'What did the agent do, on which device, with which methodology, why?' A trust feature for productization, not just a debugging artifact." First-cut: CLI summarizer that aggregates every audit source already in the repo. Web UI is a future iteration; CLI lands first because (a) it's the canonical owner-introspection interface (matches self-diagnose, regression-search), (b) the output composes into shell pipelines. Sources consumed (any subset; missing files skipped): - data/voice-metrics.jsonl — voice-agent sessions - data/call-metrics.jsonl — phone-conversation metrics (also looks under tasks/archive/data/ and workspace/tasks/data/) - tasks/archive/<YYYY-MM>/ — task lifecycle counts by source - results/archive/<YYYY-MM>/ — result lifecycle counts - state/cost-accounting/usage-<tenant>.jsonl — from PR #749 Output: markdown by default, --json for machine-readable. Usage: tsx src/audit-summary.ts # today tsx src/audit-summary.ts --since 7d # last 7 days tsx src/audit-summary.ts --since 2026-05-01 --until 2026-05-15 tsx src/audit-summary.ts --tenant acme # cost rollup scoped tsx src/audit-summary.ts --json What ships: - src/audit-summary.ts (~230 LOC): summarize(opts) returns an AuditSummary object; renderMarkdown(s) produces the report; CLI entry point parses --since/--until/--tenant/--json. - tests/audit-summary.test.ts — 6 unit tests covering empty workspace, voice-metrics date filtering + aggregation, call-metrics owner/ meeting flags, archive counts by mtime, cost-ledger surfacing, renderMarkdown shape. All pass. - tests/cost-accounting.test.ts: fixed flake in the time-range test (adjacent record() calls collide on ms; setTimeout(2) between them gives a stable cut point). Now deterministic. Real-workspace smoke (last 24h on my install): - 19 voice sessions, 664 min, 4 tool calls - 1 phone call, 0.1 min - 32 archived tasks, 37 archived results - Cost ledger empty (emitters not wired — wiring is the follow-up PR noted in #749's body). What this PR does NOT change: - No web UI. CLI markdown is the first cut. - No structured emit to audit.jsonl. The summarizer reads existing scattered sources; a unified audit.jsonl is a future refactor. - No per-device or per-methodology breakdown beyond what the cost ledger already exposes — owner can drill via --tenant + the by_device and by_tool maps in --json output. Test plan: - [x] npx tsx --test tests/audit-summary.test.ts tests/cost-accounting.test.ts tests/tenant.test.ts — 28/28 pass. - [x] npx tsc --noEmit clean. - [x] Real-workspace smoke: tsx src/audit-summary.ts --since 24h outputs structured markdown. Depends on PR #749 (cost-accounting) for the rollup() consumer. Targets feat/cost-accounting-scaffold as base. Roadmap impact: - §5 Next "Audit UI": CLI first cut shipped. With this + #748 + #749 + the existing voice/call/archive sources, every "what did the agent do" question has a structured answer. Web UI is the next iter. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
liususan091219
left a comment
There was a problem hiding this comment.
Reviewed: new cost-accounting.ts append-only JSONL ledger per tenant. Single-line events under PIPE_BUF (atomic via O_APPEND), no in-process aggregation that could be lost to crash. Same pattern as learn-collector.jsonl. Event shape supports tenant/device/source/kind/amount/unit/model — enough for downstream roll-ups in #750. — Lucy (Mac Studio bot)
liususan091219
left a comment
There was a problem hiding this comment.
LGTM — append-only JSONL with openSync(path,'a')+writeSync(line)+closeSync is the right shape for a usage ledger that has to survive crashes + concurrent writers without an in-process buffer. POSIX O_APPEND guarantees atomicity for single-line writes ≤ PIPE_BUF (4096 Linux / 512 macOS), and a typical event JSON line lands at 100-300 bytes — well under. The discipline mirrors learn-collector.py's ledger and the comment makes that explicit. (Note: sonichi's own #749 review already called out the atomicity story in depth — this re-review fills in the deferred per-PR critique I owed.)
Things I like:
-
record()is a pure write — no in-process aggregation. Crash-resistance follows directly: at worst you lose the last in-flight event, never a buffer of unflushed counters. Aggregation lives inrollup()on the read path where it can be re-run anytime. This is the right place to draw the read/write line for a metering ledger. -
_ensuredDirs: Set<string>mkdir memoization.mkdirSync(..., {recursive:true})is idempotent and cheap individually, but called per-record (per tool / per session-end) it stacks up. Caching bydirname(path)is the right key — across all events for a tenant the dir is the same, so one syscall amortizes the whole process lifetime. -
tenantPath(... , forTenant)extension is backward-compatible. Optional 4th arg added in this PR's chain with#748doesn't break existing callers. The comment ("Cross-tenant queries passforTenantexplicitly; default uses the current cached tenant id") names the rationale — admin tooling can read another tenant's ledger without polluting the current cache. -
Per-event
tenant_idin the JSON line even though the path already routes by tenant. Defensive belt-and-suspenders: multi-machine sync via append-concat preserves filtering correctness inrollup()(theif (event.tenant_id !== tid) continueline). If the file ever gets misrouted or mixed during a manual move, the per-line tag self-corrects. -
rollup()zero-default empty when no ledger. First-call / no-data case returns a valid emptyUsageRollup, notundefinedor a throw. The audit-summary consumer in #750 doesn't need a null-check before destructuring. -
__ledgerPathForTests+__resetLedgerForTestsannotated@internal. Consistent with#739 device-session.ts,#746 device-access-policy.ts,#748 tenant.ts. Test surface is parallel across the §5 scaffold series, keeping the cognitive load low for the test writers. -
Truncate-by-reopen in
__resetLedgerForTestswith the right rationale comment: avoidsfs.rmSyncbecause the ledger path may be under a private dir not owned outright in tests.closeSync(openSync(path,'w'))is the minimum-rights operation.
Non-blocking observations:
-
rollup()reads the whole file into memory.readFileSync+.split('\n'). Fine for the small near-term ledger sizes; will scale linearly when long-running tenants accumulate weeks/months of events. The TODO comment in the file already names the trigger ("~50MB+") and the fix (createReadStream+readlinestreaming). Tracked. -
by_modeldoesn't break outmsdurations ortool_calls. Theby_modelmap currently aggregatesin_tokens+out_tokens+requests+ms—tool_callis missing. If the Audit UI ever wants "tools-invoked-per-model" or "p95 latency by model with N>=K requests," theby_modelbranch needs an additionalif (event.unit === 'tool_call')line. Not in this PR's contract; flag for if/when that view becomes relevant. -
No retention / rotation policy. Long-running tenants accumulate forever. The eventual fix is either (a) external
logrotate(8)-style daily file split (usage-<tenant>-YYYY-MM.jsonl) withrollup()glob-reading, or (b) an in-script compaction CLI. For a first-cut scaffold this is the right cut — record-and-aggregate, defer retention until a real owner-feedback signal arrives. -
record()accepts an unboundedsource: string. No allowlist. A misconfigured caller can polluteby_tool/by_devicekeys with junk strings that never get cleaned up. Cheap guard: regex-validatesourceagainst^[a-z0-9][a-z0-9_:.-]{0,63}$at write time, warn-and-drop on mismatch. Same posture astenant.ts'sTENANT_ID_PATTERN. Cosmetic but cheap. -
Concurrent
mkdirSyncrace on first emit. Two processes racing to write to a tenant's first-ever ledger both hit themkdirpath.{recursive:true}is idempotent so neither fails, but the_ensuredDirs.add(dir)is per-process so both populate the set independently. Functionally fine; just calling out it's not a coordinated lock.
Tests: tests/cost-accounting.test.ts present, 168 lines — uses mkdtempSync for isolated tmpdir, exercises record + rollup against a populated ledger, validates per-tenant routing, the cross-tenant forTenant arg, the __resetLedger boundary, and (per the diff tail) a 14-test regression check against tenant.ts for backward-compat. Solid for the foundation PR.
Ready to merge. Unblocks #750 (audit-summary consumer) + #751 (voice-agent session-end emit) + #752 (phone-conversation per-call emit). The append-only-JSONL discipline + tenantPath extension together set the shape for any future metering surface; everything that lands on top inherits the crash-resistance. — Lucy (Mac Studio bot)
41f5b19 to
441cf1b
Compare
Implements §5 Next "Cost accounting. Per-tenant + per-device token + tool
usage metering. Required to price anything; useful even single-tenant."
Same design discipline as src/learn-collector.py:
- Append-only JSONL ledger per tenant.
- Atomic single os.write under PIPE_BUF (~512 macOS / 4096 Linux);
matches the existing learn-collector idiom for shared-file writes.
- No in-process aggregation buffer that could be lost on crash.
- Rollup is the read path (separate from emit) so summarizer flow can
evolve independently.
What ships:
- src/cost-accounting.ts (~150 LOC):
* UsageEvent shape: {ts, tenant_id, device_id?, source, kind,
amount, unit, model?, tool?, request_id?}.
* record(event) — atomic O_APPEND write to the tenant's ledger.
Caller passes optional explicit tenant_id; default uses tenantId().
* rollup({tenant_id?, from?, to?}) — read + aggregate by model /
tool / device. Per-tenant scoped; passes forTenant to tenantPath
so cross-tenant queries route correctly regardless of the current
cached tenant.
* Helpers: ledgerExists(), ledgerSizeBytes(), __ledgerPathForTests().
- src/tenant.ts: extended tenantPath() with optional 4th parameter
forTenant. Cross-tenant readers (cost-accounting rollup, future audit
UI) pass it explicitly to route to the right tenant directory without
invalidating their own cached id. Backward compatible — default keeps
using tenantId() when omitted.
- tests/cost-accounting.test.ts — 8 unit tests covering record/rollup,
per-tenant isolation, time-range filter, by_device fanout, missing
ledger graceful-zero, explicit tenant_id override, corrupt-line skip.
All 8 pass. The existing 14 tenant tests stay green.
What this PR does NOT change:
- No callers emit usage events yet. voice-agent + bodhi-realtime-agent
+ skill execution all have the data to record, but wiring them is the
follow-up PR (one call site at a time, easier to review separately).
- No UI / report generator. `rollup()` is the contract the future
Audit UI (§5 Next) consumes.
Test plan:
- [x] npx tsx --test tests/cost-accounting.test.ts — 8/8 pass.
- [x] npx tsx --test tests/tenant.test.ts — 14/14 pass (regression
check after the tenantPath forTenant parameter addition).
- [x] npx tsc --noEmit clean.
Depends on PR #748 (tenant scaffold). Targets feat/tenant-identity-
scaffold as base.
Roadmap impact:
- §5 Next "Cost accounting": contract + ledger landed. Emission +
reports are follow-ups.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ale TODO Three fixups per sonichi's #749 review: 1. mkdirSync(target, recursive=true) is cheap but non-zero; record() fires per tool-call + per session-end, so an _ensuredDirs Set saves N-1 syscalls per ledger over the process lifetime. Pure perf, no correctness risk. 2. Add 'ms' to by_model. Previously by_model only carried in_tokens, out_tokens, requests — when audit-summary (#750) eventually adds a per-model average-latency view, it needs ms broken out by model. Free roll-up: emit shape unchanged; rollup() picks up ms when the event has both .model and unit='ms'. New test verifies. 3. TODO(scale) comment in rollup() docstring: 'reads whole file into memory' — fine near-term, but flagged to switch to createReadStream + readline when long-running tenants hit ~50MB+ ledgers. Also pulled in the time-range test's setTimeout(2) fix from #750 so #749's CI stays deterministic in isolation (was flaky due to ms- precision adjacent record() calls colliding on the same timestamp). Tests: 9/9 pass (was 8 — added 'by_model now breaks out ms' test). tsc clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
90fcf16 to
8a36cab
Compare
…dger Second consumer of PR #749's per-tenant cost ledger. Mirrors voice-agent's emit shape from PR #751: 1 request + total duration in ms + 1 tool_call per session.toolCalls entry. Token counts deferred until bodhi exposes in/out totals — same shape lights them up when ready. device_id='phone-call:<callSid>' distinguishes phone-originated usage from laptop/glasses in audit-summary (#750)'s by_device breakdown. Non-owner calls emit with source='phone-conversation:non-owner' so spam-attribution is preserved without leaking the caller number into device_id keys. Dynamic import + try/catch keeps cost-accounting an optional dep — if the module is absent (older worktree, partial deploy), phone session finalization silently skips emit rather than crash the call-end path. Mirrors #751's defensive shape. Test plan: - [x] npx tsc --noEmit clean. - [ ] Manual: complete a phone call → verify state/cost-accounting/usage-default.jsonl gains api_call(request=1) + api_call(ms=N) + per-tool entries with device_id='phone-call:<callSid>'. - [ ] Manual: audit-summary --since 24h shows the Cost ledger populated with phone-call.<callSid> in by_device. Depends on PR #749 (cost-accounting). Targets feat/cost-accounting- scaffold as base. Independent of PR #751 — both emit into the same ledger but live at separate call sites. Roadmap impact: - §5 Next 'Cost accounting' second consumer landed. Remaining emit sites: skill execution (gws-gmail-voice, image-generation, etc.) and the Mentra bridge (PR #740). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Activates PR #749's per-tenant cost-accounting ledger from voice-agent's session-end path. Without this, the ledger stays empty and the cost section of audit-summary (PR #750) reports 'No events recorded yet (cost-accounting emitters not wired)'. What ships: - writeVoiceMetrics() now also emits to cost-accounting after the existing voice-metrics.jsonl write. Records: * 1 'request' event tagged with the voice model (gemini-2.5-flash today). * Total session duration in ms. * 1 'tool_call' per voiceToolCalls entry, with the tool name. - Token in/out counts are intentionally NOT emitted yet — the realtime transport doesn't expose them today. When bodhi adds them (or we add a thin counter around sendFile/sendContent), the same emit point lights up the token totals; no schema change needed. - Dynamic import of './cost-accounting.js' so the module is optional — older worktrees / partial deploys silently skip emit rather than crash the session-end path. Device-aware shape: 'laptop' deviceId is hardcoded here because the voice-agent's local session IS the laptop. Per-device emission for Mentra glasses / phone calls / future devices lives in the bridge that owns the relevant DeviceSession (PR #740 for Mentra; phone- conversation's conversation-server.ts is the next consumer). Test plan: - [x] npx tsc --noEmit clean. - [ ] Manual: start voice-agent, complete a session, verify `state/cost-accounting/usage-default.jsonl` exists with api_call(request=1, ms=N) + per-tool entries. - [ ] Manual: `tsx src/audit-summary.ts --since 24h` shows the Cost ledger section populated. Depends on PR #749 (cost-accounting scaffold). Targets feat/cost- accounting-scaffold as base. Roadmap impact: - §5 Next 'Cost accounting' consumer landed for voice. Phone- conversation + skill execution + Mentra bridge are follow-ups (one call site per PR keeps review focused). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…profile (#751 nit) Per sonichi's #751 review: 3 record() calls per session-end raised a question about fsync-per-write cost. Reality (per #749's implementation): record() does open+write+close with no explicit fsync — kernel batches via the page cache, so syscalls amortize cleanly. The _ensuredDirs cache added in #749's fixup commit further reduces the mkdir cost. Doc-comment explains the cost profile + flags the revisit trigger (hundreds-of-events-per-session, e.g. when bodhi exposes per-utterance token counts) so the next reader doesn't re-debate batching at the current scale. No code change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (§5 Next) Implements §5 Next "Audit UI built on existing audit.jsonl files. 'What did the agent do, on which device, with which methodology, why?' A trust feature for productization, not just a debugging artifact." First-cut: CLI summarizer that aggregates every audit source already in the repo. Web UI is a future iteration; CLI lands first because (a) it's the canonical owner-introspection interface (matches self-diagnose, regression-search), (b) the output composes into shell pipelines. Sources consumed (any subset; missing files skipped): - data/voice-metrics.jsonl — voice-agent sessions - data/call-metrics.jsonl — phone-conversation metrics (also looks under tasks/archive/data/ and workspace/tasks/data/) - tasks/archive/<YYYY-MM>/ — task lifecycle counts by source - results/archive/<YYYY-MM>/ — result lifecycle counts - state/cost-accounting/usage-<tenant>.jsonl — from PR #749 Output: markdown by default, --json for machine-readable. Usage: tsx src/audit-summary.ts # today tsx src/audit-summary.ts --since 7d # last 7 days tsx src/audit-summary.ts --since 2026-05-01 --until 2026-05-15 tsx src/audit-summary.ts --tenant acme # cost rollup scoped tsx src/audit-summary.ts --json What ships: - src/audit-summary.ts (~230 LOC): summarize(opts) returns an AuditSummary object; renderMarkdown(s) produces the report; CLI entry point parses --since/--until/--tenant/--json. - tests/audit-summary.test.ts — 6 unit tests covering empty workspace, voice-metrics date filtering + aggregation, call-metrics owner/ meeting flags, archive counts by mtime, cost-ledger surfacing, renderMarkdown shape. All pass. - tests/cost-accounting.test.ts: fixed flake in the time-range test (adjacent record() calls collide on ms; setTimeout(2) between them gives a stable cut point). Now deterministic. Real-workspace smoke (last 24h on my install): - 19 voice sessions, 664 min, 4 tool calls - 1 phone call, 0.1 min - 32 archived tasks, 37 archived results - Cost ledger empty (emitters not wired — wiring is the follow-up PR noted in #749's body). What this PR does NOT change: - No web UI. CLI markdown is the first cut. - No structured emit to audit.jsonl. The summarizer reads existing scattered sources; a unified audit.jsonl is a future refactor. - No per-device or per-methodology breakdown beyond what the cost ledger already exposes — owner can drill via --tenant + the by_device and by_tool maps in --json output. Test plan: - [x] npx tsx --test tests/audit-summary.test.ts tests/cost-accounting.test.ts tests/tenant.test.ts — 28/28 pass. - [x] npx tsc --noEmit clean. - [x] Real-workspace smoke: tsx src/audit-summary.ts --since 24h outputs structured markdown. Depends on PR #749 (cost-accounting) for the rollup() consumer. Targets feat/cost-accounting-scaffold as base. Roadmap impact: - §5 Next "Audit UI": CLI first cut shipped. With this + #748 + #749 + the existing voice/call/archive sources, every "what did the agent do" question has a structured answer. Web UI is the next iter. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… (§5 Next) (#750) Implements §5 Next "Audit UI built on existing audit.jsonl files. 'What did the agent do, on which device, with which methodology, why?' A trust feature for productization, not just a debugging artifact." First-cut: CLI summarizer that aggregates every audit source already in the repo. Web UI is a future iteration; CLI lands first because (a) it's the canonical owner-introspection interface (matches self-diagnose, regression-search), (b) the output composes into shell pipelines. Sources consumed (any subset; missing files skipped): - data/voice-metrics.jsonl — voice-agent sessions - data/call-metrics.jsonl — phone-conversation metrics (also looks under tasks/archive/data/ and workspace/tasks/data/) - tasks/archive/<YYYY-MM>/ — task lifecycle counts by source - results/archive/<YYYY-MM>/ — result lifecycle counts - state/cost-accounting/usage-<tenant>.jsonl — from PR #749 Output: markdown by default, --json for machine-readable. Usage: tsx src/audit-summary.ts # today tsx src/audit-summary.ts --since 7d # last 7 days tsx src/audit-summary.ts --since 2026-05-01 --until 2026-05-15 tsx src/audit-summary.ts --tenant acme # cost rollup scoped tsx src/audit-summary.ts --json What ships: - src/audit-summary.ts (~230 LOC): summarize(opts) returns an AuditSummary object; renderMarkdown(s) produces the report; CLI entry point parses --since/--until/--tenant/--json. - tests/audit-summary.test.ts — 6 unit tests covering empty workspace, voice-metrics date filtering + aggregation, call-metrics owner/ meeting flags, archive counts by mtime, cost-ledger surfacing, renderMarkdown shape. All pass. - tests/cost-accounting.test.ts: fixed flake in the time-range test (adjacent record() calls collide on ms; setTimeout(2) between them gives a stable cut point). Now deterministic. Real-workspace smoke (last 24h on my install): - 19 voice sessions, 664 min, 4 tool calls - 1 phone call, 0.1 min - 32 archived tasks, 37 archived results - Cost ledger empty (emitters not wired — wiring is the follow-up PR noted in #749's body). What this PR does NOT change: - No web UI. CLI markdown is the first cut. - No structured emit to audit.jsonl. The summarizer reads existing scattered sources; a unified audit.jsonl is a future refactor. - No per-device or per-methodology breakdown beyond what the cost ledger already exposes — owner can drill via --tenant + the by_device and by_tool maps in --json output. Test plan: - [x] npx tsx --test tests/audit-summary.test.ts tests/cost-accounting.test.ts tests/tenant.test.ts — 28/28 pass. - [x] npx tsc --noEmit clean. - [x] Real-workspace smoke: tsx src/audit-summary.ts --since 24h outputs structured markdown. Depends on PR #749 (cost-accounting) for the rollup() consumer. Targets feat/cost-accounting-scaffold as base. Roadmap impact: - §5 Next "Audit UI": CLI first cut shipped. With this + #748 + #749 + the existing voice/call/archive sources, every "what did the agent do" question has a structured answer. Web UI is the next iter. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vidhuUC
left a comment
There was a problem hiding this comment.
APPROVE
Solid cost-accounting implementation. The O_APPEND atomic write (PIPE_BUF) pattern is correct for JSONL — matches learn-collector discipline. The audit-summary CLI as first-cut UI is the right call before a web UI. Key things verified:
rollup()is the clean contract — correct to leave the audit UI as a follow-up.forTenant4th-param extension to tenantPath is backward-compatible.- Dynamic import + try/catch for cost-accounting in callers is the right defensive shape.
- 8 tests covering record/rollup/ledger helpers.
The audit-summary CLI output (voice sessions, call metrics, cost rollup, task counts) is exactly what an owner needs for 'what did the agent do and what did it cost.' --json flag for machine-readable output composes well.
Implements §5 Next "Cost accounting" — append-only JSONL ledger per tenant. Mirrors src/learn-collector.py discipline (atomic O_APPEND write under PIPE_BUF, no in-process buffer, separate emit/read paths).
Base: feat/tenant-identity-scaffold (#748). Depends on #748 for tenantPath (extended in this PR with a forTenant override for cross-tenant reads).
What ships
Out of scope
Test plan
🤖 Generated with Claude Code