Repository navigation
Description
Summary
Related: #332 (default raw: true injection) and #333 (no retention/rotation/size-cap) — this one is a distinct, complementary knob: compact an ended session's events.jsonl down to its resume-relevant accounting lines instead of asking users to choose between full payloads forever and whole-session deletion.
Two facts about the read side of events.jsonl that make a "small retained history" well-defined:
- The only component that reads the file at runtime is
cost_history.pyin amplifier-app-cli, which sumsdata.usage.cost_usdacrossllm:responseevents to re-seed cumulative session cost on resume. It tolerates a file containing only those lines (substring pre-filter; extra lines skipped; missing file → graceful skip). - Every other reference (session repair / truncate utilities, fork/export) only truncates or copies the file proportionally — line content beyond
llm:responseis never interpreted. In-session consumers (web UI streaming, etc.) read from memory, not the file.
So for a finished session, ~99% of the file is write-only data: tool:pre/post, content_block:*, provider:*, llm:request, context:compaction, artifact:*. Already-filtered/raw: false settings help the inflow rate (#332) but don't reclaim the structural bulk.
Measured
Compacting 12,791 end-of-life session logs (>1 week untouched) — keep each "event": "llm:response" line trimmed to {ts, event, duration_ms, status, data.usage, data.model, data.provider}, drop all other lines:
- 44.21 GB → 0.06 GB (≈750× smaller)
- Cost accounting identical: summed
cost_usdfrom a compacted file matches the untrimmed original exactly, so cumulative-cost restore on resume (sale.order: Mostrar Comisión, Costo Envío e Ingreso Neto en tab Financiero Canal #284) is preserved byte-for-byte.
Proposal
Module hooks-logging, opt-in config, consistent with the existing exclude_events (#8) style:
hooks:
- module: hooks-logging
config:
compact_on_session_end: true
compact_keep_events: ["llm:response"] # default
compact_keep_data_fields: ["usage", "model", "provider"] # defaultBehavior: on session:end, after the final write, the hook rewrites its own session log atomically (temp file + rename, preserve mtime) dropping lines not matching compact_keep_events (fnmatch) and pruning kept lines to the whitelisted fields. Idempotent; a second compact is a no-op.
Also worth considering in the same pass: session_log_enabled: false as a documented master switch (strictly cleaner than exclude_events: ["*"], which works today but reads as an accident).
Why not the existing knobs
exclude_events(refactor: move subagent logs from .data/ to logs/ directory #8) andstrip_rawact at write time: the immediately-debuggable recent tail is lost, which is the part you usually want. Compaction keeps full detail for the live session and shrinks it only once it's over.- Truncation-based mechanisms ([hooks-logging] No retention/rotation/size-cap option for events.jsonl — append-only files reach 1.1 GB per session #333) drop the
llm:responsetail along with everything else, so cost resume loses data; compaction keeps it.
Interaction notes
amplifier-bundle-team-tracking's session sync can uploadevents.jsonl(include_events: true); docs should note compaction reduces what gets synced, so default stays off (opt-in).- Fork/session-copy utilities that duplicate
events.jsonlkeep working on compacted files (they only copy/truncate).
Happy to send a PR against microsoft/amplifier-module-hooks-logging if the shape looks right.
Activity