Skip to content

Fix #70: Add opt-in SQLite event log (runs, events, tool calls) mirroring --json output - #77

Merged
vicgupta merged 1 commit into
mainfrom
amf/issue-70
Oct 25, 2026
Merged

vicgupta merged 1 commit into
mainfrom
amf/issue-70

Conversation

@vicgupta

Copy link
Copy Markdown
Owner

Summary

Adds an opt-in local SQLite event log that records the same events as --json, so runs can be queried afterwards.

  • OnEvent now fans out to sinks (internal/eventlog): the JSONL writer and an optional SQLite sink get identical events. --json output is byte-for-byte the same as before, except a new provider.retry event when a model request is retried.
  • Turn it on with --log-db, --log-db=path, or config log_db.enabled/path/level/max_days/max_runs/max_payload_bytes. Default path is <cwd>/.tinyagent/tinyagent.db. Off by default; nothing is created unless enabled.
  • Uses pure-Go modernc.org/sqlite, so no cgo. WAL mode, 10s busy timeout, one connection, all queries parameterized, embedded migrations tracked in schema_migrations.
  • Tables: runs, events (run_id, seq, step, ts, type, JSON payload), tool_calls (args, result summary, duration, is_error).
  • Default level: metadata stores no bodies: text, arguments, and output become *_len, and the task is stored only as a sha256 hash plus its length. level: full stores bodies after redaction (*_KEY/_TOKEN/_SECRET/_PASSWORD env values, MCP/hook env secrets, common token shapes). Payloads are capped at 64 KiB by default.
  • Old runs are pruned when a run starts (30 days / 500 runs by default; 0 keeps all).
  • If the database fails at open or during writes, tinyagent prints exactly one warning and the run continues. A DB error never fails a run or changes the exit code.

Decisions

  • Default level is metadata and the DB is per-repo under --cwd. The file is created 0600 and its dir 0700. The README suggests adding .tinyagent/ to .gitignore.
  • The trajectory JSON is still the resume format. The DB sits beside it and doesn't replace it.
  • No tinyagent logs subcommand yet. The README shows how to query with sqlite3.
  • The optional flag value has to be written --log-db=path, because Go's flag package can't take an optional value after a space.
  • I didn't add per-request model events, so --json keeps its shape. Per-run tokens and cost go in runs. tool_calls.exit_code is reserved and currently NULL.
  • Writes are batched on a background goroutine. If the process crashes, the last few events may not be written.

Testing

  • New tests for: fresh schema and WAL, migrating forward, metadata storing no bodies, full storing bodies, redaction, retention by max_runs and max_days, a write failure warning once, 4 concurrent runs × 200 events, injection-like input, payload truncation, an unwritable path, JSONL↔DB mirror equality, off by default (no .tinyagent), the default path being created, config path relative to --cwd, and the retry observer.
  • gofmt -l, go vet ./..., go test ./... all pass. The eventlog tests also pass with -race -count=5.
  • CGO_ENABLED=0 builds succeed for linux/darwin/windows × amd64/arm64.

Binary size

linux/amd64, -trimpath -ldflags="-s -w": 9,506,976 → 13,738,144 bytes (+4,231,168, about +4.0 MiB / +44.5%). Other targets: 12.8–13.8 MB.

Closes #70

@vicgupta

Copy link
Copy Markdown
Owner Author

Rebased onto main by the hourly merge job (head now bcd69c5), which now includes #75 (format errors) and #76 (durable checkpoints, plan tool).

Rebase notes: all conflicts were additive (flag/config/docs/event-table rows, the OutputSchema + cur/Step() agent fields, and the Plan + LogDB config types). One fix beyond conflict resolution: the merged --help golden text in cmd/tinyagent/main_test.go had a stray blank line before Other:; removed so the golden matches the actual --help output (verified by the passing TestUsage). No behavior change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add opt-in SQLite event log (runs, events, tool calls) mirroring --json output

1 participant