Durable mid-task resume: per-tool write-ahead checkpoints, compacted-run resume, and a persistent plan/todo tool #69

Problem

#42 / #54 added --resume from the trajectory JSON, but it is not durable enough for long tasks:

• The checkpoint is written only once per step, after all tool calls in that step finish (internal/agent/loop.go:212-222, OnStep → t.Save in cmd/tinyagent/main.go:223,376). If the process crashes during a tool call, the whole step is lost, including its model response and cost. No record shows the tool started.
• Save already writes atomically (temp file + rename, internal/trajectory/trajectory.go:95-130). It does not fsync, so the write can be lost on power failure.
• Resume refuses compacted runs: internal/trajectory/resume.go:123-124 ("resuming compacted model views is not yet supported"). Long runs, which are the ones most worth resuming, are always compacted.
• Background processes are recorded only as a summary string (trajectory.go:38-40). On resume, nothing tells the model those processes are gone.
• There is no plan or todo state. The task's progress lives only in chat messages, so compaction (internal/agent/compact.go:134-210) can elide or summarize it away.

Proposal (two features designed together)

1. Durable per-step checkpoints

• Add a write-ahead pending_calls record. Persist the assistant message and the list of pending tool calls before running the tools. After each result, persist again: one tool result per save, not one save per step.
• On resume, any call still pending has no result. Synthesize a tool result for it: "[interrupted: tool did not complete before the agent stopped; it may have partially run — verify state before retrying]", with is_error=true. Do not re-run the call automatically, because bash and edit calls are not idempotent.
• Resume compacted runs. Rebuild the view from the last Compaction.View + Summary (already stored, trajectory.go:76-92), then replay any messages after it.
• Add fsync of the file and its directory, and an optional .bak of the previous checkpoint.
• Background processes: on resume, add a notice listing the processes that were registered and are now gone. Do not reattach to them.
• Bump SchemaVersion 1→2. Keep v1 loadable; v1 has no pending calls and no plan.

2. Persistent plan/todo (opt-in, off by default)

• The plan feature is disabled by default. Enable it with --plan or config plan.enabled: true (default false). When it is disabled, runs behave exactly as they do today.
• When enabled, add a new built-in tool update_plan. It replaces the whole list each time it is called:

  {"items":[{"id":"string","text":"string","status":"pending|in_progress|done|blocked"}], "note":"optional string"}

  Rules: at most one item in_progress, a cap of about 50 items, and a cap on text length. The tool returns the rendered plan.
• Store the plan inside the checkpoint as plan: {version, items, updated_step}, outside messages. Also keep plan_history (step + diff) for auditing.
• Put the plan back in front of the model each step as a short [Current plan] block after the system prompt or summary. Compaction never elides or summarizes this block, and the summarizer prompt is given the plan. Count the block toward the size estimate.
• Allow it in --read-only: it changes no files.

How they interact

• One file, not a sidecar. The plan goes in the trajectory JSON, so one atomic rename covers messages, pending calls, and plan together, with no ordering risk between two files. --plan-file <path> (which requires --plan/plan.enabled; it is an error without it) can optionally export a read-only mirror, written after the checkpoint.
• Write order per step: model response + pending_calls → save → run tool i → record its result (and the plan change, if the tool was update_plan) → save → … → clear pending_calls → save → emit step.saved.
• Resume restores: messages, the compacted view, the plan (when enabled), accounting counters, limits, and the format-error counter from Recover from malformed tool calls and stop after repeated format errors #66. Then it closes interrupted calls with synthetic results and adds the continuation notice plus the current plan.
• Compaction: the plan is pinned in the view and passed to the summarizer, and Compaction records plan_version.
• --json events: new plan.updated {step, version, items}, checkpoint.saved {step, path, pending}, resume.interrupted_call {id, name}, resume.process_lost {pid, cmd}.

Rough implementation sketch

• trajectory: add the PendingCalls, Plan, and PlanHistory fields, schema v2, fsync in save, and new migration and Validate rules. Remove the compaction refusal.
• agent/loop.go: split step() into checkpoint() calls around each tool, add the plan block to the view, and add a reinjection hook in compact.go.
• tools: add an update_plan spec and handler. The handler writes to agent state, not the filesystem. Bump ToolSignature (resume.go:19-27), which correctly refuses checkpoints made with the old tool set.
• config/CLI: opt-in --plan, --plan-file (requires the plan to be enabled), and config key plan.enabled (default false). When disabled, update_plan is not advertised, there is no pinned plan block, and checkpoints have no plan field. The checkpoint records whether the plan was enabled, and resume must match that setting (the tool-set check enforces this).
• Hooks (Run user-configured hook commands before and after tool calls to block, audit, or add feedback #67): pre/post hooks run inside the pending window, and a hook that blocks a call clears it from pending. Execution environments (Run bash and background commands in pluggable execution environments (local, sandbox, container) #65): the interrupted-call notice and the lost-process notice are per environment, since a container may still be alive.

Acceptance criteria

• Kill -9 during a long bash call, then --resume. The model sees an interrupted result for that call, and no step, cost, or message is lost.
• A run compacted at least once can be resumed, and the model view after resume equals the view before the crash plus the notice.
• With the plan enabled, the plan survives compaction and resume, and appears in every request.
• By default (no --plan, plan.enabled false), update_plan is not advertised, there is no plan block, checkpoints have no/empty plan, and runs behave exactly as before.
• --plan-file without the plan enabled is rejected with a clear error.
• Background processes from the earlier run are reported as lost on resume.
• v1 checkpoints still resume. Invalid or partially written files are rejected with a clear error.
• --json emits the new events. Tests cover crash points before, during, and after tools.

Decisions

• The plan tool is opt-in and off by default (--plan / plan.enabled).

Open questions

• When the plan is enabled, should a plan be required for long tasks?
• Should an idempotent read-only call (read/grep/glob) that was interrupted be re-run automatically?
• One save per tool vs. per step, given the I/O cost on large trajectories. Should we switch to an append-only JSONL log plus periodic snapshot?
• Is the --output must differ from source rule still right, or should resume be allowed to continue in place?
• Should a resumed run be able to reattach to processes that are still alive under Run bash and background commands in pluggable execution environments (local, sandbox, container) #65 container environments?

Related: #42, #65, #66, #67.