Compaction injects a stale summary when the model does not call compact_chat #629

Summary

When the LLM answers the compact prompt with plain text instead of calling the compact_chat tool, ECA still completes the compaction and injects the chat's previous :last-summary as "The conversation was compacted/summarized, consider this summary:". The new summary, written as text, is discarded. The chat then continues from a summary of an older state, with no warning to the user or the model.

In a chat that was never compacted before, the injected summary is empty instead.

Environment

• ECA 0.161.2 (also present on master at edca8608a)
• Model: a local Qwen3.8-Flash-Next over an OpenAI-compatible endpoint (openai-chat API); probably reproducible with any model that sometimes ignores the tool instruction
• Custom compact prompt (prompts.compact), which says "Call eca__compact_chat with the summary as its argument"; the default compact.md says the same
• Auto-compaction during a running turn (autoCompactPercentage: 85)

What happened

One long chat, compacted three times by auto-compaction, same model and same compact prompt each time:

1. Compactions 1 and 2: the model called compact_chat; the injected summary matched the tool argument. OK.
2. Compaction 3 (mid-turn, right after a long subagent result): the model's reasoning ended with "Now, writing the summary." and it answered with the full summary (~12.8k chars) as a normal assistant message. No tool call, no malformed tool-call markup.
3. ECA appended compact_marker and then injected the summary stored by compaction 2. That summary described a task from hours earlier, with obsolete state and instructions. The text answer from step 2 was not used.
4. ECA resumed the turn ("Continue with the task. The previous user request was: …"), so the agent continued with the stale summary as its only memory of the session.

The persisted chat shows: assistant message with the new summary → compact_marker → user message whose text equals the chat's last-summary field (the old one).

Cause in the code

• compact_chat is the only place that stores a summary: it sets :last-summary and :compact-done? true (src/eca/features/tools/chat.clj, compact-chat).
• A mid-turn compaction finishes through compact-finished-side-effect! → lifecycle/complete-compact! → shared/compact-side-effect! (src/eca/features/chat.clj, trigger-compact!; src/eca/features/chat/lifecycle.clj, complete-compact!; src/eca/shared.clj, compact-side-effect!).
• compact-side-effect! always reads (get-in db [:chats chat-id :last-summary]) and injects it. Nothing checks that compact_chat was called during this compaction; :compact-done? is only used to skip postRequest hooks (chat.clj, on-tools-called).
• The finish side effect runs however the compact prompt ends, also when it ends with a text-only answer.

Steps to reproduce

1. Use a model (or a fake OpenAI-compatible server) that answers the compact prompt with plain text and no tool call.
2. In a chat that was compacted once before (so :last-summary holds summary A), trigger a second compaction (/compact or auto-compaction).
3. The model answers with summary B as text.

Expected: the chat continues with summary B, or the compaction fails visibly.
Actual: the chat continues with summary A (or an empty summary, if there was no earlier compaction). Nothing tells the user.

Suggested fixes

Any of these would prevent the silent stale summary:

1. Detect it: clear :last-summary (or record the compaction's start) before sending the compact prompt, and in compact-side-effect! treat "no compact_chat call in this compaction" as a failure.
2. Recover: when the compact turn ends without the tool call, re-prompt once ("call eca__compact_chat with the summary you just wrote"), or use the final assistant text as the summary.
3. Fail loudly: if neither works, do not append the marker; keep the history, and show a system message such as "Compaction failed: the model did not call compact_chat".

A test with a fake LLM that answers the compact prompt with text only would cover this.

Workaround

None on the user side, except watching for the problem: after a compaction, check that the injected summary describes the current work. Hooks cannot fix it: postCompact runs after the summary was chosen. A postCompact hook can only add context, for example re-inject the current task state from a file.