ECA becomes unresponsive when tool output causes LLM context overflow

Description

When a tool (especially shell commands) produces very large output, ECA hits the LLM provider's token limit with:

LLM response status: 400 body: {"error":{"message":"prompt token count of 174837 exceeds the limit of 168000","code":"model_max_prompt_tokens_exceeded"}}

After this error, the ECA server and Emacs become unresponsive/frozen, requiring a restart.

Root Cause Analysis

The issue involves multiple contributing factors on both server and client side.

Server-side

No proactive token budget check before API call: The server sends the full message history to the LLM provider without estimating token usage. Context overflow is only detected reactively (HTTP 400 response) and recovered via prune + auto-compact. While the recovery logic works, it doesn't prevent the initial expensive roundtrip and the cascade of effects on the client.
Tool output truncation exists but may be insufficient: outputTruncation defaults to 2000 lines / 50 KB, but a single tool result at the maximum (50 KB ≈ 12,500 tokens) combined with a long conversation history can still exceed the context limit. The truncation also doesn't account for the cumulative token budget across all messages.
Potential infinite recovery loop: If auto-compact succeeds but the subsequent resume prompt still triggers context overflow, auto-compacting? has already been reset to false, allowing another compact cycle. There is no counter or recovery-attempted? guard to prevent Compact → Resume → Overflow → Compact → … loops (analogous to the existing auto-continued? guard for truncated responses).

Client-side (Emacs)

Synchronous JSON parsing of large messages: The process filter in eca-process.el parses incoming JSON messages synchronously with no size limit. Before the overflow error, the server sends toolCalled notifications containing the full tool output. Parsing a multi-MB JSON message blocks the Emacs main thread.
font-lock-ensure refontifies entire buffer: After each text content insertion (eca-chat.el), font-lock-ensure runs over the entire chat buffer. In gfm-mode with markdown-fontify-code-blocks-natively, this becomes very expensive for large buffers.
align-tables / beautify-tables scan entire buffer: On "finished" status, both functions are called with (point-min), scanning the entire chat buffer for markdown tables — expensive for long conversations.
Expandable content has no size limit: Tool call outputs are stored and rendered in expandable overlays without any truncation. Large outputs bloat the buffer.

These client-side issues compound: large tool output → expensive JSON parse → expensive rendering → expensive fontification → expensive table alignment, all running synchronously on the Emacs main thread.

Steps to Reproduce

Start an ECA chat session with a model that has a moderate context window (e.g., 128K-168K tokens)
Have a conversation long enough to consume a significant portion of the context
Execute a tool/shell command that produces very large output (e.g., find / -name "*.log" or similar)
The LLM provider returns HTTP 400 (token limit exceeded)
Emacs becomes unresponsive

Fix potential infinite auto-compact loop on context overflow

@snoopier

@snoopier I wanna understand better the frozen, can you try latest release, I did some improvements related to loop in compact, but not sure it's what you are seeing.

Using 0.125.0 i can't reproduce because of several other issues.

Maybe you can try a test prompt with Copilot/opus that have lower sessionj limit

**Create a ~130KB Markdown test file (`test.md`) to test ECA session limits.**

The file must:

1. Contain valid, realistic Markdown content (headings, tables, code blocks, lists, etc.) totaling approximately **130KB**.
2. End with a clear task/instruction that triggers a **shell command** producing at least **20KB of output**.
3. The instruction must explicitly require the LLM to **analyze and summarize** the command output — forcing the model to process the full response.

**Purpose:** This creates an edge case where the combined input (~130KB file) plus tool response (~20KB+) exceeds the **144KB session limit**, allowing me to test how my ECA package handles session truncation/overflow.

first time eca server crashes second time i get

[OPENAI-CHAT] Failed to parse JSON arguments for tool 'eca__write_file': Unexpected end-of-input: expected close marker for Object (start marker at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 1, column: 1])
at [Source: REDACTED (StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION disabled); line: 1, column: 44]

if i can help/test/whatever contact me :-)

@snoopier

@snoopier I tested your prompt and it doesn't even get to 30% of context window :/

with this prompt: "read all files in this repo until the context window is blown, do not stop, no subagents", I can fulfill the context, but then the compact is triggered which is expected and everything keeps working.

Happened to me too with a long output coming from `clj -M:cljd test`` with abotu 2.2MB output of errors (ROFL).

This was the tool wich corrupted the sesion.

Tool: shell_command
Server: eca
Arguments: 
  command: "cd /home/devgiu/dev/lignum-app && clj -M:cljd test 2>&1"
  timeout: 300000

Output: Exit code: 1
Stdout:
=== Compiling cljd.core to Dart 3 ===

=== Compiling to Dart... === @08:47:40
  lignum.agenda
  lignum.agenda.core
  lignum.agenda.model
  lignum.core.document
  lignum.core.parser
  lignum.core.sample-data
  lignum.core.util
  lignum.denote.core
  lignum.index.core
  lignum.main
  lignum.storage.attachments
  lignum.storage.filesystem
  lignum.storage.permissions
  lignum.storage.settings
  lignum.ui.editing.text-ops
  lignum.ui.screens.agenda
  lignum.ui.screens.home
  lignum.ui.screens.library
  lignum.ui.screens.search
  lignum.ui.screens.settings
  lignum.ui.screens.viewer
  lignum.ui.theme
  lignum.ui.widgets.drawer
  lignum.ui.widgets.highlighter
  lignum.ui.widgets.org-render
  lignum.ui.widgets.util
INFO: isEmpty is a property and should be prefixed by a dash to work even when the type of the object is dynamic: -isEmpty  at line: 61, column: 24, file: lignum/ui/widgets/highlighter.cljd
DYNAMIC WARNING: can't resolve member path on target type dynamic of library dart:core  at line: 140, column: 16, file: lignum/ui/screens/library.cljd
DYNAMIC WARNING: can't resolve member path on target type dynamic of library dart:core  at line: 174, column: 8, file: lignum/ui/screens/library.cljd
  Bravissimo! 👏 @08:47:50

=== Running tests... ===
Resolving dependencies...
Downloading packages...
  _fe_analyzer_shared 91.0.0 (99.0.0 available)
  analyzer 8.4.1 (12.1.0 available)
  characters 1.4.0 (1.4.1 available)
  matcher 0.12.17 (0.12.19 available)
  material_color_utilities 0.11.1 (0.13.0 available)
  meta 1.17.0 (1.18.2 available)
  test 1.26.3 (1.31.0 available)
  test_api 0.7.7 (0.7.11 available)
  test_core 0.6.12 (0.6.17 available)
  vector_math 2.2.0 (2.3.0 available)
  win32 5.15.0 (6.0.1 available)
Got dependencies!
11 packages have newer versions incompatible with dependency constraints.
Try `flutter pub outdated` for more information.

00:00 +0: loading /home/devgiu/dev/lignum-app/test/cljd-out/lignum/ui/widgets-test_test.dart                                                                                                           test/cljd-out/lignum/ui/widgets-test_test.dart:7:8: Error: Error when reading 'lib/cljd-out/cljd/test.dart': No such file or directory
import "../../../../lib/cljd-out/cljd/test.dart" as lcoc_test;
       ^
test/cljd-out/lignum/denote/core-test_test.dart:7:8: Error: Error when reading 'lib/cljd-out/cljd/test.dart': No such file or directory
import "../../../../lib/cljd-out/cljd/test.dart" as lcoc_test;
       ^

This was the report from the AI.

Session permanently bricked by untruncated tool output (context window overflow)

Summary

When a tool (particularly shell_command) returns a very large output, the entire chat session becomes permanently unusable. The oversized output is stored in full in the session history and sent as context to the LLM API on every subsequent call, exceeding the model's context window. The session cannot recover — all future prompts fail silently or with API errors, and the only fix is to start a new conversation.

Steps to Reproduce

Start a chat session with any model (confirmed with z-ai/GLM-5.1)
Have the agent run a shell command that produces massive output (hundreds of thousands of characters). For example:
- clj -M:cljd test with missing generated test infrastructure (hundreds of repeated Dart compilation errors)
- flutter build with many compilation errors
- grep -r over a large codebase without limits
- npm install with a large dependency tree
- Any command that produces >100K chars of output
The tool output is stored in full in the session history
All subsequent API calls fail because the conversation context (history + system prompt + new prompt) exceeds the model's context window
User sends new messages — model never responds
Session is permanently dead

Observed Behavior

In my case, the agent ran clj -M:cljd test which produced ~2.2 MB (~560K estimated tokens) of Dart compilation errors (a missing lib/cljd-out/cljd/test.dart caused every generated test file to fail with repeated errors). This single tool output was stored verbatim in db.transit.json.

The model (GLM-5.1, 128K context window) never responded again. The user sent 4 follow-up messages — all ignored:

User: "Te has parado?"          → no response
User: "Debió haber algún problema" → no response  
User: "Qué ha pasado en las últimas calls?" → no response
User: "reintenta"               → no response

After switching to an OpenAI model to try to recover, the session produced a secondary error because the OpenAI Responses API rejected the UUID-format message IDs from the previous GLM responses (Expected an ID that begins with 'rs').

Root Cause

No output truncation: shell_command output is stored and sent to the LLM in full, regardless of size
No context overflow detection: eca does not detect or handle the case where the accumulated conversation history exceeds the model's context window
No session recovery: once the history is bloated, there is no mechanism to trim it or remove the offending messages
Silent failure: the API call fails (timeout, payload limit, or context error) but the user receives no error feedback — the model just "stops responding"

Evidence

From the session's db.transit.json:

Tool output size:         2,241,442 chars (~560K tokens)
Model context window:     128K tokens  
Overflow ratio:           4.4x the limit
Model responses after:    0

The tool output was a single shell_command result containing hundreds of repeated Dart compilation errors spanning 2.2 MB.

Suggested Fixes

Truncate tool outputs — Enforce a maximum size for tool outputs (e.g., 10K–50K chars). Truncate with a message like [output truncated: X chars omitted]. This should apply both at storage time and when building the API payload.
Context overflow detection — Before sending an API request, estimate the token count. If it exceeds the model's context window minus a safety margin, automatically trim or summarize older messages.
Automatic recovery — If an API call returns a context-length error, attempt to recover by:
- Truncating the largest tool outputs in the history
- Removing older messages from the conversation
- Notifying the user that context was trimmed
User feedback on failure — When the API call fails for any reason, display an error to the user instead of silently dropping the response.

Environment

eca version: latest (as of 2026-04-19)
Model: z-ai/GLM-5.1 (also confirmed secondary failure with openai/gpt-5.4-mini)
Project: ClojureDart/Flutter project with ~7 generated test files
OS: Linux

Impact

This is a session-ending bug with no recovery path other than abandoning the conversation. It can be triggered by any command that produces large output, which is a common occurrence during debugging and development workflows.

Looking for a way to reproduce it LLM told me to ask in session to run this:

# Option 1: slow command that builds up output gradually
find / -name "*.log" 2>/dev/null    # scans entire filesystem, takes minutes

# Option 2: simulate a long-running compilation
bash -c 'for i in $(seq 1 50000); do echo "file_$((i%50)).dart:$i: Error: Method not found: result_$i"; sleep 0.001; done'

But I was not able to reproduce it outside the scope of the project.
The only way I can reproduce it was asking to run directlyu the command.

Had similar issue here trying to summarize a huge pile of my email messages using Gmail-MCP. At some point ECA becomes unresponsive, sending any cue ticks for 4 seconds "Generating" and then just stops, doing nothing.

I'm most definitely hit by the problem when the agent gets a lot of output from its MCP calls — it becomes unresponsive and does not react to followup prompts. Perhaps deserves a separate issue though

Yeah, please open a new issue with more details and maybe a way to repro that

Auto-compact again when the context refills in the same turn

Navigation Menu

Repository navigation

ECA becomes unresponsive when tool output causes LLM context overflow #391

Root Cause Analysis

Server-side

Client-side (Emacs)

Steps to Reproduce

Session permanently bricked by untruncated tool output (context window overflow)

Summary

Steps to Reproduce

Observed Behavior

Root Cause

Evidence

Suggested Fixes

Environment

Impact

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ECA becomes unresponsive when tool output causes LLM context overflow #391

Description

Root Cause Analysis

Server-side

Client-side (Emacs)

Steps to Reproduce

Activity

ericdallo commented on Apr 4, 2026

snoopier commented on Apr 13, 2026

ericdallo commented on Apr 13, 2026

DevGiuDev commented on Apr 19, 2026

Session permanently bricked by untruncated tool output (context window overflow)

Summary

Steps to Reproduce

Observed Behavior

Root Cause

Evidence

Suggested Fixes

Environment

Impact

lockie commented on May 1, 2026

lockie commented on May 14, 2026

ericdallo commented on May 14, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions