Skip to content

session/processor: text/reasoning delta accumulation is O(N²); long agent loops hang at 50-80 turns #30067

Closed

Description

@meitalbensinai

Description

When I ran opencode in headless batch mode (opencode run --format json) against MiniMax-M2.7 on a long-horizon agent loop, runs that went past ~50 turns started slowing down dramatically per step, eventually hanging at the 2-hour wall-clock cap I had set. I expected per-step time to stay roughly flat, but it was growing: 6s → 30s → 60s → 100s+ per step as the session got longer.

When I attached perf record to the running bun process I saw something weird:

  • __memmove_avx_unaligned_erms in libc.so.6 was 30.27% of CPU samples
  • All 7 JSC HeapHelper GC threads were pinned at ~4% CPU each (normally idle)
  • Main JS thread at 85% CPU, state R, but no syscalls — pure userspace burn
  • All Bun Pool and HTTP Client threads idle at 0%

That __memmove pattern is the giveaway for V8/JSC rope-flattening on str += chunk. I tracked it down to two spots in packages/opencode/src/session/processor.ts:

  • reasoning-delta case: ctx.reasoningMap[value.id].text += value.text
  • text-delta case: ctx.currentText.text += value.text

The issue is that anything reading .text between writes (the UI render loop, the NDJSON serializer, the bus event broadcaster) forces V8/JSC to flatten the rope. Then the next += re-copies all the prior text — O(N²) in cumulative length per part. For thinking-mode models emitting 1500+ small reasoning tokens per turn, this destroys the agent loop after enough turns accumulate.

This is the same shape as the bug that was fixed in @ai-sdk/mcp in vercel/ai#14619 and that I just landed for vercel/ai#15669 in processUIMessageStream + DefaultStreamTextResult. But opencode's processor has its own second copy of the pattern, so even with the AI SDK fix, opencode still hangs.

The bug is not the same as #25094. That issue is about the TUI not showing a "thinking…" indicator during long reasoning blocks (a UX/visibility issue) and is observed in interactive sessions. This issue is an algorithmic O(N²) in the processor that affects every mode of opencode — headless batch, TUI, server — and is independent of whether thinking is shown to the user.

Distinct from related issues I checked:

Effect when patched (with the matching vercel/ai#15669 fix applied as well):

Run config per-step (avg) 2h timeouts
unpatched 61 s 5 of 18 (28%)
patched 21 s 0 of 53

Plugins

@tabnine/tabnine-provider (Tabnine LLM gateway provider). The bug reproduces regardless of provider — same pattern hit on local vLLM, OpenRouter, and Anthropic direct.

OpenCode version

1.15.11 (also confirmed on 1.15.12 install path)

Steps to reproduce

  1. Run a long agent loop with a thinking-mode model that emits many small reasoning deltas (MiniMax M2.x, DeepSeek-V4-thinking, Kimi K2, GPT-o, Qwen3-thinking — Claude extended-thinking is less severe but also affected)
  2. Drive it through 50+ turns (SWE-bench batch eval is an easy way to hit this naturally)
  3. Watch per-step latency grow within the run
  4. Attach perf record -F 99 -g -p <bun_pid> -- sleep 60 to a hung process and you'll see __memmove_avx_unaligned_erms dominating

Operating system

Debian 12 (in container; symptom reproduces on host too)

Terminal

Headless — opencode run --format json invoked from a Python batch runner. Symptom does not require a TUI.

Activity

  1. meitalbensinai commented on May 31, 2026

    @meitalbensinai
    Author

    Submitted PR with the fix: #30072

    (An earlier attempt — PR #30058 — was auto-closed by the contributing-guideline bot before my template/footer fixes could be re-validated within the 2-hour window. #30072 is the same branch + commits, resubmitted with all bot checks passing from the start.)

  2. AdityaPagare619 commented on Jun 23, 2026

    @AdityaPagare619

    I saw this firsthand on v1.15.7. Was running a 100-step sequential thinking
    review and the per-step latency went from ~6s to 100s+ by step 50. The
    session eventually hung.

    The reasoning-delta handler was the bottleneck — each small token addition
    triggered V8 string flattening because the UI render loop reads the text
    between writes.

    Haven't tested the fix yet but the array accumulator approach makes sense.

  3. anmarui74 commented on Aug 19, 2026

    @anmarui74

    Affected here on v1.18.18. I can reproduce the progressive per-turn slowdown with DeepSeek V4 Flash (OpenCode Go) and Qwen 3.5 local: sessions start fast and degrade to a crawl after many turns with long reasoning blocks — the agent appears frozen emitting one word per second. PR #30072 has the fix but was closed without merging. Could it be reopened/merged? This one and #36037/#36043 are making long agentic sessions painful. Thanks!

  4. neriousy commented on Oct 1, 2026

    @neriousy
    Member

    The OpenCode v2 session runner no longer accumulates deltas this way. Text and reasoning deltas are collected in an array and joined once when the part finishes, and the deltas themselves are ephemeral events that aren't persisted one by one. Long agent loops shouldn't slow down quadratically anymore.

    Changed in #35272.

    Closing as completed. If long sessions still slow down step by step on v2, please open a new issue, ideally with a profile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions