Skip to content

LLM retry has no max attempts: stream errors cause infinite retry loop, UI stuck on Thinking #41848

Closed

Description

@teran-netizen

Summary

LLM retry has no maximum attempt limit. RETRY_MAX_DELAY is set to 2147483647ms (~24 days). When DeepSeek returns a stream error, the retry loop continues indefinitely — the UI shows "Thinking..." forever with no error feedback.

Evidence

  • 5 processes died on llm runtime selected immediately after stream error
  • RETRY_MAX_DELAY = 2147483647 (should be ~30 seconds for typical API retries)
  • No RETRY_MAX_ATTEMPTS constant exists
  • retryable() matches stream errors against patterns like /stream/ and /error/ — always retryable
  • RETRY_MAX_DELAY_NO_HEADERS = 30000 exists but only applies when no response headers present
  • With response headers, cap is full RETRY_MAX_DELAY (24 days)

Code location

File: retry.ts (bundled in main chunk)

RETRY_MAX_DELAY = 2147483647  // ~24 days — effectively unlimited
RETRY_MAX_DELAY_NO_HEADERS = 30000  // only used without headers
// RETRY_MAX_ATTEMPTS is MISSING

function policy2(opts) {
  return fromStepWithMetadata(succeed((meta3) => {
    const error50 = opts.parse(meta3.input);
    const retry6 = retryable(error50, opts.provider);
    if (!retry6) return done(meta3.attempt);  // only stops if NOT retryable
    // continues forever otherwise
  }));
}

Expected behavior

After N attempts (e.g., 3-5), emit a providerError event and stop — let the UI show an error instead of eternal "Thinking..."

Version: 1.18.16, Electron 42.3.3

Activity

  1. github-actions commented on Aug 11, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

    Please review the above issues and consider adding any new information there rather than opening a separate report.

  2. martialebeau commented on Aug 12, 2026

    @martialebeau

    Corroboration — MCP error triggers infinite retry, Windows 1.18.16\n\n### Evidence from incident 2026-08-05 (PID 39476)\n\nWe observed a zombie process that spun for 3 days at ~136% CPU (≈39 CPU-hours, 4.5GB RAM) with zero network, zero disk writes, zero log output. The trigger was an MCP server error:\n\n\nnotebooklm MCP error: -32601: Method not found\n→ session boot + dispose within 14s\n→ server agent loop stayed "active" but unsettled\n→ compute-bound spin with no I/O for 3 days\n\n\n### The -32601 error is NOT transient — but retry treats it as retryable\n\nA JSON-RPC -32601: Method not found is a protocol-level permanent error (the method does not exist). A correct client MUST NOT retry it — it\u0027s not transient and retrying will never succeed. However, given the broad retryable() pattern matching (/stream/, /error/), this error likely matches and enters the infinite retry loop.\n\nThis confirms the original report\u0027s finding: retryable() is too permissive and with no RETRY_MAX_ATTEMPTS, any error coded as retryable spins forever.\n\n### Context-size amplifier\n\nWe also found that larger opening context increases the probability of triggering this bug: with ~43K tokens of opening context (vs ~18K before), API streams are 2.5x longer, increasing the window for mid-stream errors (timeouts, rate limits, connection drops) that trigger the unbounded retry.\n\n### WAL bloat as post-mortem indicator\n\nDuring the zombie\u0027s 3-day spin, the opencode SQLite WAL file grew to 596MB (normal is \u003c10MB). This is a fingerprint of the runaway DB write loop inside the retry spin. WAL size could serve as a secondary detection signal — a process with WAL \u003e 100MB is almost certainly in a retry spin.\n\nVersion: 1.18.16, Windows 11

  3. martialebeau commented on Aug 12, 2026

    @martialebeau

    Addendum from a Windows environment (opencode v1.18.16 / Electron, native Windows Terminal via ConPTY) — 4 incidents over 6 days, supporting the unbounded-retry diagnosis and linking it to the sibling #41806.

    Corroborating evidence:

    Windows-specific symptom (new): the hung/dead server leaves a torrent of ANSI SGR codes printed as literal text in the terminal scrollback (e.g. [555;40;30M strings with the ESC byte stripped — a Mode-2 TUI render loop residue, confirmed by OCR of a capture and a clean router-stdout audit — not MCP output). This is a strong post-mortem fingerprint on Windows/ConPTY.

    Storage collateral: the zombie server leaves a runaway SQLite WAL — our opencode.db reached 13.3 GB (checkpoint starvation). Worth confirming the bundled SQLite version is ≥ 3.51.3 (the 2026 WAL-reset fix); ours could not be verified from the shipped bundle.

    Request: (1) a RETRY_MAX_ATTEMPTS cap (3–5) so a persistent stream error surfaces as a providerError instead of eternal "Thinking…"; (2) confirmation that the exit-handler race (#41806) settles on fast/immediate git-child exits — on Windows we see the same un-reaped-child pattern under index.lock contention from concurrent instances.

    Happy to provide an instrumented build pull, captured raw bytes from a live repro (to disambiguate the ESC-strip: console ENABLE_VIRTUAL_TERMINAL_PROCESSING flag vs. a zombie duplicate writer), or a full opencode.log + ps snapshot on request.

    Continuation of our prior corroboration (4 Windows incidents, context trigger, WAL fingerprint, ANSI flood symptom).

  4. ImperioCivek commented on Sep 23, 2026

    @ImperioCivek

    Independent confirmation on 1.18.32 (Linux, npm opencode-ai@1.18.32).

    Setup: OpenCode pointed at a self-hosted OpenAI-compatible gateway (multi-provider failover chain). When every upstream provider refused the content (provider-side moderation), the gateway initially returned a generic overload-style error and OpenCode re-issued the identical request indefinitely — no max attempts, turn stuck with no terminal error.

    Gateway-side workaround we had to add: return HTTP 400 with a deliberately non-transient body (code: content_filter_rejected, avoiding any exhausted/overload substring) so the client would stop. The fact that a gateway must signal "do not retry" via body fingerprinting is the symptom of the missing client-side bound.

    Suggestion (matches the report): a RETRY_MAX_ATTEMPTS (e.g. 3-5) plus emitting a terminal providerError/finish_reason after N attempts, with RETRY_MAX_DELAY capped to a human timescale (the current 2147483647 ≈ 24 days is effectively unlimited). Also treat 4xx content-policy/terminal errors as non-retryable. Related: #17648.

  5. rekram1-node commented on Oct 2, 2026

    @rekram1-node
    Collaborator

    Fixed in #41939, available in v1.18.17.

    Also recommend updating to: https://opencode.ai/v2/docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions