Skip to content

DuckDB helper crashes lose their cause and are reported as a "row cap clamp" in sql_materialize #1494

Open

Description

@anandgupta42

Problem

When the DuckDB helper process dies (typically out of memory on a heavy join), the tool reports only "helper process exited unexpectedly"; in sql_materialize the crash occurs during the pre-flight count and is presented as a cap problem.

Mechanism

  • The helper is spawned with stderr: "ignore" (packages/drivers/src/local-engine.ts:416); the exit handler reports only "helper process exited unexpectedly" (:438) without exit code or signal.
  • In sql_materialize the failed count falls into "Pre-flight estimate unavailable (...); extraction row cap clamped to 200,000 rows" (packages/opencode/src/altimate/tools/sql-materialize.ts:321-328).
  • No DuckDB memory_limit or temp_directory is set.

Evidence

7 crashes in 7 sessions, each after 32-150 s (about 460 s total), all from json_each over a 277,813-row scratch table with correlated EXISTS/LIKE. Scratch tables survived (the file database reopened), and each crash restarted the helper.

Impact

About 460 s of wasted time; the model cannot tell OOM from a bad query, and the misleading cap message hides the cause.

Proposed fix

  • Capture helper stderr and exit code/signal; report "DuckDB helper killed (signal / out of memory)".
  • Configure memory_limit and temp_directory sized to the host.
  • Never describe a crash as a cap clamp; propagate the crash error.

Acceptance check

Killing the helper mid-query (SIGKILL) yields an error naming the signal; a pre-flight crash in sql_materialize surfaces as a crash, not a cap clamp.

Activity

  1. devin-ai-integration commented on Oct 20, 2026

    @devin-ai-integration

    This looks fixed on main. #1521 (Closes #1494) merged into feat/analyst-harness-dab-2026-10, not main, so the issue didn't auto-close. That branch landed on main via #1441 (squash commit b12bce3, 2026-10-16).

    I checked the files #1521 changed against main: 212 of its 212 substantive added lines are on main as written.

    Suggest closing.

    Written by Devin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions