Skip to content

GrowthBook experiment tengu_amber_redwood silently reduces autocompact window to ~80k on Sonnet 200k (v2.1.92+ regression) #46331

Closed as not planned

Description

@tavaresgmg

Summary

GrowthBook experiment tengu_amber_redwood silently reduces the autocompact window on Sonnet 4.6 (200k context) to an effective ~80k tokens, causing compaction at ~23-25% context usage. This is the same class of bug as #43989 (Opus 1M → 400k) but affecting standard 200k context models — a much wider user base.

The built-in notification jz7() that should warn users about experiment-driven compaction ("compacted at {X} · override with CLAUDE_CODE_AUTO_COMPACT_WINDOW=...") does not appear to fire or render visibly.

Evidence

Measured compaction events from JSONL session transcripts (v2.1.100)

Session Model preTokens % of native window Expected threshold
bb7608fc sonnet-4-6 48,275 24.1% of 200k 167k
bb7608fc sonnet-4-6 45,281 22.6% of 200k 167k
bb7608fc sonnet-4-6 46,771 23.4% of 200k 167k
02395a86 sonnet-4-6 50,206 25.1% of 200k 167k
904ab744 opus-4-6 75,414 7.5% of 1M 967k
904ab744 opus-4-6 47,353 4.7% of 1M 967k
904ab744 opus-4-6 50,231 5.0% of 1M 967k

Cluster: 6/7 events trigger at 45,281–50,231 tokens (mean: 47,886). All well below the expected threshold.

Reverse-engineering the experiment value

From the source code (deobfuscated from v2.1.100 binary):

autoCompactThreshold = contextWindow - maxOutputTokens(20k) - buffer(13k)

If threshold ≈ 47k → effectiveWindow ≈ 60k → experiment is setting window ≈ 80k.

For Sonnet 200k: 80k is a 60% reduction of the native context window.
For Opus 1M: the 400k reported in #43989 is also a 60% reduction — consistent ratio.

Window resolution priority (from binary)

function vt(model, settingsWindow) {
    let nativeWindow = getModelWindow(model);
    // 1. env var CLAUDE_CODE_AUTO_COMPACT_WINDOW → highest priority
    // 2. user settings (autoCompactWindow)
    // 3. GrowthBook "tengu_amber_redwood" ← the problem
    // 4. model native window ← never reached when enrolled
}

Notification that should fire but doesn't

function jz7(app, model, window) {
    let {source, configured} = vt(model, window);
    if (source !== "experiment") return;
    app.addNotification?.({
        key: "autocompact-experiment-hint",
        text: `compacted at ${configured} · override with CLAUDE_CODE_AUTO_COMPACT_WINDOW=1000000`,
        priority: "medium"
    });
}

This notification exists specifically for experiment-enrolled users but either doesn't render or is dismissed before it can be seen.

Steps to Reproduce

  1. Install Claude Code v2.1.100 on macOS arm64
  2. Use model sonnet-4-6 (standard 200k context, NOT the 1M variant)
  3. Have a conversation with 3-5 turns involving tool use (file reads, bash commands)
  4. Observe compaction triggers at ~45-50k tokens (~23% of 200k)
  5. Check JSONL transcript: compact_boundary events show preTokens ≈ 47-50k

Expected Behavior

Sonnet 4.6 with 200k context should compact at:

  • 200k - 20k(output) - 13k(buffer) = 167k tokens (~83.5% usage)

Actual Behavior

Compaction triggers at ~47-50k tokens (23-25% usage), wasting 75% of the context window.

Sessions become unusable: after compaction reduces to ~20-30k tokens, only 2-4 tool-use turns fit before the next compaction — creating a thrashing loop. In one session, 3 compactions occurred within 6 minutes.

Additional issues

  1. CLAUDE_AUTOCOMPACT_PCT_OVERRIDE cannot raise the threshold — the code uses Math.min(userPct, defaultThreshold), silently ignoring any value above default (CLAUDE_AUTOCOMPACT_PCT_OVERRIDE cannot raise threshold above default (~83%) #31806)
  2. DISABLE_AUTO_COMPACT=1 does not affect microcompacts — three additional silent compaction mechanisms (time-based microcompact, cached microcompact, tool-result budget via tengu_hawthorn_window) run independently
  3. No opt-out for experiments — users cannot inspect or disable GrowthBook flags without reverse-engineering the binary. The experiment silently overrides model capabilities that users are paying for.

Workaround

Add to ~/.claude/settings.json:

{
  "env": {
    "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "1000000"
  }
}

This takes priority position 1 in vt(), overriding the experiment. Note: reportedly does not work on all arm64 native builds (#43989 comment by @creatornader).

Environment

Related issues

Activity

  1. github-actions commented on Apr 10, 2026

    @github-actions

    Found 2 possible duplicate issues:

    1. v2.1.92 regression: autocompact threshold reduced to 400k on Opus 4.6 (1M context) #43989
    2. Context compaction triggers at ~16% usage on Opus 4.6 (1M context) #34254

    This issue will be automatically closed as a duplicate in 3 days.

    • If your issue is a duplicate, please close it and 👍 the existing issue instead
    • To prevent auto-closure, add a comment or 👎 this comment

    🤖 Generated with Claude Code

  2. tavaresgmg commented on Apr 10, 2026

    @tavaresgmg
    Author

    Update: env var workaround is insufficient — two additional findings

    Finding 1: CLAUDE_CODE_AUTO_COMPACT_WINDOW in the env block does not survive compaction cycles

    After applying the workaround (CLAUDE_CODE_AUTO_COMPACT_WINDOW=200000 in settings.json env block), compaction still reverted to the ~47k threshold within the same session.

    Observed from JSONL (session bb7608fc):

    Compaction # Timestamp (UTC) preTokens Status
    4 17:26:56 121,805 ✅ env var took effect
    5 17:30:59 45,010 ❌ reverted to experiment threshold

    Compaction 4 shows the env var working (121k instead of ~47k). But 4 minutes later, compaction 5 triggered at 45k — back to the experiment threshold.

    Root cause hypothesis: The env block in settings.json is injected into the process environment at startup only. When a compaction event triggers an internal context/sub-session reset, the new context re-evaluates GrowthBook from scratch without the env var in scope — because the env injection happened in the parent process, not re-read from the JSON at runtime.

    Finding 2: Top-level autoCompactWindow field is the correct fix

    The window resolution chain in the binary (priority order):

    1. CLAUDE_CODE_AUTO_COMPACT_WINDOW env var — injected at process startup only
    2. autoCompactWindow (top-level in settings.json) — read directly from JSON at runtime
    3. GrowthBook experiment tengu_amber_redwood
    4. Model native window

    The top-level autoCompactWindow field bypasses the startup-injection problem because it's parsed from the settings file on each read, not from the process environment.

    Correct workaround (updated)

    Add both to ~/.claude/settings.json:

    {
      "autoCompactWindow": 200000,
      "env": {
        "CLAUDE_CODE_AUTO_COMPACT_WINDOW": "200000"
      }
    }

    The autoCompactWindow top-level field (priority 2) is the reliable fix. The env block entry alone (priority 1 in theory, but startup-only in practice) is insufficient for sessions that involve multiple compaction cycles.

    After applying this, restart Claude Code completely (Cmd+Q + reopen) — not just a new tab or session.

    Note: this issue may still close as a duplicate of #43989 — thumbed-down the bot comment to prevent auto-close.

  3. junaidtitan commented on Apr 23, 2026

    @junaidtitan

    A silent experiment reducing your effective compaction window to 80k is especially painful because you lose context you thought you had. Cozempic's guard daemon prunes session bloat continuously at 4 configurable thresholds, so even if the internal autocompact window is artificially low, your actual useful context stays high. It also detects 1M context automatically for supported models. pip install cozempic && cozempic init https://github.com/Ruya-AI/cozempic

  4. github-actions commented on May 29, 2026

    @github-actions

    Closing for now — inactive for too long. Please open a new issue if this is still relevant.

  5. github-actions commented on Jul 28, 2026

    @github-actions

    This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.

  6. locked as resolved and limited conversation to collaborators on Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:corebugSomething isn't workinghas reproHas detailed reproduction stepsplatform:macosIssue specifically occurs on macOSregressionstaleIssue is inactive

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions