Skip to content

pi_models_json.cjs registers aw-gateway models with no contextWindow, so pi falls back to a 128k default well below the real model's capacity #63240

Closed

Description

@ekbritecore

Analysis

actions/setup/js/pi_models_json.cjs generates the models.json that
registers pi's aw-gateway custom provider (the bridge every engine: pi
workflow routes its inference calls through). buildModelsJSON() writes
a bare model entry with no metadata at all:

providers: {
  "aw-gateway": {
    baseUrl, api: api || "openai-completions", apiKey: apiKeyEnvVar,
    models: [{ id: modelId }],
  },
},

No contextWindow field. Per pi's own docs (custom-provider.md,
extensions.md), an unspecified context_window on a custom-provider
model falls back to a generic default of 128000. That default is
usually far short of the real underlying model's actual context length,
and pi has no way to know better — the same model ID under the real,
recognized github-copilot provider (as when running pi directly, not
through gh-aw) already carries its correct spec in pi's own built-in
model catalog. Confirmed for real: claude-sonnet-5 is registered
natively as "contextWindow": 1000000 under provider: "github-copilot"
— but gh-aw's aw-gateway provider is an unrecognized custom identity to
pi (different provider name, different api value), so none of that
built-in metadata carries over; it silently gets the generic 128k
fallback instead.

This matters because pi auto-compacts conversation history once
contextTokens > contextWindow - reserveTokens (default reserveTokens
16384) — i.e. at ~111,616 tokens under the current 128k fallback,
regardless of what the real underlying model can actually hold. A
128000 cap is roughly 1/8th of what this specific model can really do,
so any sufficiently long-running engine: pi workflow compacts away
early context far sooner than necessary, independent of whatever that
context happened to contain.

Reproduction / Impact

Found while investigating #63239 (a create_pull_request safe-output
call that got lost after a run's context was compacted). That issue's
proximate trigger was the model-specific tool-usage instructions being
compacted away — but the reason compaction fired at all, this early, in
a run that was otherwise well within the real model's actual capacity,
traces back to this gap. It's independent of #63239's prompt-wording
problem though: any engine: pi workflow that runs long enough will hit
this same premature-compaction ceiling regardless of whether it uses
safe-outputs, and regardless of whether #63239 is fixed.

Confirmed via a real run's raw stream: tokensBefore: 111952 at
compaction, matching 128000 - 16384 = 111616 within normal token-count
estimation variance.

Implementation Plan

Add contextWindow to the model entry buildModelsJSON() constructs,
using the real value for the resolved model (via whatever spec source
gh-aw already uses to resolve/validate model IDs elsewhere in the
compiler, e.g. the model alias/spec validation in
model_identifier.go/model_alias_validation.go, if that data is
available at the point this script runs) rather than leaving it
unspecified and inheriting pi's generic 128k default. At minimum, for
claude-sonnet-5 specifically, 1000000 is the value pi's own built-in
catalog already carries for the same model under its native provider.

Related: #63239 (found while investigating that issue; independently
reproducible and fixable, hence filed separately rather than folded in).

Activity

  1. locked and limited conversation to collaborators on Sep 24, 2026
  2. unlocked this conversation on Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions