Repository navigation
Description
Analysis
actions/setup/js/pi_models_json.cjs generates the models.json that
registers pi's aw-gateway custom provider (the bridge every engine: pi
workflow routes its inference calls through). buildModelsJSON() writes
a bare model entry with no metadata at all:
providers: {
"aw-gateway": {
baseUrl, api: api || "openai-completions", apiKey: apiKeyEnvVar,
models: [{ id: modelId }],
},
},No contextWindow field. Per pi's own docs (custom-provider.md,
extensions.md), an unspecified context_window on a custom-provider
model falls back to a generic default of 128000. That default is
usually far short of the real underlying model's actual context length,
and pi has no way to know better — the same model ID under the real,
recognized github-copilot provider (as when running pi directly, not
through gh-aw) already carries its correct spec in pi's own built-in
model catalog. Confirmed for real: claude-sonnet-5 is registered
natively as "contextWindow": 1000000 under provider: "github-copilot"
— but gh-aw's aw-gateway provider is an unrecognized custom identity to
pi (different provider name, different api value), so none of that
built-in metadata carries over; it silently gets the generic 128k
fallback instead.
This matters because pi auto-compacts conversation history once
contextTokens > contextWindow - reserveTokens (default reserveTokens
16384) — i.e. at ~111,616 tokens under the current 128k fallback,
regardless of what the real underlying model can actually hold. A
128000 cap is roughly 1/8th of what this specific model can really do,
so any sufficiently long-running engine: pi workflow compacts away
early context far sooner than necessary, independent of whatever that
context happened to contain.
Reproduction / Impact
Found while investigating #63239 (a create_pull_request safe-output
call that got lost after a run's context was compacted). That issue's
proximate trigger was the model-specific tool-usage instructions being
compacted away — but the reason compaction fired at all, this early, in
a run that was otherwise well within the real model's actual capacity,
traces back to this gap. It's independent of #63239's prompt-wording
problem though: any engine: pi workflow that runs long enough will hit
this same premature-compaction ceiling regardless of whether it uses
safe-outputs, and regardless of whether #63239 is fixed.
Confirmed via a real run's raw stream: tokensBefore: 111952 at
compaction, matching 128000 - 16384 = 111616 within normal token-count
estimation variance.
Implementation Plan
Add contextWindow to the model entry buildModelsJSON() constructs,
using the real value for the resolved model (via whatever spec source
gh-aw already uses to resolve/validate model IDs elsewhere in the
compiler, e.g. the model alias/spec validation in
model_identifier.go/model_alias_validation.go, if that data is
available at the point this script runs) rather than leaving it
unspecified and inheriting pi's generic 128k default. At minimum, for
claude-sonnet-5 specifically, 1000000 is the value pi's own built-in
catalog already carries for the same model under its native provider.
Related: #63239 (found while investigating that issue; independently
reproducible and fixable, hence filed separately rather than folded in).
Activity