Skip to content

[awf] API Proxy: Copilot CLI model validation fails with classic PAT when COPILOT_MODEL is set #2010

Closed

Description

@lpcox

Problem

Copilot CLI 1.0.21 (shipped in gh-aw v0.67.3) added a startup model validation step: when COPILOT_MODEL is set, the CLI calls GET /models to validate the model name before execution. This endpoint rejects classic PATs (ghp_* tokens), causing the agent step to fail with exit code 1 immediately on startup.

Context

Original report: github/gh-aw#25593

  • Affected: Workflows that set COPILOT_MODEL and use a classic PAT as COPILOT_GITHUB_TOKEN.
  • Unaffected: Workflows without COPILOT_MODEL; workflows using OAuth or fine-grained tokens.
  • Regression: Introduced in gh-aw v0.67.3 (Copilot CLI 1.0.21); v0.67.2 (CLI 1.0.20) was not affected.

The AWF api-proxy sidecar already has special handling for Copilot GET /models (see containers/api-proxy/server.js:891-914), which routes using COPILOT_GITHUB_TOKEN to api.githubcopilot.com/models. However, a classic PAT is not accepted by the Copilot inference API /models endpoint regardless of proxying.

Root Cause

The api-proxy correctly proxies GET /models to api.githubcopilot.com/models using the Copilot OAuth token. The issue is that when only a classic PAT is available (no OAuth exchange), the /models endpoint returns an error. The proxy cannot synthesize an OAuth token from a classic PAT.

Additionally, the note in stored memory confirms: "Copilot BYOK/offline workflows must set COPILOT_MODEL explicitly to avoid Copilot CLI BYOK startup failures" — this is now in tension with classic PAT workflows that also set COPILOT_MODEL.

Proposed Solution

  1. Intercept GET /models in the api-proxy and return a synthetic response (containers/api-proxy/server.js): When COPILOT_GITHUB_TOKEN is a classic PAT (detected by ghp_ prefix), return a mock 200 response with a hardcoded models list, bypassing the upstream validation call. This maintains backward compatibility without requiring token type changes.
  2. Document the token type requirement: Add a warning in the workflow template or docs when COPILOT_MODEL is set but the token is a classic PAT.
  3. Alternative: Detect classic PAT in src/docker-manager.ts and automatically unset COPILOT_MODEL when the token cannot support model validation, falling back to the default model (with a logged warning).

Generated by Firewall Issue Dispatcher · ● 2.1M · ◷

Activity

  1. github-actions commented on Apr 16, 2026

    @github-actions
    Contributor

    👋 Hello! I noticed this issue might be related to existing issues:

    If one of these addresses your concern, please consider closing this issue as a duplicate. Otherwise, feel free to clarify how your issue differs!

    This is an automated message from the issue duplication detector.

    Generated by Issue Duplication Detector for issue #2010 · ● 235.7K · ◷

  2. lpcox commented on Apr 16, 2026

    @lpcox
    CollaboratorAuthor

    Addressed by #2006 (explicit model for Copilot BYOK smoke test) and #2018 (config ingestion with COPILOT_MODEL fallback). Merged to main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions