Skip to content

Claude engine on Copilot inference compiles tools.web-fetch / web-search, but they are unavailable at run time (COPILOT_OFFLINE=true) #73079

Closed

Description

@kdowswell

Summary

The compiler rejects tools.web-fetch and tools.web-search for engine: copilot because offline BYOK mode disables native web tools. A Claude-engine workflow on Copilot inference (engine.model-provider: github) runs through the same offline proxy, and AWF logs:

[WARN] Copilot API proxy mode sets COPILOT_OFFLINE=true: native web_fetch and web_search tools are unavailable, even for allowlisted domains.

But the workflow compiles without a word. A workflow that depends on fetching upstream release notes, for example, loses that ability silently when its provider is switched to Copilot.

Root cause

pkg/workflow/agent_validation.go:257 validateCopilotWebToolsSupport returns early unless engine.GetID() is copilot. The Claude engine with a GitHub provider (claudeUsesCopilotInference in pkg/workflow/claude_copilot_model_overrides.go) never reaches the check, and validateWebSearchSupport treats Claude as supporting web search.

Implementation plan

  1. In pkg/workflow/agent_validation.go, extend validateCopilotWebToolsSupport, or add a sibling function, to cover engine.id == claude when the resolved LLM provider is GitHub. Use the same message pattern, plus the suggestion "use engine.model-provider: anthropic, or an MCP fetch server".
  2. Treat it as an error, consistent with the Copilot engine. Alternatively, make it a warning and drop WebFetch/WebSearch from Claude's allowed tools.
  3. Tests in pkg/workflow/copilot_web_tools_validation_test.go:
    • claude + github provider + web-fetch → error;
    • claude + anthropic provider → OK.
  4. Docs: in reference/engines/claude.md, under "GitHub Copilot inference", and in the feature comparison in reference/engines.md, note that web tools are unavailable on Copilot inference.

Acceptance criteria

gh aw compile fails, or at least warns, for engine: claude, model-provider: github with tools.web-fetch or tools.web-search.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions