Skip to content

Warn when a cost limit cannot be enforced for an unpriced model - #40

Merged
vicgupta merged 1 commit into
mainfrom
issue-31-warn-unpriced-cost-limit
Oct 24, 2026
Merged

vicgupta merged 1 commit into
mainfrom
issue-31-warn-unpriced-cost-limit

Conversation

@vicgupta

Copy link
Copy Markdown
Owner

Fixes #31

What changed

  • internal/model/provider.go:
    • The longest-prefix price lookup moved out of cachedPrice into lookupPrice. Pricing works the same as before.
    • New Unpriced(spec) returns true only when the provider is openai, openrouter or anthropic (including an implicit openai), the model has no Prices entry, and PriceOverride is zero. It uses the same override rule as cost calculation, so any non-zero input or output override counts as priced.
    • New splitSpec holds the "/" parsing that New already did, so New and Unpriced resolve providers the same way.
  • cmd/tinyagent/main.go: new warnUnpriced runs after config and flags are resolved and model.New succeeds, and before the run starts. When the effective cost limit is above 0 and the model is unpriced, it writes one line to stderr:
    warning: no pricing available for openai/unlisted-model; --cost-limit cannot be enforced. Set input_price_per_mtok and output_price_per_mtok in your config to enable cost tracking.
    It doesn't depend on --verbose, and stdout, the trajectory, requests and the exit status don't change.
  • README: the cost-limit note now describes the warning.

Testing

Go 1.27.1 from go.mod couldn't be downloaded here (proxy.golang.org returned Forbidden). I ran everything with the preinstalled Go 1.25.1 in a scratch copy of the repo. In that copy I set go 1.25 in go.mod and added a local replace pointing gopkg.in/yaml.v3 at a clone of go-yaml/yaml tag v3.0.1. Those go.mod changes stayed in the scratch copy and aren't committed.

  • Tests first. I added TestUnpriced (model), TestUnpricedCostLimitWarning and TestNoWarningForFreeProviders (cmd). They didn't build at first (undefined: Unpriced / warnUnpriced). Next I added no-op stubs, and the assertions failed: 4 Unpriced cases and 3 warning cases. Then I wrote the implementation.
    • TestUnpricedCostLimitWarning calls run() against an httptest fake OpenAI server. It covers a --cost-limit flag, the implicit openai provider, a config cost_limit, --cost-limit 0 overriding a config limit, no limit, a priced model, and a price override. Each run checks for exactly one warning or none, that the warning comes before everything else on stderr, that nothing goes to stdout, exit 0, and exactly 1 request.
  • gofmt -l .: no output
  • go vet ./...: clean
  • go test -race -count=1 ./...: all packages pass
  • End to end. I built the binary and ran it against a local fake OpenAI-compatible server on 127.0.0.1 (OPENAI_BASE_URL=http://127.0.0.1:18766 OPENAI_API_KEY=fake --no-stream):
    • Exactly one warning, exit 0, 1 request each: -m openai/fake --cost-limit 1, -m fake --cost-limit 1, and -m openai/fake with config cost_limit: 2.0.
    • No warning, exit 0: config cost_limit: 2.0 plus --cost-limit 0; no limit; -m openai/gpt-4o with a config limit; and a config with output_price_per_mtok: 3.
    • The -o trajectory JSON doesn't contain the warning.
    • ollama/x --cost-limit 1: no warning. Its request went to localhost, where nothing was listening.
    • codex/x --cost-limit 1: no warning. It stopped first on the missing Codex login.
    • anthropic/claude-2 --cost-limit 1: printed the warning. I left ANTHROPIC_BASE_URL unset by mistake, so this run also sent one request to api.anthropic.com with no API key, and it failed (exit 1). That's unrelated to this change.

Generated by Claude Code

With a positive effective cost limit (from --cost-limit or the config) and
an openai, openrouter or anthropic model that has no price table entry and
no price override, print one warning on stderr before the first request.
The run continues unchanged. Ollama and Codex are free by design and never
warn.

model.Unpriced reuses the same longest-prefix lookup and override rules as
cost calculation, so the warning agrees with the cost actually reported.

Fixes #31

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Warn when a cost limit cannot be enforced for an unpriced model

2 participants