Skip to content

feat(providers): add LLMTR as an OpenAI-compatible gateway - #1181

Open
knowhycodata wants to merge 1 commit into
EverMind-AI:mainfrom
knowhycodata:add-llmtr-provider
Open

knowhycodata wants to merge 1 commit into
EverMind-AI:mainfrom
knowhycodata:add-llmtr-provider

Conversation

@knowhycodata

Copy link
Copy Markdown

Summary

Adds LLMTR (https://llmtr.com) as a gateway provider. LLMTR serves other vendors' models, and a few of its own, behind one OpenAI-compatible API at https://llmtr.com/v1. Disclosure: I maintain LLMTR.

The change follows the Opper addition (#869):

  • raven/providers/registry.py: a ProviderSpec at the end of the resale gateway block. It uses OpenAI's driver, is identified by llmtr.com in the address, uses OPENAI_API_KEY as the driver's variable, and has no default model and no shortlist. strip_model_prefix stays off. A model LLMTR publishes as llmtr/<name> is stored as llmtr/llmtr/<name>, so that segment is sent upstream. LLMTR declares a chat address only. A family guess that would send a model to /v1/responses steps aside, so a chat-only id such as qwen/qwen3.8-27b-free stays on chat completions.
  • That chat-only address is native_api_bases=(("chat", ...),). Raven infers responses for qwen, deepseek, gpt, kimi and minimax ids, and a gateway with no declared addresses accepts that guess at its default base; many LLMTR models are chat-only (qwen/qwen3.8-27b-free returns a 400 on /v1/responses). It works the same way as for Moonshot and Groq. A user who sets protocol explicitly still gets what they set.
  • raven/config/schema.py: ProvidersConfig.llmtr.
  • raven/cli/onboard_commands.py: a row in the onboarding key group.
  • ui-web/src/assets/providers/llmtr.svg and ProviderMark.tsx: the vendor mark, hybrid tone (crescent in currentColor, star kept red), with a LICENSES/README.md entry.
  • Counts written elsewhere now say fifty-seven specs, twenty-three of them gateways (ui-web/CONTEXT.md, ProviderSide.tsx, Roles.tsx, the registry docstring and the gateway block comment).
  • Tests: the pinned provider set and count, the LiteLLM name tables, NO_SHORTLIST_VENDORS, tests/data/wire_model_baseline.json (only the new llmtr block), and llmtr added to test_an_inferred_native_protocol_steps_aside_where_the_provider_has_no_address_for_it.

Keys: https://llmtr.com/dashboard/api-keys

Runcrate (#1143) and Atlas Cloud (#1009) also move the counts to fifty-seven. Whichever lands first, I'll rebase and bump the numbers.

Type

  • Fix
  • Feature
  • Docs
  • CI / tooling
  • Refactor
  • Other

Verification

  • uv run --frozen --python 3.12 --all-extras pytest -q tests/test_provider_protocol.py tests/test_provider_catalog.py tests/test_provider_wire_model.py tests/test_litellm_provider_model_names.py: 839 passed, 7 skipped

  • pytest tests -k 'protocol or provider or registry or factory': the same set of failures as on main, nothing new.

  • ruff check raven tests: clean. ruff format --check on the touched Python files: already formatted.

  • Live check with a real key through Raven's own factory: Config with providers.llmtr.apiKey and model llmtr/qwen/qwen3.8-27b-free gives provider llmtr and get_api_base https://llmtr.com/v1. make_provider logs that the responses guess steps aside to chat and returns a LiteLLMProvider. One chat(..., max_tokens=32) call went to POST https://llmtr.com/v1/chat/completions with model qwen/qwen3.8-27b-free and returned LLMTR_OK (finish_reason stop, 20 prompt / 5 completion tokens, about 1.3 s). Before the native_api_bases line was added, the same call went to /v1/responses and got a 400.

  • Relevant tests pass locally

  • Relevant lint / type checks pass locally (ruff; ty and the full make ci not run)

  • User-facing docs or screenshots are updated when needed (no screenshot attached; the row shows up in Settings → Providers)

Risk

  • Security impact considered (no new code path; the key travels the same way as for the other OpenAI-driver gateways)
  • Backward compatibility considered (additive; no existing provider changes)
  • Rollback path is clear for risky changes (revert the commit)

Related Issues

Fixes #1180

LLMTR serves other vendors' models, and a few of its own, at
https://llmtr.com/v1. The entry matches the other resale gateways:
OpenAI's driver, the address in config, no default model, and the
published id left intact. A model published as llmtr/<name> is stored
as llmtr/llmtr/<name> so that leading segment is sent upstream.
@LivXue

LivXue commented Nov 4, 2026

Copy link
Copy Markdown
Member

Windows compatibility review — PR #1181

Reviewed head 0641738be644a4fa354e048f0577d0e262315293 against base f401dfccd3cca42dd33d58dbeb0857ec44d447bb (15 files, +69/−10). The full diff (raven/providers/registry.py, config/schema.py, cli/onboard_commands.py, the pinned test baselines, and the ui-web provider mark) was read.

Findings

No Windows-specific issues found.

The change is a provider registry entry for LLMTR plus its config field, onboarding row, pinned test counts/baselines, docs licence note, and an SVG asset. It adds no OS-specific API, path handling, subprocess, installer, or packaging surface; default_api_base/native_api_bases are ordinary HTTPS URLs, and the provider keeps its model prefix (strip_model_prefix=False) by design.

Verification (native Windows 11, isolated worktree)

  • Detached worktree D:/tmp/rv-1181 at the reviewed head; uv sync --frozen --offline --all-extras --no-extra segmenter --no-extra browser --dev succeeded.
  • uv run --no-sync pytest tests/test_provider_catalog.py tests/test_litellm_provider_model_names.py tests/test_provider_protocol.py tests/test_rpc_model.py -q → 751 passed, 8 skipped (the skips are the pre-existing "LiteLLM does not name an environment variable" cases); the updated 57-provider and wire-baseline assertions pass.
  • ui-web: ./node_modules/.bin/vitest run src/components/ProviderMark.test.tsx → 1 file, 15 tests passed (node_modules borrowed through a junction from D:/livxu/Documents/Raven/ui-web; package.json/package-lock.json byte-identical).
  • Gap: no live LLMTR endpoint was called (no credentials/network); provider behavior is verified through the registry/baseline tests only.

No issues to report; no changes requested.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add LLMTR as an OpenAI-compatible gateway provider

2 participants