[Astro-Han]

Summary

Refs #860 (Phase 7: contract foundation; the issue stays open).

Provider conformance today is hand-enumerated per provider, so ready providers can silently lack coverage (anthropic, openai, google, deepseek and others have no cell in provider-conformance.test.ts) and every new provider must remember to add its own. This PR makes conformance an interpretation of the registry declarations instead:

• A new pure module packages/core/src/provider-contract-matrix.ts derives a matrix plan from providerRegistry: 42 rows (every status === 'ready' entry with a wired runtime adapter) × 4 dimensions (discovery, exact-model-id, tool-loop, reasoning-replay). Each cell resolves to exactly one of generated (145: expectations recoverable from protocol / runtimeAdapter / modelDiscovery), override (8: provider-specific contracts owned by named hand-written tests under stable provider:dimension keys), or not-applicable (15: machine-readable reasons; fallback-discovery providers still execute a reverse assertion that /models is never requested).
• A new runtime suite executes every generated cell against scripted local HTTP servers (openai-chat ×34, anthropic-messages ×5, google-generate ×1, cohere-v2 ×1) and fails CI when a ready provider's applicable dimension has neither a generated cell nor a registered override, naming the provider id and missing dimension. Future ready providers enter the matrix by declaration alone.

The row set is intentionally not READY_PROVIDER_TYPES: its membership is "has a readyOrder", which would silently drop github-copilot.

Per the Phase 7 constraints in #860 the change is additive only: no runtime behavior, existing suite, READY_PROVIDER_TYPES consumer, or compatibility path changes. Overlap with the hand-written provider-conformance.test.ts is intentional; consolidation belongs to Phase 8.

Verification

• @maka/core: 968/968 pass, typecheck clean (includes the new 19-test plan-derivation suite).
• @maka/runtime: 1691 pass / 0 fail, 7 pre-existing skips, typecheck clean (includes the new 89-test generated-conformance suite).
• Both run after rebasing onto latest main.
• Not run locally: Desktop E2E (deferred to the CI gate per feat(providers): support common providers from models.dev #860); no desktop surface is touched.
• All coverage is deterministic against scripted local servers; no secrets, credentials, or raw provider responses.

Review focus

sampleModelIdFor knowingly duplicates the runtime's /^gpt-5/i Responses-routing fact (commented at the source). Lifting it into an apiProtocol declaration is deferred to a follow-up PR: the regex is an open-world pattern (a newly discovered gpt-5.x id still routes to Responses) while apiProtocol is declared per exact model id today, so a naive lift would regress routing for newly discovered models. The follow-up needs a pattern-level declaration seam; both duplication sites disappear with it.

[Astro-Han]

Review hardening landed in ac8f060a (+284/−34, test-only), addressing an external review round against the matrix's own contract ("generated cell = dimension actually verified"):

• Generated discovery cells now enforce the full derived request contract before the scripted server responds: the declared pathname, exact query equality, and credential placement. auth: 'none' cells must carry no Authorization / x-api-key / x-goog-api-key header; openai-family cells assert the Bearer credential, anthropic-family x-api-key, google the key query with no credential header, and providers without API-key auth must send no credential at all. responseShape: 'array-or-data' cells run both the data-object and bare-array fixtures.
• The Anthropic, Google, and Cohere wire runners now parse the second-turn body and assert the tool result is actually replayed with the first-turn call id (tool_result / functionResponse / tool message respectively), matching what the OpenAI runner already asserted.
• Override registry entries now bind to the exact hand-written test titles in provider-conformance.test.ts, checked against the source text, so deleting or renaming an owned test fails the matrix instead of drifting silently. Full executable override binding stays with the Phase 8 consolidation per the Phase 7 additivity constraint in feat(providers): support common providers from models.dev #860.

Every new assertion was mutation-verified: temporarily breaking the fetcher's path/query/auth consumption, each SDK's tool-result serialization, and one registry title each turned exactly the corresponding cells red with provider- and dimension-named messages, then restored.

One seam surfaced by lm-studio: the plan derives discovery auth default for providers whose authKind is none, while the fetcher correctly sends no credential; the executor asserts per providerAuthSupportsApiKey for now. A third derived auth state in the core plan would be the root fix — deferred as a small follow-up since it changes plan semantics.

[Astro-Han]

Follow-up: the lm-studio seam noted above is closed in 2b2e1b80 — the plan now derives discovery auth none whenever the provider has no credential to send, and the executor's providerAuthSupportsApiKey workaround is removed. A generated none cell uniformly means the request must carry no credential.

[Astro-Han]

Second external review round handled in 08bcdac0 (+74/−21, test-only):

• Copilot second-turn contract (P1): the bound hand-written test now asserts turn two carries the exact model id, replays the assistant tool_calls id (call-copilot-echo) and reasoning_content verbatim, and links the tool result via tool_call_id — closing the same false-green class the generated Anthropic/Google/Cohere runners were hardened against in ac8f060a. Mutation-verified: tampering the SDK's second-turn tool_call_id, reasoning_content, and model serialization each turned the test red, then restored byte-identical.
• github-copilot:discovery rebinds to the dedicated picker/endpoint-gating test in model-fetcher.test.ts (policy-disabled / picker-false / no-tools / unsupported-wire decoys); override bindings now name their source file instead of assuming one.
• Binding check rejects disabled tests (P2): a bound title must appear as an enabled test( call — test.skip / test.todo no longer count as coverage, with a distinct failure message for each case.

Recorded deferral: optional_api_key (LocalAI) is folded into the with-key default case by the plan today; the keyless branch stays covered by model-fetcher.test.ts. Before Phase 8 deletes overlapping hand-written coverage, the plan must either grow per-cell credential cases or mark optional-auth discovery as an override.

E2E note: the earlier red gate was the pre-existing MiniMax catalog spec timing out in window teardown with all assertions passing — no overlap with this diff; a clean rerun is green, so it is classified as infrastructure per the #860 guardrail.

[Astro-Han]

Third external review round returned no blockers; two P2 follow-ups, one fixed in 976aa122 (+13/−8, test-only):

• Binding check now requires the complete quoted title: hasEnabledTestCall matched any test( call merely opening with the bound title, so renaming a hand-written test by appending a suffix still counted as coverage. It now matches test('<title>' through the closing quote (safe for the parametric Ollama binding, whose stored text is the full template literal). Mutation-verified: suffix-renaming the bound ZenMux title turned the binding test red naming zenmux:reasoning-replay, then restored. Known residual: a fully commented-out test(...) line still matches textually — documented in-source and owned by the Phase 8 executable override binding.
• optional_api_key per-cell credential cases: identical to the deferral already recorded above (Phase 8 precondition before deleting overlapping hand-written coverage); no new action.