Skip to content

test(verify): add the live MCP verification procedure - #1214

Merged
mforce merged 5 commits into
mainfrom
test/verify-mcp-procedure
Oct 11, 2026
Merged

mforce merged 5 commits into
mainfrom
test/verify-mcp-procedure

Conversation

@mforce

@mforce mforce commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Refs #789

Adds a repeatable procedure for checking /mcp with real MCP clients against a running sim stack. In-process tests cannot cover discovery behind a real host, dynamic registration, the consent screen in a browser, the token exchange, or real tool calls. This procedure covers them. I ran it against #1209's head before that PR merged, and every step passed.

What is in it

  • tools/verify/features/mcp.md is the procedure. It lists the prerequisites and the exact commands, has a table of what each check proves, and records the gotchas found on the first run.
  • tools/verify/mcp/ holds the clients and the TLS proxy, with a short README.
  • tools/verify/features/README.md gains one row in the feature map.

The three clients

  • client.ts uses the official TypeScript MCP SDK. Its connect command runs discovery, dynamic registration, PKCE and the RFC 8707 resource parameter, and Playwright signs in and approves the consent screen with the password. The tools, call and disconnect commands reuse a saved token, and disconnect drives Account › Connected apps. This SDK speaks MCP 2025-11-25.
  • probe.cs is a file-based app on the C# ModelContextProtocol 2.2.0 client, the server's own SDK. It covers MCP 2026-07-28, which the TypeScript SDK does not support yet.
  • MCP Inspector CLI runs as an optional third client. With --strict it also lints the tool schemas.

Gotchas the procedure records

  • The sim stack serves plain http, and OpenIddict refuses OAuth requests over http outside Development. tls-proxy.mjs terminates TLS at the issuer's port, keeps the Host header and sets X-Forwarded-Proto.
  • .env.sim needs three changes: OAuth__Issuer=https://localhost:8495/, AllowedHosts=localhost and TRUSTED_PROXY_CIDR=172.16.0.0/12. Docker's port forward makes the app see the bridge gateway rather than loopback. If the trusted range does not cover it, the app ignores X-Forwarded-Proto and every OAuth request answers This server only accepts HTTPS requests.
  • The TypeScript SDK requests every scope the resource metadata lists and ignores clientMetadata.scope. client.ts narrows the scope on the authorize URL. Otherwise the write-only check proves nothing.
  • The fixture has no reason text, so the redaction check seeds a write-off and a void with a marker reason through REST first.
  • probe.cs opts out of the repo's central package versions and lock files in its header.

No product code changes. The tracked-file guards (SchemaDocsTests, including both image-pin tests) pass locally.

A real MCP client (TypeScript SDK + Playwright consent) and the C# SDK
client drive /mcp through the real OAuth flow on a sim stack behind a
local TLS proxy. Procedure and checks in tools/verify/features/mcp.md.
…harness browser

client.ts ignores certificate errors only for a localhost MCP_BASE and then
aborts browser requests to any other origin; probe.cs trusts exactly the
certificate in MCP_TLS_CERT. The browser comes from resolveBrowser. The
empty-password click is labelled as form validation, and call prints the
tool's text so replies pipe into jq.
Requires exclusive use of cluckwork-sim and forbids reset.sh on another
project. Adds the refusal, step-up, Worker foreign-id, write-only call and
protocol commands, a real lot id, and a redaction positive control that
finds the marked rows before asserting the marker is absent.
@mforce

mforce commented Oct 11, 2026

Copy link
Copy Markdown
Owner Author

All six points from the Codex review are fixed in 019cdb2 and 00a94da. Both are new commits; nothing was force-pushed.

  1. Isolation. The procedure now requires exclusive, agreed use of cluckwork-sim. It forbids running reset.sh for another project or with a renamed compose file, and states that the repo has no isolated recipe. A closing step restores the three .env.sim values and resets the stack.

  2. TLS. client.ts ignores certificate errors only when MCP_BASE is localhost or 127.0.0.1. With that exception on, it aborts any browser request to another origin. probe.cs trusts exactly the certificate named by MCP_TLS_CERT and verifies every other certificate normally. Without the variable, it rejects the proxy's certificate.

  3. Browser. client.ts launches whatever resolveBrowser() from tools/simulation/ui/src/browser.ts picks. The doc adds npx playwright install chromium for a fresh host.

  4. Negative calls. Each table row now has a numbered step with its command and the expected output:

    • step 3: the session JWT and both invalid_target refusals;
    • step 5: a real lot id taken from cluckwork_list_egg_lots;
    • step 7: Worker reads, stock parity with REST, and a foreign lot and a missing lot giving the same refusal;
    • step 8: a forbidden tools/call with the write-only token;
    • step 9: both protocol versions.

    Client ID metadata documents are marked as not covered.

  5. Redaction. Step 6 seeds a marked discard and a marked void through REST and confirms REST keeps the marker. It then fetches exactly those rows through MCP, and two jq -e checks confirm they are present before the marker count is checked.

  6. Step-up. The empty-password click is now labelled as form validation. Step 4 sends the consent request directly with the session JWT. With no grant, the reply is the consent payload with redirectUri: null. With a forged grant, the reply is 403 Identity.StepUpRequired.

I ran every command in the doc against a live stack built from #1210's head 681e8f6a, with the ports changed. Each output matched its expectation. The tracked-file guards (SchemaDocsTests, both image-pin tests included) passed 4 of 4 locally.

@mforce
mforce merged commit 1d2063d into main Oct 11, 2026
18 checks passed
@mforce
mforce deleted the test/verify-mcp-procedure branch October 11, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant