Repository navigation
JWT signing-key rotation needs a multi-key validation ring + kid #268
Description
Deployment-readiness gap (#244), RECOMMENDED — flagged by 2 of 3 reviewers.
Problem
The runbook lists "JWT key rotation," but the code cannot rotate keys without downtime. Validation accepts exactly one public key via IssuerSigningKey (singular) (Program.cs:322-333), and issued tokens carry no kid header (JwtTokenService.cs). There is no IssuerSigningKeys collection and no key resolver, so the validator cannot accept old + new tokens during an overlap window — swapping the key instantly 401s every already-issued access token, and you cannot stage the next key ahead of cutover.
Blast radius is bounded by the 15-min access-token lifetime (refresh tokens are DB-hash-based and survive rotation), but a zero-downtime or emergency rotation is impossible as written.
Fix
Bind a plural Jwt:PublicKeyPems into IssuerSigningKeys, stamp a kid on issued tokens, designate one active signing key, and retain the previous public key for at least one access-token lifetime + rollout margin.
Verify
Integration-test old + new tokens validating across overlapping instances, then final retirement of the old key 401ing correctly.
Part of the #244 deployment-readiness audit; tracked in epic #15.
Activity
[-]Deploy: JWT signing-key rotation needs a multi-key validation ring + kid[/-][+]JWT signing-key rotation needs a multi-key validation ring + kid[/+]mforce commentedon Sep 12, 2026
Note from #788: a second key-management story is arriving, and it does not fix this one.
#788 adds OpenIddict as an OAuth 2.1 authorization server for MCP clients. OpenIddict manages its own signing and encryption credentials, publishes a JWKS, stamps
kidon what it signs, and supports key rollover natively — i.e. it has, built in, roughly the capability this issue says the hand-rolled path lacks.That does not close this issue, for two reasons worth stating so nobody reads the overlap as a fix:
IssuerSigningKey, and itskid-less tokens. Everything this issue describes remains exactly as described.What does change is that the repo will have two key-management stories side by side: the hand-rolled RSA PEM pair from config (with #510's boot check requiring both keys present and importable) and OpenIddict's own credential management. Neither knows about the other. That is tolerable but worth recording, because a future reader will reasonably assume one system governs signing keys and it will not.
The connection worth tracking: #793 considers migrating the SPA's login to OpenIddict eventually. If that ever happens, it very likely closes this issue as a side effect, since key rollover with
kidand a validation ring is exactly what OpenIddict does natively. That makes this issue's cost a factor in #793's argument — fixing key rotation by hand is work that a migration would subsume.No action needed here now. Flagged so the overlap is deliberate rather than discovered.
mforce commentedon Sep 13, 2026
Correction to my earlier note: this is more relevant to #788 than I said, though still not closed by it.
That note claimed OpenIddict's key management would "mostly matter for ID tokens and discovery metadata, not for the access tokens assistants actually present", because #788 chose reference tokens. Verified against OpenIddict's documentation and that reasoning was wrong.
Reference tokens do not avoid key management
A reference token only changes what is returned to the caller — an opaque 256-bit identifier. The payload is still generated through the normal crypto pipeline and stored in the database. Quoting the docs: the server "enforces encryption for all the token types it supports", and that enforcement "cannot be disabled for authorization codes, refresh tokens and device codes for security reasons". Only access-token encryption specifically can be relaxed — and signing still applies.
The server also "requires registering at least one asymmetric key to sign identity tokens".
Production requirement, quoted: "it is recommended to use two RSA certificates, distinct from the certificate(s) used for HTTPS: one for encryption, one for signing" — and they must be identical across replicas, since a token protected by one instance must validate on another.
What that means for this issue
Three things, none of which close it:
OAuth slice 1: stand up OpenIddict — tables, migration, token issuance #795 introduces a SECOND key set. The repo goes from one key-management story — hand-rolled RSA PEM pair from config, with fix(api): the Jwt PEM 'not configured' guard is dead for the shipped config, and the key is only parsed per-request #510's boot check — to two, neither aware of the other. That was flagged before; what is new is that it is unavoidable rather than incidental.
The new set will have the rotation this issue asks for, while the old one still will not. OpenIddict supports additive rollover: "When multiple keys/certificates are registered (which can be useful to implement keys rotation), OpenIddict chooses the most appropriate key", preferring certificates with the furthest expiration and ignoring those not yet valid. That is functionally what this issue proposes building by hand — a validation ring with graceful cutover. An odd asymmetry to ship knowingly: the newer, less-central credential path gets zero-downtime rotation, the one guarding every SPA session does not.
It likely forces new required config keys (certificate thumbprints or PFX paths). Under Sim harness (#243) rotted silently: 4 breakages, no CI ever ran it #370 and Developer experience: add an Aspire AppHost for local orchestration and observability #565 those must be taught to
tools/simulation/bootstrap.sh,docker-compose.sim.yml,verify-harness.shandsrc/Cluckwork.AppHost/Program.csin the same PR — both deliberately outside CI, so nothing reports the breakage. Recorded on OAuth slice 1: stand up OpenIddict — tables, migration, token issuance #795.Still not closed by #788
#788 is scoped to third-party clients; the SPA keeps its hand-rolled login, its single
IssuerSigningKey, and itskid-less tokens. Everything this issue describes remains true of that path.#793 remains the thing that would close this as a side effect — and this finding strengthens that argument, because it means the rotation capability will already be present in the codebase, just wired to the wrong credential set.
One alternative worth knowing
OpenIddict can use ASP.NET Core Data Protection format instead of JWT, which "relies on their own key ring, distinct from the credentials discussed in this documentation". That trades certificate management for Data Protection key-ring management — which is #794, currently unconfigured. Not a free escape; a different key story.
mforce commentedon Sep 13, 2026
Closing — deferred to #793, as an accepted risk
Closed by the owner on 2026-09-13 during the issue cleanup, on the grounds that #793 will take
care of it: migrating the SPA's hand-rolled login to OpenIddict brings JWKS,
kidstamping andnative key rollover with it, rather than building a second rotation mechanism by hand first.
That reasoning is sound — the comments above already established that OpenIddict has, built in,
roughly the capability this issue says the hand-rolled path lacks. Building the ring by hand now
and then adopting OpenIddict would mean building it twice.
What is being accepted, stated plainly
Until #793 lands, the production app cannot rotate its JWT signing key without 401ing every live
access token. Specifically:
IssuerSigningKey(Program.cs), notIssuerSigningKeys.kidheader (JwtTokenService.cs), so no resolver can pick among keys.Blast radius is bounded — access tokens live 15 minutes and refresh tokens are DB-hash-based, so
they survive a rotation and clients recover on their next refresh. A planned rotation is therefore
a ~15-minute degradation, not a lockout. An emergency rotation on a compromised key is the case
that hurts, and it is the case this issue existed for.
Why this cannot be silently lost
#793 is titled "Consider migrating…" — it is
priority:tier3, unscheduled, and conditionalon MCP proving out first. If it resolves as "no, keep the hand-rolled login", the gap described
above comes straight back with nothing tracking it.
So the requirement has been written into #793 as an explicit condition: a "no" answer there
reopens this issue rather than closing the subject. See the note on #793.
If a key is ever compromised before #793 lands
Rotate anyway and accept the ~15-minute window: set the new key, restart, and let clients refresh.
recover-admin(#265) is unaffected — it does not depend on the JWT key. That is the operationalanswer in the meantime, and it is worth having in the runbook rather than discovered under pressure.