fix(sts): key the ADK token cache by acting subject, not session alone (Python) - #2460
fix(sts): key the ADK token cache by acting subject, not session alone (Python)#2460QuentinBisson wants to merge 11 commits into
Conversation
…e (Python) Rebased onto current main; adapts to the RFC 8707 resource/audience constructor params. Signed-off-by: QuentinBisson <quentin@giantswarm.io>
6c8b048 to
e4fb32e
Compare
There was a problem hiding this comment.
Pull request overview
Fixes a correctness/security bug in the Python ADK STS token-propagation plugin where delegated (exchanged) tokens were cached only by session ID, causing shared sessions with multiple acting subjects to reuse the first subject’s exchanged token. This aligns the Python runtime behavior with the Go-side fix and prevents identity collapse across callers within the same session.
Changes:
- Key the token cache by
(session_id, subject_key(subject_token)), wheresubject_keyprefersiss+suband falls back to a hash for opaque/sub-less tokens. - Resolve the subject token before cache lookup in
before_run_callback, and sweep expired entries across the cache inafter_run_callbackto handle per-subject entries. - Update/extend integration tests to validate multi-subject session behavior, issuer scoping, opaque token handling, and fail-closed behavior when no subject token is present.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| python/packages/agentsts-adk/src/agentsts/adk/_base.py | Implements per-subject cache keying, adds subject discriminator logic, adjusts cache lookup + eviction behavior. |
| python/packages/agentsts-adk/tests/test_adk_integration.py | Updates existing assertions to use cache_key(...) and adds new cases covering multi-subject behavior and cache eviction semantics. |
Suppressed comments (1)
python/packages/agentsts-adk/src/agentsts/adk/_base.py:263
- cache_key() now calls the configured get_subject_token callback, which can raise. Since header_provider uses cache_key during tool invocation, an exception here would break tool calls; it’s safer to fail closed (return a key with an empty subject) when subject resolution fails.
session = invocation_context.session
return self._cache_key_for(session.id, self._read_subject_token(session.state))
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # subject's key is derivable here, so sweep every expired entry rather | ||
| # than leaving the other subjects' behind. | ||
| for key in [key for key, entry in self.token_cache.items() if _has_token_expired(entry.expiry)]: | ||
| logger.debug("Removing expired subject token from cache") | ||
| self.token_cache.pop(cache_key, None) | ||
| self.token_cache.pop(key, None) |
There was a problem hiding this comment.
Partly fixed in bc252de. The sweep is now gated on the earliest expiry across the cache, so after_run_callback returns without walking anything until a cached token can actually be evicted; the scan recomputes the earliest expiry as it goes, and inserts keep it up to date.
I kept the sweep global rather than scoping it to the running session: a session-scoped sweep never reaches the entries of sessions that stop running, so those would be retained for the lifetime of the process. Trading unbounded growth for an occasional walk seemed like the wrong side of that.
There was a problem hiding this comment.
Closed out in 6471f57. The gate had a hole: a token with no exp cached a None expiry, _earlier_expiry skipped it, and a cache holding only such entries left _earliest_expiry unset so the sweep never ran. Those entries now get a bounded lifetime, so the gate is always armed and every entry stays evictable.
| subject_token = self._read_subject_token(invocation_context.session.state) | ||
| if not subject_token: | ||
| logger.debug("subject token not found in session state for token propagation") | ||
| return None |
There was a problem hiding this comment.
Fixed in bc252de. _read_subject_token now catches and logs, returning None, so a raising caller-supplied get_subject_token skips propagation instead of aborting the run. One change covers both call sites (before_run_callback and cache_key), and it matches how exchange_token and fetch_actor_token failures are already handled here.
There was a problem hiding this comment.
Also handled for the other caller of the same code path in 6471f57: header_provider reaches _read_subject_token via cache_key on every tool call, and it was dereferencing readonly_context._invocation_context without a guard. It now returns no headers instead of raising into the invocation.
…he sweep A caller-supplied get_subject_token that raises no longer aborts the agent run; token propagation is skipped instead. The expired-token sweep stays global, since scoping it to the running session would keep the entries of sessions that never run again forever, but it is now gated on the earliest expiry in the cache so a growing cache is only walked when there is something to evict. Signed-off-by: QuentinBisson <quentin@giantswarm.io>
…ut a context Keying the cache by (session, subject) multiplies entries per session, so an entry whose token carries no exp claim now pins one slot per caller instead of one per session. Give those a bounded lifetime so every entry stays evictable, which also arms the sweep gate that a None expiry left unset. header_provider runs on every tool call and dereferenced a private attribute of an optional argument; it now returns no headers instead of raising into the invocation. An issuer-less token leaves sub unqualified; those partition by token hash rather than by a key two issuers could both produce. Signed-off-by: Quentin Bisson <quentin@giantswarm.io>
3b8755d to
4e72a3c
Compare
An empty subject identifies no principal, so an entry stored under it would be shared by every credential-less caller in a session. The key helper now yields None for it and the cache paths skip. Signed-off-by: Quentin Bisson <quentin@giantswarm.io>
4e72a3c to
e348494
Compare
get_subject_token receives the whole session state, so an implementation reading a session-scoped field returns one value for every caller and collapses the key back to a single entry per session, which is the bug this branch fixes. Derive the key from the inbound Authorization header instead, and keep the hook for the exchange payload. Without an inbound credential there is nothing caller-scoped to key on, so the hook's output is still used: that mode carries no per-caller identity, and one entry per session is the correct partitioning for it. Signed-off-by: Quentin Bisson <quentin@giantswarm.io>
a1794bb to
5b8643b
Compare
A cache hit returns a delegated token without performing an STS exchange, so the cache key decides who receives someone else's authority. Deriving it from the unverified iss/sub claims let a forged, unsigned token select a victim's entry and never reach the STS that would have rejected it. _subject_key now hashes the raw token, so a forged token is a cache miss and goes to the STS. _acting_credential also guards a non-dict headers value, which reached _extract_jwt_from_headers from header_provider on every tool call, and cache_key tolerates a context without a session. Signed-off-by: QuentinBisson <quentin@giantswarm.io>
…edential's expiry The key now names both the caller's credential and the token the entry was exchanged from, so a get_subject_token returning something other than the caller's own credential cannot produce an entry that another caller matches. The entry also stops expiring later than the credential that keyed it: a caller replaying an expired bearer kept hitting the cached delegated token instead of reaching the STS. header_provider runs on every tool call and resolved the key itself, which called a caller-supplied get_subject_token per tool call. The key is resolved once per run and read from there. Signed-off-by: QuentinBisson <quentin@giantswarm.io>
A run that dies before after_run_callback never releases its entry. Signed-off-by: QuentinBisson <quentin@giantswarm.io>
The key named the token the entry was exchanged from as well as the caller's credential. Only the credential decides who may receive an entry; the token hash bought freshness when a get_subject_token hook rotates its value mid-session, which the entry's own expiry already bounds. Naming it cost more than it bought: the hook part is not derivable without calling the hook, so header_provider had to read a key resolved once per run, and that memo needed a size cap and an eviction rule. Two callers whose runs carried no invocation id shared one memo entry, which is the leak this fix exists to close. The caller's credential comes from the run's own session state, which the executor writes once per run as a state delta and the session service hands out per run. header_provider rebuilds the key from it on every tool call without consulting the hook, so no state is kept between callbacks. Also refuse to cache a session with no id, matching the Go plugin. Signed-off-by: QuentinBisson <quentin@giantswarm.io>
7f35760 to
a82abb9
Compare
Supersedes #2333 (auto-closed by stale-bot; a rebase onto main conflicted with the resource/audience support, so GitHub would not let it be reopened).
Ref #2181. The Go counterpart is #2459 and carries the closing reference.
Problem
Same bug as the Go plugin: the cache was keyed by session ID alone, so a session shared by several subjects reused whichever caller's exchanged token was cached first.
Change
Key the cache by
(session_id, caller credential), the credential hashed.before_run_callbackandheader_providerboth resolve the caller before the cache lookup, so each caller gets its own exchanged token.What the key is derived from
The caller part of the key comes from the inbound
Authorizationheader, not fromget_subject_token. That hook receives the whole session state, so an implementation reading a session-scoped field returns the same value for every caller and collapses the key back to one entry per session, which is the bug being fixed here.The header is read from the run's own session state, which
_agent_executor.pywrites once per run as astate_deltaand the session service hands out per run. Soheader_providerrebuilds the key on every tool call without consulting the hook and without state held between callbacks.With no inbound credential, the hook's output stands in for the caller part. That mode has no per-caller identity to preserve, so one entry per session is correct for it, and keying on the header alone would drop propagation entirely.
An empty subject identifies no principal, so it yields no key at all, and neither does a session with no id.
Why the key is a hash and not iss+sub
A cache hit returns a delegated token without performing an exchange, so the key decides who receives whose authority.
issandsubare not verified anywhere on this path: a forged, unsigned token carrying a victim's claims would select the victim's entry and never reach the STS that would have rejected it. Hashing the raw token makes a forged token a cache miss, so it goes to the STS and fails there.The cost is one extra exchange when a caller's bearer rotates mid-session.
Cache lifetime
An entry never outlives the credential that keyed it. The expiry is the earlier of the exchanged token's
expand the caller bearer'sexp. Without that cap, a caller replaying an expired bearer keeps hitting the cached delegated token instead of reaching the STS that would reject it.One entry per caller instead of one per session also means a token with no
expat all would pin an entry per caller for the lifetime of the process. Those entries fall back to a 5 minute lifetime, which also arms the sweep gate that aNoneexpiry left unset.An entry outliving the value a
get_subject_tokenhook has since rotated is bounded by that same expiry.The sweep removes expired entries only, so the cache is sized by the number of distinct
(session, subject)pairs seen within a token lifetime. No size cap or LRU is added; the bounded TTL is what bounds it.Failing closed in header_provider
header_providerreturns no headers instead of raising into the invocation when there is no invocation context, no key for the caller, or a non-dictheadersvalue in session state. This matches how_read_subject_tokenhandles a raisingget_subject_token.Scope
Propagate-only mode (no STS configured) forwards the caller's own token. This fix stops one caller's token reaching another caller, but does not add exchange, audience narrowing or an
actclaim to that mode: there is no STS to exchange with when none is configured.Note
go-unit-testsfails for a reason unrelated to this diff, which touches no Go.go test ./...runsapi/v1alpha2andapi/v1alpha3in parallel and both download the envtest binaries into the same bin dir on demand, so on a cold runner cache one package execs a half-written etcd ("text file busy") while the other fails to unpack kubectl. It reproduces onmainwith an emptygo/bin/k8s, and passes once the binaries are provisioned first.The two
prev-stableupgrade jobs fail inhelm-install-providerwithUPGRADE FAILED: context deadline exceeded, which is also unrelated to a Python-only diff.