Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ Reviewers: treat a hardcoded provider name in code, config, or a committed doc l

**Run one serving instance.** Every instance runs `DurableJobWorker`, but its poll and the three recurring sweeps now run **only under a single-leader gate** — a session-scoped Postgres advisory lock (`pg_try_advisory_lock`) on a dedicated, non-pooled connection (**#271**, closed): at most one instance leads, crash recovery is automatic (a dead leader's session releases the lock), and the contract is at-most-one-leader with at-least-once, idempotent handlers — never "exactly once". That guarantee holds on a **session-pinned** Postgres endpoint (a direct connection or a session-pooled proxy); under a **transaction-pooling** proxy (e.g. PgBouncer in transaction mode) the lock can migrate across backends and single-leader is *not* guaranteed — a backend-PID affinity check narrows but does not close the window, so that topology relies on the at-least-once + idempotent contract until a dedicated session-pinned lease endpoint (**#556**) lands. The per-account report concurrency cap (#311) was the last independent in-process blocker, and **#545 closed it**: it now runs on the shared renewable lease (#543), keyed per `AccountId`, each permit pinned to the backend that granted it — so N replicas enforce one combined per-account permit count, degrading to a bounded per-instance ceiling (never fail-open) under a store outage. The IP-keyed auth limiters (#143) were another — and **#544 closed it**: they now run on the shared `IFixedWindowCounter` (#543, Redis-backed with an in-process fallback + alarm), so N replicas enforce one combined per-IP budget instead of N. The step-up grant registry was another — the blocker with teeth — and **#338 closed it**: replay now lives in the shared `IClaimOnceStore` (#543) and logout revocation in the durable per-user `ApplicationUser.StepUpLogoutEpoch`, an integer compared for equality (never a timestamp), so both survive across replicas without a shared clock. **#307 is closed and does not license scaling.**

**Do not extend that list from memory** — it was twice derived wrongly, both times a process-local limiter. Re-derive it by walking every `AddSingleton`/`AddHostedService` under `src/` plus every in-memory state primitive, then excluding deliberately. As of 2026-08-25 that walk finds **19 `AddSingleton` call sites and 1 `AddHostedService`** — but count call SITES, not live registrations: `SharedStateRegistration.cs` registers `IClaimOnceStore` and `IFixedWindowCounter` twice each on mutually exclusive branches (Redis vs the in-process fallback), so at most **17** are live in one process, and **16** when Redis is unconfigured. A bare count here has already gone stale twice; re-run the walk rather than trusting this sentence. The run-then-exit verbs are unaffected: they never start hosted services. The #543 shared-state registrations (`IConnectionMultiplexer` + the `IClaimOnceStore`/`IFixedWindowCounter` ports) are **not** blockers — they are the shared store: #338 wired the `IClaimOnceStore` grant-replay caller and #544 wired the `IFixedWindowCounter` auth limiters (both closed above). #545 wired the report cap onto the lease backends directly: `ReportConcurrencyCapRegistration` constructs `RedisLease`/`InProcessLease` itself and pins each permit to its granting backend, rather than resolving a shared `ILease` port from DI. Redis-backed they are multi-replica-safe, their in-process fallbacks a deliberate alarmed degradation. → [`271-single-serving-instance.md`](docs/decisions/271-single-serving-instance.md)
**Do not extend that list from memory** — it was twice derived wrongly, both times a process-local limiter. Re-derive it by walking every `AddSingleton`/`AddHostedService` under `src/` plus every in-memory state primitive, **plus what each package `Add*`/`With*` extension `Program.cs` calls registers internally** — that last part is not optional and `src/` cannot show it: one innocuous `builder.Services.AddThing()` line can register several services, hosted services included, and the walk will not see one of them (#786). Graph the package at the version `packages.lock.json` resolves and ask it directly — `graphify clone <repo> --branch <tag>`, `graphify update . --no-cluster`, then `graphify explain "<ExtensionName>"` lists what it registers with line numbers. Then exclude deliberately. As of 2026-08-25 that walk finds **19 `AddSingleton` call sites and 1 `AddHostedService`** — but count call SITES, not live registrations: `SharedStateRegistration.cs` registers `IClaimOnceStore` and `IFixedWindowCounter` twice each on mutually exclusive branches (Redis vs the in-process fallback), so at most **17** are live in one process, and **16** when Redis is unconfigured. A bare count here has already gone stale twice; re-run the walk rather than trusting this sentence. The run-then-exit verbs are unaffected: they never start hosted services. The #543 shared-state registrations (`IConnectionMultiplexer` + the `IClaimOnceStore`/`IFixedWindowCounter` ports) are **not** blockers — they are the shared store: #338 wired the `IClaimOnceStore` grant-replay caller and #544 wired the `IFixedWindowCounter` auth limiters (both closed above). #545 wired the report cap onto the lease backends directly: `ReportConcurrencyCapRegistration` constructs `RedisLease`/`InProcessLease` itself and pins each permit to its granting backend, rather than resolving a shared `ILease` port from DI. Redis-backed they are multi-replica-safe, their in-process fallbacks a deliberate alarmed degradation. → [`271-single-serving-instance.md`](docs/decisions/271-single-serving-instance.md)

## Writing a guard (a test that asserts an invariant)

Expand Down
34 changes: 34 additions & 0 deletions docs/decisions/271-single-serving-instance.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,40 @@ rediscovered.
Both misses were the same shape — a process-local limiter — and the second one
lived in a file the first sweep had already opened.

**A third shape appeared in 2026-09 (#786), and it is not a miss of the list —
it is a miss the METHOD cannot catch.** The walk below is scoped to `src/`. A
NuGet package's own `IServiceCollection` extension can register services that
never appear there: only the one-line call site does, and it looks like any other
registration. Designing MCP support (#770) surfaced a concrete case —
`ModelContextProtocol.AspNetCore`'s `WithHttpTransport` registers three
singletons **and an `AddHostedService`**, unconditionally, from one call. Two of
three independent reviewers followed the walk correctly and still concluded "this
adds no hosted service"; only the one who read the package source got it right.

The outcome there was benign — the service is inert under
`SessionMode.Stateless` — so the blocker list did not change. That is luck, not
method. Per this repo's own rule that two misses of the same shape mean the
method is wrong, the defect here is the walk's **scope**, and the fix is to walk
into the package:

```bash
graphify clone <package-repo-url> --branch <tag-matching-packages.lock.json>
cd ~/.graphify/repos/<owner>/<repo> && graphify update . --no-cluster
graphify explain "<ExtensionMethodName>" # lists its registrations, with line numbers
```

Two caveats that must not be dropped when this is repeated. It graphs the
package's **published source at a tag**, not the restored assembly, so read the
version from `packages.lock.json` rather than typing a branch — otherwise the
graph faithfully describes code that is not what restores. And it needs the
package to have public source at all.

The stronger fix, still open: a test that builds the real service collection and
asserts the `IHostedService` descriptors and singleton lifetimes against a
reviewed inventory. That is the only form that catches a package **upgrade**
changing registrations under an unchanged call site, which neither prose nor an
on-demand graph query will.

**So do not extend this list from memory.** Re-derive it: enumerate **every**
`AddSingleton`/`AddHostedService` under `src/`, plus every in-memory state
primitive (`ConcurrentDictionary`, `IMemoryCache`, `PartitionedRateLimiter`,
Expand Down
Loading