feat(supervisor-middleware): add network egress middleware by pimlock · Pull Request #2027 · NVIDIA/OpenShell

pimlock · 2026-06-26T20:30:32Z

Summary

Implements the first usable RFC 0009 supervisor middleware slice: proto-backed, host-selected HTTP egress middleware for HttpRequest/pre_credentials, with both in-process built-ins and statically registered operator-run gRPC services.

The implementation covers RFC 0009 Phase 1 and adds basic external-service support from Phase 2. It establishes the contract, policy plumbing, ordered chain execution, built-in secret redaction, static gateway registration, relay integration, validation before policy persistence, body limits, audit events, and user-facing configuration and operations documentation.

< />!tip< />
See example middleware implementation: #2169
See high-level overview with interactive walk-through: https://gistpreview.github.io/?1f07f93cab1dab587d114fa6eec59c11/pr-2027-middleware-overview.html#request-lab

Related Issue

Closes #2010
Part of #1733
Design/RFC: #1738

Changes

Add the supervisor middleware contract and runtime, including the built-in openshell/secrets redactor and statically registered operator-run gRPC services.
Add ordered, host-selector-based network_middlewares policy configuration and validation, independent of the network policy rule that admits a request.
Enforce middleware on inspected HTTP requests before credential injection, with transformations, safe header additions, body limits, and explicit fail-open/fail-closed behavior.
Deliver effective service configuration to supervisors with resilient registry reload and last-known-good behavior.
Add OCSF observability plus gateway, policy, operations, architecture, and extensibility documentation.

Testing

mise run pre-commit passes
Unit and integration-style tests added or updated
E2E coverage considered
- No separate gateway E2E was added. The request relay, policy validation, registry, and remote gRPC paths are covered by automated unit and integration-style tests.

Checklist

Follows Conventional Commits
Commits are signed off (DCO)
Architecture docs updated
Published docs updated

copy-pr-bot · 2026-06-26T20:30:35Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

github-actions · 2026-06-30T19:48:46Z

🌿 Preview your docs: https://nvidia-preview-pr-2027.docs.buildwithfern.com/openshell

pimlock · 2026-07-02T21:20:43Z

/ok to test c4b0dcf

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

…are outages An unreachable operator-registered middleware service previously aborted sandbox startup via a hard error in load_policy, contradicting the per-request on_error contract and the resilient live-reload path. Retry the initial connect and, on failure, degrade to the built-in registry so matched requests are governed by each config's on_error (deny for fail_closed, allow for fail_open) instead of blocking the whole sandbox. The policy poll loop now reconciles the registry on every poll while an install is pending, so a recovered service is adopted without waiting for a config change; a failed reconcile also no longer blocks unrelated policy updates. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

…limit A chain entry whose binding did not resolve reported a zero body limit, which dragged the whole chain's buffer cap to zero and spuriously failed body-bearing requests over capacity even when a resolved middleware could have processed them. Exclude unresolved entries from the limit via a new DescribedChainEntry::is_resolved(); when no entry resolves, skip buffering and apply each entry's on_error directly. Also fix two parallel-test flakes found while validating the change: - Build middleware OCSF events into a Vec and assert on it directly instead of capturing through the global tracing pipeline, whose callsite-interest cache is process-global and raced under parallel runs. - Accumulate the websocket deny response until the reason marker arrives rather than assuming a single read returns the full body. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock · 2026-07-16T20:10:42Z

Addressed the latest findings in fdc5c34e.

Finding	Resolution
P1: fail-closed middleware failures returned policy-denial guidance	Added a dedicated platform-owned `middleware_failed` response for unstructured/fail-closed failures. It returns `Request could not be processed by configured middleware` and omits `rule`, `rule_missing`, `next_steps`, and `agent_guidance`. This is used by REST, JSON-RPC, MCP, GraphQL, passthrough relays, and the absolute-form forward proxy.
Runtime diagnostic text could reach response construction	Removed the diagnostic `reason` from `MiddlewareApplyResult::Denied`. Relay and forward-proxy response builders can now receive only an optional validated structured denial; otherwise they must use the platform failure response. Sanitized diagnostic codes remain available in operator telemetry.
Relay coverage	Added direct response coverage and strengthened fail-closed, body-capacity, transformed JSON-RPC, transformed GraphQL, audit-mode parse failure, and forward-proxy regressions to assert `middleware_failed` and the absence of policy-advisor fields.
Non-blocking deterministic hash note	Added a regression proving `deterministic_policy_hash` is stable for equivalent middleware configs constructed with different field insertion orders, including nested structs.
Documentation	Documented the `middleware_failed` response contract in the supervisor middleware and observability pages.

Validation:

mise exec -- cargo test -p openshell-supervisor-network --lib — 958 passed.
Focused deterministic policy-hash test passed.
mise run pre-commit passed, including workspace lint, checks, and tests.

pimlock · 2026-07-16T20:28:39Z

gator-agent

Blocked

Thanks @pimlock. I checked your July 16 update on head fdc5c34e765381c9d35eeb7597de49b15b203e03 addressing the fail-closed middleware failure response finding. The independent code-only reviewer found that the previous finding is resolved and did not report new blocking code findings for this head.

Gator is blocked by merge conflicts against main.

Head SHA: fdc5c34e765381c9d35eeb7597de49b15b203e03

GitHub currently reports mergeable: false and mergeable_state: dirty. Required checks are also still pending for this head, but the merge-conflict blocker takes precedence, so gator cannot advance to pipeline watch yet.

Next action: @pimlock needs to rebase or merge main into the PR branch and resolve the conflicts, then push the updated head for re-check.

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock · 2026-07-16T20:33:50Z

Merged current main (aa483ecb) in c9b4e43e.

The single conflict was in Cargo.lock. The resolution preserves both sides of the merged manifests: the middleware branch's prost-types dependency and main's explicitly versioned rustls 0.23.38 entry. cargo metadata --locked accepts the resulting lockfile.

Validation: mise run pre-commit passes after the merge, including workspace formatting, lint, checks, and tests.

pimlock · 2026-07-16T20:49:25Z

gator-agent

Re-check After Author Update

Thanks @pimlock. I re-evaluated latest head c9b4e43e4c1e1eb09f85a4fc6528328d5b755815 after your July 16 update merging current main (aa483ecb) and resolving the Cargo.lock conflict.

What I checked: the current merge commit, the combined Cargo.lock conflict resolution, the current PR diff around the middleware failure-response paths, the required CI state, and the independent principal-engineer reviewer result for this head.

Disposition: resolved. The prior merge-conflict blocker is resolved (mergeable: true), and the independent code-only review found no blocking findings for this head. The reviewer also found the Fern docs and docs/index.yml navigation sufficient for the user-facing middleware behavior.

Remaining items:

OpenShell / E2E is still pending for the required test:e2e gate.

Next state: gator:watch-pipeline

pimlock · 2026-07-17T00:58:37Z

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: the last active gator state was gator:merge-ready; maintainer approval was present and the required checks had completed before merge.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.

Resolve conflicts with the corporate HTTP proxy work (NVIDIA#2245), the CLI commands/common extraction (NVIDIA#2359), the supervisor middleware runtime (NVIDIA#2027), and the shared L7 endpoint validation refactor (NVIDIA#2389): - openshell-core/src/net.rs: keep both import sets — main's `ipnet` CIDR types alongside `SocketAddr`/`TcpStream` for the TCP_NODELAY helpers. - openshell-cli/src/run.rs: main moved the `progress` constants into the test module; keep only the `net` import at the top level. - openshell-server/src/lib.rs: union of both import lines. - supervisor-network/src/proxy.rs: main replaced the CONNECT tunnel's direct dial with proxy-aware `dial_upstream`. Take main's call and push TCP_NODELAY down into `dial_upstream`'s direct paths plus `upstream_proxy::connect_via_inner`, so tunnel hops keep it whether or not they chain through a corporate proxy. The plain-HTTP forward path keeps main's direct-dial comment with the NODELAY-setting connect. Signed-off-by: Jim Meyer <jim@meyer4hire.com>

pimlock force-pushed the 1733-supervisor-middleware/pmlocek branch 2 times, most recently from 595191e to 97b750f Compare June 26, 2026 23:58

pimlock self-assigned this Jun 30, 2026

pimlock force-pushed the 1733-supervisor-middleware/pmlocek branch from 358906a to 1fbcdbc Compare June 30, 2026 19:48

This comment was marked as outdated.

Sign in to view

pimlock added 18 commits July 2, 2026 15:17

feat(middleware): add in-process supervisor middleware

6ef9490

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(supervisor-middleware): harden middleware relay handling

e4e6f8f

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(supervisor-middleware): default stored policy rule fields

2765055

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(supervisor-middleware): resolve rebase policy conflicts

e487f2c

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

feat(supervisor-middleware): implement phase one runtime

9f892ef

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(supervisor-middleware): harden selection and buffering

da486b2

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

feat(supervisor-middleware): support external services

bb08a33

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

refactor(supervisor-middleware): clarify service contract

762887a

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

docs(supervisor-middleware): refine preview warning

31662d8

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

docs(extensibility): add supervisor middleware guide

2304e2f

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(server): remove stale middleware import

97bef95

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(network): remove needless test struct updates

ab2daba

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(middleware): avoid enabling core telemetry

a820dd2

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

refactor(supervisor-middleware): simplify service endpoints

49424a0

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

fix(supervisor-middleware): harden policy enforcement

9d4b631

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

feat(ocsf): enrich middleware shorthand logs

2b7cf4e

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock force-pushed the 1733-supervisor-middleware/pmlocek branch from c4b0dcf to 2b7cf4e Compare July 2, 2026 22:18

pimlock added gator:in-review Gator is reviewing or awaiting PR review feedback and removed gator:approval-needed Gator completed review; maintainer approval needed labels Jul 16, 2026

pimlock mentioned this pull request Jul 16, 2026

fix(server): prevent unrelated sandbox deletes from blocking deletion events #2326

Closed

fix(middleware): separate runtime failure responses

fdc5c34

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock added gator:blocked Gator is blocked by process or repository gates and removed gator:in-review Gator is reviewing or awaiting PR review feedback labels Jul 16, 2026

chore: merge main into middleware branch

c9b4e43

Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>

pimlock added gator:watch-pipeline Gator is monitoring PR CI/CD status gator:approval-needed Gator completed review; maintainer approval needed and removed gator:blocked Gator is blocked by process or repository gates gator:watch-pipeline Gator is monitoring PR CI/CD status labels Jul 16, 2026

drew approved these changes Jul 16, 2026

View reviewed changes

pimlock added gator:merge-ready and removed gator:approval-needed Gator completed review; maintainer approval needed labels Jul 17, 2026

pimlock merged commit d556748 into main Jul 17, 2026
110 of 117 checks passed

pimlock deleted the 1733-supervisor-middleware/pmlocek branch July 17, 2026 00:47

pimlock removed the gator:merge-ready label Jul 17, 2026

This was referenced Jul 17, 2026

feat: verify provider-authenticated call ingress before sandbox relay #2167

Open

feat: governed realtime media membrane #2166

Open

johntmyers mentioned this pull request Jul 20, 2026

refactor(network): consolidate proxy egress pipeline #2373

Open

9 tasks

prekshivyas mentioned this pull request Jul 20, 2026

feat(supervisor): isolate and declassify authenticated MCP discovery responses #2378

Open

18 tasks

This was referenced Jul 22, 2026

Supervisor middleware #1733

Open

feat: implement initial supervisor middleware #2010

Closed

feat: implement RFC 0009 phase 2 external middleware services #2011

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(supervisor-middleware): add network egress middleware - #2027

feat(supervisor-middleware): add network egress middleware#2027
pimlock merged 65 commits into
mainfrom
1733-supervisor-middleware/pmlocek

pimlock commented Jun 26, 2026 •

edited by johntmyers

Loading

Uh oh!

copy-pr-bot Bot commented Jun 26, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

pimlock commented Jul 2, 2026

Uh oh!

pimlock commented Jul 16, 2026

Uh oh!

pimlock commented Jul 16, 2026

Uh oh!

pimlock commented Jul 16, 2026

Uh oh!

pimlock commented Jul 16, 2026

Uh oh!

Uh oh!

pimlock commented Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

pimlock commented Jun 26, 2026 • edited by johntmyers Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Testing

Checklist

Uh oh!

copy-pr-bot Bot commented Jun 26, 2026

Uh oh!

github-actions Bot commented Jun 30, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

pimlock commented Jul 2, 2026

Uh oh!

pimlock commented Jul 16, 2026

Uh oh!

pimlock commented Jul 16, 2026

Blocked

Uh oh!

pimlock commented Jul 16, 2026

Uh oh!

pimlock commented Jul 16, 2026

Re-check After Author Update

Uh oh!

Uh oh!

pimlock commented Jul 17, 2026

Monitoring Complete

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pimlock commented Jun 26, 2026 •

edited by johntmyers

Loading