Skip to content

Retain external facts and harden protocol delivery recovery - #274

Open
patrickleet wants to merge 11 commits into
v5from
perf/cache-protocol-surface-manifests
Open

patrickleet wants to merge 11 commits into
v5from
perf/cache-protocol-surface-manifests

Conversation

@patrickleet

@patrickleet patrickleet commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This extends the existing protocol-manifest cache and live-stream recovery work with authenticated external facts. An external ledger or webhook can now retain its own source identity and position without pretending to be an event-sourced aggregate.

  • Cache immutable selected protocol manifests per export; authorization, actor scopes, issuance and visibility remain per request.
  • Recover a live receiver blocked by a retired page owner without accepting stale frames or weakening active-owner fences.
  • Classify application reload admission as retryable retain-and-stop, preserving the delivery for bounded host retry.
  • Add explicit external occurrence provenance and source-ordered snapshots, preserving existing aggregate APIs and wire behavior.
  • Preserve permanent content fingerprints while accepting identical logical messages at new broker positions. Advance the delivery checkpoint without repeating projection effects; conflicting content and cursor substitution still fail closed.
  • Share strict live/archive identity and kind validation. Malformed reserved headers remain visible permanent failures without automatic acknowledgement.
  • Rebuild source snapshots with origin-distinct identities: sparse external positions and multi-member facts no longer collide at empty aggregate fields.
  • Keep public-archive aggregate-prefix validation fail-closed. Explicit bounded original-log coverage can qualify authored private records without publishing them, with declared head, exact public fingerprints and single-publication contract checks.

Compatibility and migration

Migration 0009 adds delivery aliases while preserving the canonical unique message binding. A replay into a new projection generation still applies the event once. External facts participate in durable transport archival and replay; source position and broker delivery position remain distinct.

No global authority cache, new inbox subsystem, cursor reset, forced poison acknowledgement, or application polling workaround is introduced. See docs/external-facts.md, docs/protocol-manifest-reuse.md, and docs/live-retired-owner-handoff.md.

Original-log coverage is an offline operator/source-adapter trust boundary, not authentication of arbitrary private payloads. Retain and review original export provenance/digests; missing public occurrences, changed identity/content, unknown private contracts and truncated streams remain errors. No generic one-to-many publication completeness is inferred.

Validation

  • Framework library with GraphQL, SQLite, NATS and PostgreSQL: 1,061 passed, 0 failed, 3 explicitly ignored.
  • Final real NATS transport suite: 12 passed, including malformed live/archive identity, duplicate headers, kind validation and fail-closed settlement.
  • Real PostgreSQL protocol conformance and concurrent delivery-identity/cursor race tests passed.
  • Memory/SQLite new-generation and duplicate-delivery regressions passed.
  • JavaScript: 403 tests passed; TypeScript/generated-contract checks passed.
  • Existing HTTP GraphQL protocol, identity and causal transport coverage: 43 passed.
  • Focused offline rebuild regressions: 5 passed (memory/SQLite, external and original private-log coverage); separately configured real PostgreSQL rebuild passed. Existing CLI lifecycle suite passed 20 repeated runs (240 tests); added caller-location diagnostics for a non-reproduced CI fixture wait timeout without changing its semantics.
  • Changed Rust formatting and git diff --check passed.

Earlier application measurements demonstrated authenticated-query latency declining from roughly 1.1 seconds to 14–22 milliseconds after manifest reuse; these observations are not SLA assertions. Application-wide migration and end-to-end adoption are outside this framework PR's completion claim.

Retain engine-owned selected exports and share compiled metadata across request seeds. Principal, preset values, scope tokens and visibility authorization remain per request. Public manifests remain independent clones.

Validated 1052 library and43 protocol integration tests. In the retained Forge runtime, repeated authenticated GraphQL fell from about1.1s to14–22ms; warm page DOM times fell from3–7s to97–291ms. Cold dev compilation remains separately documented.
@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 598748a8-9f75-47d7-83d0-e71ce23c03fd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Preserve pre-disposal frame fencing, then reopen only the blocked receiver beyond the retirement boundary. Keep unrelated layout subscriptions active. Refs Forge repository preparing incident; all403 JS tests and actual fresh repository live transition pass.
@patrickleet patrickleet changed the title Reuse immutable protocol manifests for authenticated requests Cache protocol manifests and recover retired-page live streams Sep 21, 2026
@patrickleet patrickleet changed the title Cache protocol manifests and recover retired-page live streams Retain external facts and harden protocol delivery recovery Sep 28, 2026
patrickleet and others added 7 commits September 28, 2026 14:13
Keep public archive prefix checks fail-closed. Qualify reviewed original single-publication aggregate logs without synthesizing private publications; retain external identity and source snapshot fences.
A failed @LiVe execution (e.g. a statement timeout while storage is
unreachable) reached clients as an error-only frame with only the base
envelope. The replica required snapshot/live metadata on every live frame
with errors, so it replaced the real GraphQL error with "Invalid
Distributed GraphQL protocol envelope at extensions.distributed.live".

Spec (docs/live-query-delivery.md) now defines that frame as a terminal,
receipt-only failure frame. The client surfaces its errors, admits no
data, cursors, ownership or command receipts, and reopens the
subscription with 1s-30s backoff so live queries recover without a
reload. The server producer stops after its first error so a failure can
never take a later frame's snapshot/live metadata. Data-bearing frames
without live metadata remain invalid.

Resolves [[incidents/forge-dashboard-protocol-20260929]]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A single sequential event consumer ran every route for a delivery before
receiving the next, so latency-sensitive process policies waited 1-4s
behind unrelated projections and kernel effects. A permanently failing
message NAK'd without delay was redelivered ~6/s between every new
message.

- Opt-in delivery lanes (`Service::lane`) within one durable consumer:
  each lane runs deliveries in broker order; a delivery is acknowledged
  only after every lane settles it; bounded in-flight window. Sources
  that settle only in order (Kafka, SQL) keep the sequential loop.
- Retryable NATS NAKs back off by delivery count (50ms doubling to 5s,
  configurable), so a failing message cannot monopolize a consumer.

Spec: docs/consumer-delivery-lanes.md (incl. cross-delivery reordering
constraints). Tests: lane unit tests, service lane tests, NATS
integration tests (publish-after-idle, backoff, lanes over JetStream).

Implements [[tasks/forge-provisioning-latency-20260930]]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pages that open several live subscriptions over the same root lookup
(e.g. git_repositories by id) let the first subscriber own the shared
root list and rejected every later whole frame, including frames whose
shared lists were identical. Fields only the rejected subscription
selects (a repository's branch refs) never updated after a push, and
stored rows could drift from rejected lists ("Loading branches…").

A live frame is now admitted when it provably agrees with the owner on
every shared list (same rows, order and nulls; owner complete and
current); it writes only its own lists. Disagreement or unprovable
agreement is still rejected, and the rejected subscription reopens when
another query rewrites a shared list.

Spec: docs/live-query-delivery.md. Tests: replica-protocol (new
agreement/rejection cases; 408/408).

Resolves [[incidents/forge-branches-page-writes-disabled-20260930]]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant