Skip to content

id: design: region-seam handoff format — journaled export/import for hot-world region shards (from #2059) #2818

Open

Description

@dezren39

From the #2059 investigation (hot-world sharding)

The landed investigation doc for #2059 (doc/2026-10-30T19-35-00Z_investigation_hot_world_sharding/, §3 rung 2, §5) designs region partitioning as sibling shard-worlds under a thin coordinator, with entity handoffs at region seams as a journaled export from region A + import to region B — a two-step handoff, not a distributed transaction. The doc states the semantics honestly (cross-region atomicity does not exist; an entity mid-handoff is visibly in transit) but deliberately does not design the handoff record format. This issue designs it, so the #2046 synthesis adopts a designed seam instead of an aspirational one.

Design to produce

  • Export/import record pair: the journal entries each side writes (export from A, import to B), their fields, and the ordering/idempotency rules — including what makes a handoff complete vs stuck.
  • Client observation: what a client holding the entity sees at each step of the handoff (before export, in transit, after import), and which reads/writes are legal in each state.
  • Failure recovery: import never arrives, import arrives twice, region B is down mid-handoff, coordinator dies between export and import — recovery path for each, with no entity duplication and no silent loss as the invariants.
  • Coordinator's role: what the coordinator world journals about a handoff (region map + handoff records per the investigation §5) and what it must never do (no cross-region transaction emulation).

Acceptance criteria

  • Design recorded in a doc under doc/ and linked from this issue
  • Export/import formats specified precisely enough to implement against pkgs/id/src/world/actor.rs's journaled-entry patterns (cf. CrossWorldWrite)
  • Each failure case above walked through with the recovery outcome stated
  • Anything buildable that falls out is filed as its own issue(s)

Program: ID platform expansion, epic #2056. Parent investigation: #2059. Consumer: #2046 (serverless edition v1 synthesis).

Activity

  1. added
    app/idPackage/app: id (pkgs/ or apps/ in this repo)
    on Oct 30, 2026
  2. added
    kind/designA design proposal (design-architect): multi-step, phased, decision-bearing
    priority/normalOrdinary priority (the default)
    effort/lLarge: multi-day or open-ended; expect staged check-ins
    on Oct 31, 2026
  3. dezren39 commented on Oct 31, 2026

    @dezren39
    MemberAuthor

    Intake review

    Related: parent #2059; sibling #2819.

    Notes: design-stage issue spun out of #2059; body already carries an acceptance section, so none was appended. No merged PR or commit found this run that implements this design — verified still live.

    Risks: design drift if the sibling design lands first and changes shared assumptions (hot-world sharding seams) — the two designs should be read together before either is implemented.

    Suggested approach: keep as design; when picked up, the design output should land as a linked implementation issue per the standing "a design is not finished work" rule.

    🤖 Agent note · filed

  4. added
    meta/enrichedIntake review done: acceptance criteria, links, and normalized labels added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    on Oct 31, 2026
  5. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · picked up

  6. dezren39 commented on Nov 1, 2026

    @dezren39
    MemberAuthor

    🤖 Agent note · triaged — CONFIRMED; design record below; design production routed as one packet (task 5802)

    • Agent: queue worker (repo-worker-code), lane F (repo-queue-worker-shift-f)
    • Session / ID: queue task 5744 (triage), claimed 2026-11-01T16:45Z
    • Verified against main a9e976ac245d94682ff7ada8dd5562cc58c9b00f (gh api contents + code search, 2026-11-01):

    Design record (the record the design doc must produce)

    • Decision to be made: the concrete journaled export/import record pair for a rung-2 region seam — fields, ordering, idempotency — plus the client-observation semantics and failure recovery, concrete enough to implement against actor.rs's journaled-entry patterns.
    • Options for the record shape: (1) dedicated JournalEntry variants on both sides (export record in A, import record in B) plus coordinator handoff open/close records; (2) handoff payloads carried inside generic Committed events; (3) coordinator-only records, shards journal nothing.
    • Recommendation: option 1. It is the only shape that keeps replay semantics explicit per variant (the world/mod.rs:475 convention), lets each shard's journal be the source of truth for its side (required for the coordinator-dies walk), and matches the CrossWorldWrite precedent. Option 2 buries a state machine inside opaque payloads; option 3 contradicts §5 and makes coordinator loss unrecoverable.
    • Invariants the design must hold: no entity duplication; no silent loss; journal-before-apply at every step; duplicate import is an idempotent no-op keyed by handoff_id; the seam stays visible (in-transit is a client-observable state, never papered over); the coordinator journals only membership, region map, and handoffs — no entity state, no cross-region transaction emulation.
    • Pinned questions for the design doc (each decided there, with trade-offs): record fields for export/import/coordinator pair; handoff_id derivation; transfer payload (full snapshot vs snapshot + journal reference) and its bounds; completion (import journaled in B and coordinator closure journaled) vs stuck (policy window after export) with a named recovery owner per state; the client state table (owned-by-A / in-transit / owned-by-B × legal reads/writes); the four failure walks from this issue's body; shared coordinator-journal conventions with sibling id: design: native-spill write lease protocol for hot worlds (from #2059) #2819, read together before writing (the intake review's drift warning).
    • Phasing — explicit one-packet statement: implementation packets are not enqueued now; before the format exists they would be speculative packet defects. The design production is one packet (below); buildable fallout gets filed as its own issues + packets by that packet, per this issue's acceptance. This issue stays open until the design doc lands.
    • Routing: packet packets/task5744-2818-region-seam-handoff-design.json, enqueued as queue task 5802, dedupe key repo-health:v1:fix:developing-today/code:issue:2818, queue repo-health-code-pilot, status queued.
    • Questions for Drew: none blocking. One deferred decision, named rather than stalled: shard re-merge/de-escalation is out of scope here — the investigation (§5) leaves it an open question for the id: design: ID serverless edition v1 (Cloudflare Workers) #2046 synthesis; recommendation is the handoff format must not preclude a later state-import design but does not design it. The design packet proceeds on that basis.
  7. added
    queue/queuedHas a queued task in the Supabase agent queue
    and removed on Nov 1, 2026
  8. 37 remaining items

  9. added
    queue/queuedHas a queued task in the Supabase agent queue
    source/creatorFiled by the new-issue-creator job
    task/triageQueue packet kind: verify, reproduce, and route this item
    on Nov 3, 2026
  10. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    agent-note:v1
    state: triaged (coverage pass)
    Coverage pass 2026-11-03: reviewed in the fleet-wide per-issue pass (drift / overlap / already-handled / stale / blockers / ordering). Verdict: still live; queued in repo-health-code-pilot as triage at priority 40 (priority/normal). If this is already handled at current main, the triage closes it with evidence; if it needs splitting, the triage files the split.
    Found via: coverage pass 2026-11-03

  11. added
    work/assignedQueue worker lifecycle state
    queue/claimedQueue worker lifecycle state
    and removed
    queue/queuedHas a queued task in the Supabase agent queue
    on Nov 3, 2026
  12. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    🤖 Agent note · picked up

  13. developing-today commented on Nov 3, 2026

    @developing-today
    Contributor

    🤖 Agent note · done (triage verdict: design deliverable ALREADY DONE; issue stays open for consumer adoption)

    Evidence (fresh, 2026-11-03):

    Routing: none new — #3870 owns the implementation, #2046 owns adoption. No branch, worktree, or competing design created. This coverage triage (task 7986) completes as triaged / already_done for the design deliverable.
    (Note: posted via the developing.today GitHub App — the user-token content-creation secondary rate limit is still active this shift.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app/idPackage/app: id (pkgs/ or apps/ in this repo)effort/lLarge: multi-day or open-ended; expect staged check-inskind/designA design proposal (design-architect): multi-step, phased, decision-bearingmeta/enrichedIntake review done: acceptance criteria, links, and normalized labels addedpriority/normalOrdinary priority (the default)source/creatorFiled by the new-issue-creator jobtask/triageQueue packet kind: verify, reproduce, and route this item

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions