Skip to content

refactor(graph): stop hashing the generation id into the code-graph namespace so heads supersede #836

Description

@ScriptedAlchemy

Context

Root-cause lane for the live-daemon memory bloat (PR #707 branch, issues #799 / #830):
code_graph_namespace(code_shard_id, generation_id) (runtime-core store_runtime/registry/graph.rs)
hashes the generation id into the graph-db namespace. Every published code generation therefore
becomes the permanent verified head of its own single-generation projection:

  • retire_replay refuses with CurrentVerifiedHead for every generation (it is never superseded),
  • retire_one_code_generation_replay blanket-retained until bc918144b/5d8fee38a (slice 1),
  • the shared staging Grafeo container (proj_*/tracedecay.grafeo, 15 GB live) accumulated the rows of
    13 generations, and Grafeo's LpgStore is heap-resident, so every open replayed all of it into RAM.

Slices 1–2 on #707 fix the symptom inside the current layout (dead heads retire; staging rows of a
sealed, installed, dependency-free generation are released after seal-install; retention never lazily
opens the staging engine). They leave the layout itself in place.

Proposed follow-up

Make the namespace generation-agnostic (per shard) so publishing generation N+1 supersedes N as an
ordinary projection-head replacement, and the existing superseded-head retirement path, CAS on the
projection head, and dependency semantics apply without per-generation special cases.

Why not in #707

  • Live persisted format: every existing container already has per-generation namespaces and sealed
    stores keyed by them; a cutover needs a migration or a typed ResetRequired → republish transition for
    every registered project, not a branch-local change.
  • Blast radius spans runtime-core (namespace derivation), store-runtime (code graph runtime, dependency
    locators), graph-db (publication CAS now conflicting between concurrent generations of one shard —
    desired, but a behaviour change), and the sealed-store identity.
  • Slice 2 + the retention fix already collapse the retained footprint; the deeper change is a structural
    simplification, not required for the memory fix. It should land as its own delivery slice with its own
    migration evidence.

Acceptance

  • Publishing a new code generation for a shard supersedes the previous head; the previous generation is
    retired through the normal superseded-head path (no CurrentVerifiedHead refusals in the release queue).
  • Existing per-generation containers migrate or republish through a typed state; no silent fallback.
  • Delete the per-generation special-casing that slices 1–2 had to add (sealed_only retention sweep for
    legacy heads, blanket-head tombstones) once no live store needs it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions