Skip to content

Pre-#211 delivery: accept the routing backfill, make On the main reading entry, and consolidate main for the storage move #219

Closed

Description

@holden

Outcome

One tested main that holds the accepted routing foundation and a working On → subject reader, with every remaining branch, worktree and database dependency accounted for, so that #211 may begin. This issue owns everything between the 27 September audit and #211's entry checkpoint. It relocates nothing.

Finish line = the entry checkpoint #211 inherits from #193: accepted main plus at most one designated integration branch; all local changes and evidence accounted for; migration-relevant restore defects addressed; preserved unaccepted code explicitly identified. Plus the owner's requirement, new here: On is the main reading entry, replacing Define, working locally on a safe dataset, with subjects linking to their distinct namespaces.

What this issue delivers and what it does not. Corrected, independently accepted code; a working On → subject reader proven on an isolated dataset; and a consolidated main handed to #211. It does not enable namespace addresses on the owner's working installation: that needs the reviewed population and reviewers, which #211 defers past relocation. That enablement is CP4, a linked follow-up issue opened at CP3, and #219 never describes itself as completing full Stage 2 acceptance.

The implementer, issue breakdown and PR sequence are open. Keep the foundation already built; do not start another architecture exercise. The prelaunch scope review (local: data/audits/2026-09-27-stage2-independent-review/PRELAUNCH-SCOPE-REVIEW.md) applies: no legacy-URL compatibility, no new lifecycle administration, no publication work.

Starting state, verified 27 September 2026 21:45 UTC

Item State
main 2a851e8 (#215)
#208 recovery repair 7860b9f, mergeable, A−. CodeRabbit Major on target_config/1 unfixed; two stale lines in docs/routing/stage-1-foundation.md
#216 backfill, stacked on #208 0415e5b, B−. The two audit defects (P1, P2) are still present: 0415e5b changed only the export reader and the task's argument parsing. A session is live in its worktree (dictionary-issue-194-stage-1-218d99)
#202 curated opening 7711e31, accepted A− on 26 September, unmerged, owner's merge decision
#210 curation runtime cb2e236, not accepted, no independent audit. Its two migrations are already applied to devils_dictionary_v2
#218 exemplar items (#212) 4839c6f, unrelated to routing; decided on its own merits, not by this issue
Consolidation branch codex/consolidation-2026-09-27 697ac8a Stale: predates #215, the last three #208 commits and #216
devils_dictionary_v2 main's schema plus #210's 20260927114948 and 20260927131023; routing and curation tables present and empty; nothing backfilled
Processes dev server 4007 SIGSTOPped since 25 September with ngrok --subdomain=wordhoard still pointed at it; #202 preview on 4017; scratch cluster on 5433 (/Volumes/LLM Models/dictionary-stage2a, 22 rehearsal copies)
Internal disk 5 GiB free; the breakdown and reclaim candidates are in the #211 comment
Independent audit local data/audits/2026-09-27-stage2-independent-review/: REPORT.md, PRELAUNCH-SCOPE-REVIEW.md, IMPLEMENTATION-HANDOFF.md, the probes in independent_backfill_test.exs, logs and artifact comparisons
The owner's example in the corpus Exactly one entity labelled Mars: object 1831413, stored kind concept, Q111, the planet. Seven lexemes with slug mars. No Mars deity exists; it is a clearly identified fixture. Mercury likewise: one entity, the planet

Re-verify all of this before writing anything. Conversational claims are not merge state.

A. Bring the backfill to acceptance

Work on #216's branch only in coordination with the live session there (or after it has committed and pushed and gone idle). Do not create a second version of the same fix.

A1 — P1, stale class evidence escapes the guard. Backfill.stale_pins/1 checks only the pins the evaluator emitted, and Policy.evidence_for/2 pins only matched paths; the run classifies from the exported graph. Required: every graph record the walk visited, matched or not, and every record that was missing at export time and now exists, is validated for currency before a record is classified. Either pin them all in the evaluator result or re-read the batch's graph subset from source_records and compare it with the export. A change defers the record as evidence_changed and requires a fresh decision and a fresh review fingerprint. Regressions: changed unmatched ancestry (the audit's Polish probe), newly available ancestry, contradictory evidence.

A2 — P2, a refused allocation leaves a permanent override. In Backfill.confirm/3 the override is written before Pages.ensure/3 and Ledger.allocate/3; available/2 never reads page lifecycle; classification_decisions refuses DELETE, so the stray override is permanent history. Required: confirmation is atomic per record: after any refusal, decision, page, path and ledger state equal what they were before the record was processed, the record's item still says refused with its reason, and the batch continues. Checking lifecycle in available/2 and wrapping override → page → allocate in an explicit SAVEPOINT / ROLLBACK TO SAVEPOINT via Repo.query! is one implementation; a nested Repo.transaction rollback is not, because it aborts the whole batch (Stage 1 decision 7: refusals are per record). Concurrency behaviour under two callers is part of the acceptance, not only the happy path. Regressions: retired page (the audit's probe), :target_has_other_role, :target_kind_mismatch, a path taken between available/2 and allocate/3, a canonical appearing between them. Each asserts decision, page, path and ledger state unchanged after the refusal.

A3 — Documentation. Correct docs/routing/stage-2/backfill.md lines 22, 35 and 75, the README's "against the evidence they reviewed", and the ADR §8 atomicity statement's coverage. Keep the audit's probes as tests, not as prose.

A4 — #208. Fix target_config/1: drop every inherited endpoint key (socket_dir, socket, endpoints, hostname, port) before merging a URL target, then resolve. Test: a URL without a port under DD_DATABASE_PORT=5433 reaches 5432. Update docs/routing/stage-1-foundation.md lines 140 and 278. This tool is what #211 will use to prove the move across clusters.

A5 — Evidence. Focused routing tests and mix precommit on a private MIX_TEST_PARTITION. On a fresh copy on 5433 restored from the frozen devils_dictionary_stage2r_c1, re-run docs/routing/stage-2/rehearsal/backfill.exs: repeat-run identity, SIGKILL resume, exact restore. The P1–P3 projection evidence and the catch-up measurements may be reused only if the final diff touches nothing the projection depends on (Absorb.*, materializers, resolver, fingerprints); state that check against the actual merged diff, not against the plan. No full-corpus re-projection is needed for this issue; say so and why.

A6 — Reassessment. An independent reading of the corrected heads against REPORT.md. Record the grade on #194. Accepted means: both probes pass as regressions, the rehearsal gates hold on the corrected code, and the documentation matches the code.

B. On is the main reading entry

B1 — Reconcile the ADR. Amend docs/adr/0004-public-routing.md §4 and §6:

  • /on/:slug is the everyday lexical entry, resolved exactly as /define/:slug is today (Lexicon.lookup/1, Lexicon.WordPage), for any lexeme. It needs no pages row: there are 1.5 million lexemes and an On page cannot depend on someone authoring an overview. That reconciles the owner's requirement with §4's "lexical availability cannot depend on someone writing an overview".
  • An authored overview stays optional and layers onto the same URL: a pages row with role overview whose path is /on/<routing slug> (the on editorial prefix in priv/routing/namespaces.json), allocated through the ledger only when authored. No second content or approval system; the page–composition binding stays the deferred Stage 4 migration.
  • /define/:slug is removed and every internal link moves in the same PR. No redirect layer: nothing has been public. /words/:id/:slug stays as the exact lexeme.
  • URL ownership. /on/:slug is the aggregate reader for everything the lexeme slug reaches, as /define/:slug is today: several lexemes share a slug by design (C++, C+ and c; Mars, mars, MARS). An exact selection of a word, from search or from a card, navigates to /words/:id/:slug, the existing identity-carrying route, and survives reload. The slug alone never identifies a word, so On never claims to.
  • Authored overview precedence. When an overview page exists, its current revision renders above the lexical aggregate on the same URL; with no lexeme behind it, the overview renders alone (200).
  • Overview ↔ lexical association is identity, never a shared label or slug (28 Sep re-audit, correction 2). An overview is loaded only by its allocated address: /on/:slug resolves the requested path through the ledger, and nothing re-slugifies a displayed lemma to find one. It is presented as the treatment of the lexical aggregate only when its current revision holds a supplies_lexical_material membership naming one of the aggregate's lexemes (or one of their senses). An overview whose address collides with an unrelated lexical slug is shown as a separate, labelled choice on that page, never as one treatment. From a lexical page (/on/:slug or /words/:id/:slug), the overviews linked are exactly those whose membership names one of the page's lexemes, at their allocated path in the current mode; the overview links back to each exact word it names. Exact selections keep their identity through navigation and reload (/words/:id/:slug, drawers included). The overview's path is Address.build("on", Policy.slug(lemma)); where that differs from the lexeme slug (c-plus-plus vs c), the lexical page links to the overview's allocated path and the overview links back to the exact word. No silent redirect between the two spellings, and never a different word or an unrelated overview chosen by a shared slug.
  • Subject pages live at /people/:slug, /organizations/:slug, /places/:slug, /events/:slug, /works/:slug, /concepts/:slug, /nature/:slug, /subjects/:slug: eight explicit routes, never a catch-all prefix, so account, evidence and operational paths are untouched. They resolve only through Routing.Resolver, which reads the ledger and nothing else. /entities/:id/:slug stays as the exact-identity route for a subject with no allocated address.
  • Reading modes. On current main, Resolver.decide/4 answers :unavailable for any page that is not published and Resolver.link/1 returns :error for it; the backfill creates only drafts and publication is Stage 5. So the reader needs one explicit internal mode, shared by link generation and direct namespace requests: Resolver.resolve(raw, mode:) and Resolver.link(page_id, mode:) with :public (today's behaviour, the default) and :internal (a draft page with a canonical path resolves and links, marked as draft in the UI). Internal mode is on in dev and test configuration and for an authenticated internal contributor; it never changes publication_state, approvals or the ledger. Tests prove public mode still withholds every draft.
  • Reading-mode eligibility, applied everywhere (28 Sep re-audit, correction 1). The mode comes only from trusted configuration (:internal_reading, set in config/dev.exs and config/test.exs and refused by a test in prod.exs, runtime.exs and config.exs) or from an authenticated internal contributor or reviewer; no request parameter can grant it. Public mode shows only published pages; internal mode shows published and draft (marked as draft). withdrawn content is withheld in both modes, and lifecycle rules (merged, split, retired, tombstones) are identical in both. The same eligibility applies to authored overviews, to their members, and to aliases and equivalent spellings, which answer with the destination's outcome in the same mode: a public request never redirects to a draft. Existing public tombstone semantics are unchanged. Configuration enabling internal mode for every local request is a development convenience: a dev server exposed through a tunnel exposes drafts, so the tunnel is closed first (D4).
  • Standards rationale: schema.org and Wikidata inform which family a subject belongs to (evidence, via classification-rules.json); they do not prescribe the URL taxonomy, which is ours. Say this once in the ADR and stop.

B2 — Routes and behaviour.

Request Public mode Internal mode
GET /on/:slug, lexeme(s) found 200: the word page (headword, senses, sources, quotations, culture, examples, the thing) plus the Subjects section; title "On "; an authored overview, if any, above same
GET /on/:slug, no lexeme, no overview 404 with did-you-mean (existing behaviour); never a subject substituted by label same
GET /on/:slug, overview only 200 if the overview is published, else 404 200 for a published or draft overview (draft marked); withdrawn 404
GET /on/:slug, lexeme(s) and a draft or withdrawn overview 200: lexical content only, the overview withheld 200 with the draft overview (marked); a withdrawn one withheld
GET /on/:slug, an overview whose lexical membership names none of the slug's lexemes 200: the lexical aggregate, with the overview as a separate labelled choice (if visible in the mode) same
GET /<family>/:slug, canonical, published 200 subject page 200
GET /<family>/:slug, canonical, draft 404 (:unavailable) 200 with a draft mark
equivalent spelling (case, Unicode form, trailing slash) 301 to the canonical if the destination is published, else the destination's outcome (404) 301 if the destination is published or draft
alias of a moved or merged page 301, one hop, same destination rule 301, same destination rule
missing 404 404
retired / tombstone 410 410
invalid request (bad encoding, dot segment, NUL) 400 400
corrupt ledger state 500 with diagnostics, logged 500
choice / split a plain list of successors; no design work, no such data exists same
GET /entities/:id/:slug, GET /words/:id/:slug unchanged unchanged
Enter in search (go) Lexicon.lookup/1 hit → /on/<lexeme slug>; miss → results same
a selected word result /words/:id/:slug (exact identity) same
a selected subject result Resolver.link(page_id, mode) when it answers a path, else /entities/:id/:slug same

Every internal link to a subject goes through one helper that asks the resolver in the current mode and falls back to the /entities route. No caller derives a namespace from a stored kind or label, and no caller rebuilds a slug.

B3 — Subjects section contract. Two sources, kept distinct on the page:

  1. Curated membership is authoritative. When an overview page has a current revision, its page_memberships in stored order supply the curated content: discusses_subject members are subjects, editorial_association members are shown as associations (Putin/poutine is an association, never identity), supplies_lexical_material members are lexical. A member whose name differs from the page title is still a member. Withdrawn or ineligible content is withheld, not replaced. Actual approvals are read, never inferred.
  2. Discovered candidates are labelled as such. The union of %WordPage{}.thing (concept and disagreement), Encyclopedia.candidates_for/2's may_refer_to set, and registry entities whose object_names.name or entities.preferred_label equals the lemma after NFC and case folding. Deduplicated by object identity, never by label; a candidate already curated is not shown twice.

Each subject is one card. Pages of role subject and edition count (the backfill creates edition pages in /works). State comes from the current classification_decisions row and the page, read in one bounded query per On page (limit, stable order: addressed first, then family, then label; locale en; active identities only), measured on corpus-scale data before merge because NFC/case-folded matching is not indexed today.

Card state Link Shown
address allocated, page resolves in the current mode canonical path family badge, label, description; "draft" mark in internal mode
address allocated, draft, public mode /entities/:id/:slug "not yet public"
mapped, no address yet /entities/:id/:slug "no public address yet"
ambiguous or unmapped /entities/:id/:slug "awaiting classification review" with the candidate families
no decision /entities/:id/:slug "unclassified"

Nothing draft is presented as public; the section reads state, it never writes it. The page–composition binding (ADR §4) stays deferred: this issue renders authored overview revisions and existing memberships, and does not claim Stage 4. The curated opening (#202) keeps its own development-only fixture reader; wiring Curation.Published into the web layer is named here as unfinished, not delivered.

B4 — Slugs. The On route accepts the lexeme slug; an authored overview's path uses Routing.Policy.slug/1. The two can differ (C++ → lexeme slug vs c-plus-plus). Fixtures for C++, C+, c, Polish/polish, capitalization collisions and a combining-mark label each have a passing outcome under B1's precedence rule: the aggregate On page lists every word the slug reaches, the exact word is reachable at /words/:id/:slug, an authored overview is reachable at its allocated path and linked from the lexical page, and nothing is silently redirected or substituted.

B5 — Wireframes.

/on/mars                                              ┌ search ─────────┐
On Mars                                               └─────────────────┘
────────────────────────────────────────────────────────────────────────
[headword · pronunciation · forms]           ┌ the thing ─────────────┐
                                             │ Mars  Q111 ↗           │
Senses (by source, as today)                 │ fourth planet …        │
  1. …                                       └────────────────────────┘
  2. …
Quotations · Culture · Examples (as today)

Subjects                                          3 subjects, 2 addressed
┌──────────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐
│ NATURE               │ │ SUBJECTS  (fixture)  │ │ WORKS                │
│ Mars                 │ │ Mars                 │ │ Mars                 │
│ fourth planet from … │ │ Roman god of war     │ │ 2012 album           │
│ → /nature/mars       │ │ → /subjects/mars     │ │ no public address yet│
└──────────────────────┘ └──────────────────────┘ │ → /entities/…        │
                                                  └──────────────────────┘
/nature/mars
NATURE · Mars                                  On Mars ↗   Q111 ↗
────────────────────────────────────────────────────────────────────────
[the existing entity rendering: description, details, cited as,
 named under, examples, culture — EntityLive's content, unchanged]

[provenance ▸]  (a drawer: address, allocated <date> by <reviewer>, evidence <fingerprint>)

Only the planet is a real Mars entity; the deity and the album are marked fixtures. The subject page shows content and identity; review actors, fingerprints and allocation timestamps live in an inspection drawer, not a permanent block. The card states use the existing card, badge and eyebrow components; run /ui before writing markup.

B6 — Tests. ConnTest for every row of the B2 table, direct HTTP status and location headers included; LiveView navigation for search → On → subject and back; the Mars pair (planet allocated to /nature/mars, the deity fixture to /subjects/mars); a same-namespace collision with distinct qualified addresses (two of the corpus's Butterfly works); B4's slug fixtures; a lexeme with three entities of which one is addressed; the /words/:id exact lookup returning 404 for a missing id; public mode withholding a draft that internal mode shows; an editorial member whose name differs from the page title; an association that is not identity; withdrawn content withheld; an edition page in /works; an authored overview whose address collides with an unrelated lexical slug (shown as a separate choice, not one treatment), checked through navigation and reload; public-mode overview/alias requests never reaching a draft; and a URL parameter unable to switch the mode. Fixtures are marked and never minted into the corpus.

B7 — Demonstration. Run the dev server against a disposable demo database on the 5433 scratch cluster, restored with the corrected tooling from the frozen devils_dictionary_stage2r_c1 and backfilled with the rehearsal review file (105 rehearsal allocations, clearly labelled rehearsal). Do not write into devils_dictionary_stage2r_bfa2, the verified restored copy: it is retained audit evidence. Allocate the Mars fixtures on the demo database with a marked rehearsal review, in internal mode. Record screenshots of search → On Mars → /nature/mars and /subjects/mars, and of a subject card in each of the five defined states (B3's table). The "real working population" on the development corpus is demonstrated only after the C decisions and the move; do not conflate the two.

C. Corpus decisions: prepare, do not act

Nothing here writes to devils_dictionary_v2.

  • Schema reconciliation — decided by the owner on 27 September: (i) merge Curation runtime stage A: one private local model behind one slot (#195) #210 after an independent audit. The audit must check one thing the PR description admits: what ran against v2 at 13:11Z was an early draft of 20260927131023. Compare the four runtime tables' live schema on v2 (columns, constraints, indexes, trigger functions) with the schema the PR's final migrations produce on a fresh database. Identical → v2 equals main on merge and nothing else is needed. Different → record the difference and the repair (a follow-up migration, never a manual edit of v2) before calling the schema reconciled. Options (ii) and (iii), the rehearsed rollbacks, remain the fallback if the audit fails.
  • Deferred to after relocation, per Repeatable dictionary migration: prove external setup against the retained working baseline #211: the corpus catch-up (option A), the six artwork images, the 170-record population, the named reviewers. Record on Routing before launch: On overviews, standards-informed namespaces, and stable subject URLs #194 exactly what remains and why the checkpoint is nevertheless satisfied. After A1–A4, regenerate the export, the population and every review fingerprint before any persistent run; rehearsal confirmations are not approvals; the other 100,614 entities stay visible with their dispositions.

D. Consolidate and hand off

D1 — Merge order. #208 (after A4) → #216 (after A1–A3, rebased onto the new main, base retargeted to main) → #210 (after the independent audit in C) → #202 (accepted; the owner's merge decision recorded) → the On reader PR(s) from B, with their docs. Merge commits, not squash. Validate main after each merge: compile with warnings as errors, the combined migration path from zero and from v2's current version, mix precommit.

D2 — Worktrees — approved by the owner on 27 September. Preservation comes first (28 Sep re-audit, correction 3): before a worktree is retired, any unique ignored configuration it holds is preserved privately (or shown to be byte-identical to a retained copy) and any evidence it alone holds is copied into the retained evidence; credentials never go into git or an issue comment. The owning tool's archive is used only for a worktree that tool actually manages (a session registered with it), not because of a directory name. Before removing any worktree: no process has it as its working directory (lsof -d cwd), git status --ignored shows nothing beyond build caches, .env and the data symlink, and the branch is merged with no commits ahead. App-managed worktrees (.claude/worktrees/*) whose sessions are idle are reclaimed through the managed archive (the session archive tool), which stops the session and removes its worktree; the rest with git worktree remove after the data symlink is confirmed a symlink and removed. claude/dictionary-word-page-grouping-2c67e4's two dirty files (a local port override) are preserved as a patch under data/worktree-reclaim-2026-09-27/ before removal. Leave the locked claude/158-build1, the two ~/.codex worktrees, and any worktree with a live process to their owners. Remove the #216 worktree only after its merge and with its session gone. Record what was removed and the space recovered.

D3 — Databases. A name is not a disposition (#211). Record one inventory row per database on both clusters: server identity (system_identifier), name, owner, purpose, what depends on it, and a proposed disposition. This issue inventories and proposes; it drops no database. A drop needs separately established authorization naming that exact database and server; an ended session, a test-partition name or this inventory is not that authorization. devils_dictionary_bing135, devils_dictionary_74_verify and devils_dictionary_dev (12 GB; config/dev.exs still names _dev as the Gate 0 baseline) and the 22 scratch copies on 5433 get rows and a recommendation; keep at least stage2r_c1, stage2r_caughtup, stage2r_bfa2 and the dumps.

D4 — Processes. The owner's terminal: fg the 4007 server and stop it cleanly, close the ngrok tunnel, stop the 4017 preview. Record that quiescence in the handoff.

D5 — Checkpoint. Re-cut the consolidation checkpoint against the new main; verify local main equals origin/main; name the integration branch, if any, and what it preserves. Then repeat the B6 journey and the B7 demonstration from that exact final main revision, with #202's curated opening present, because both changes touch the reader. Evidence from an earlier branch does not satisfy this.

D6 — Documentation. README routing paragraph, ADR §4/§6/§8, docs/routing/stage-2/README.md, backfill.md, stage-1-foundation.md, and a status comment on #194 with the honest foundation grade.

D7 — Handoff to #211. One comment on #211: accepted code SHA and schema baseline (schema_migrations head), migrations main will expect after the move (#216's 20260927200717, and #210's if unmerged), preserved outstanding work, cleared blockers, and the reclaim already done.

Schema, read against devils_dictionary_v2

Section B is expected to add no tables and no columns; if the B3 contract turns out to need the smallest binding change, the PR justifies it and this section is updated first. It reads:

pages                     public_paths              classification_decisions
  id                        id                        id
  role                      path                      object_id
  locale                    kind                      origin · status · family
  target_object_id          original_page_id          candidate_families[]
  publication_state         destination_page_id       rule_ids[] · reasons[] · warnings[]
  lifecycle_state           last_route_change_id      policy_version
  merged_into_page_id       inserted_at · updated_at  evidence_fingerprint
  current_revision_id                                 source_pins[]
  canonical_path_id                                   reviewer_actor_id · reason
  last_route_change_id                                supersedes_id · is_current
  inserted_at · updated_at                            inserted_at

plus entities, object_names (object_id, name, language_tag, name_kind, source_record_revision_id) and the existing lexicon. #216 adds routing_backfill_runs and routing_backfill_items (migration 20260927200717); per #211 it is applied to the source only after preservation passes.

Decisions

Decided on 27 September 2026:

  1. Schema reconciliation: (i), merge Curation runtime stage A: one private local model behind one slot (#195) #210 after an independent audit (C above).
  2. Worktree reclaim: approved under D2's checks. Databases are not covered by this approval; D3 records dispositions only.

Decided on 28 September:
3. Merge #202: yes. Merged as e1d60b0, in D1 order.

Still open:
4. Stop 4007, the tunnel and 4017 (owner's terminal).
5. After the move, not now: catch-up A, the six images, the population, the reviewers, and CP4's enablement on the working installation.

Checkpoints

Non-goals

Relocating PostgreSQL, source or agent storage; legacy redirects; publication, indexing, sitemaps or Stage 5; new move/merge/split/retire administration; any write to devils_dictionary_v2 (backfill, catch-up, migration) before the move; minting corpus records for demonstrations.

Acceptance

Independent review of the code and the evidence, not the implementer's grade. The final comment states what works in the app with the demonstration, which defects were fixed, checks run and their limits, what was merged, what remains outside main, the foundation grade, whether #211 can begin, and which dataset the demonstration ran on (isolated demo now; the working installation only at CP4).

Activity

  1. holden commented on Sep 27, 2026

    @holden
    MemberAuthor

    Brief audit answered: body amended, two decisions recorded

    An independent audit of this brief (27 September, local data/audits/…/ISSUE-219-BRIEF-AUDIT, grade B) made six amendments and several smaller corrections. I checked each against main 2a851e8 before applying it; the body above is the amended version.

    Verified and applied

    1. Draft pages were unreadable under B2 as written. Resolver.decide/4 answers :unavailable for any unpublished page and Resolver.link/1 returns :error for it; the backfill creates only drafts. B1/B2 now define one explicit internal reading mode (mode: :public | :internal) shared by link generation and namespace requests, on in dev/test and for internal contributors, never touching publication state; public mode is tested to keep withholding drafts.
    2. Selection identity. Lexeme slugs are non-unique by design (C++, C+, c; Mars/mars/MARS). /on/:slug is the aggregate reader; an exact word selection navigates to /words/:id/:slug and survives reload. Authored-overview precedence and the routing-slug vs lexeme-slug case now have passing outcomes, not "or refused". Eight explicit family routes, no catch-all.
    3. Curated membership is authoritative. B3 now reads the overview revision's page_memberships (discusses_subject, editorial_association, supplies_lexical_material) in stored order, shows Encyclopedia.candidates_for/2's may_refer_to set and name matches as discovered candidates deduplicated by identity, includes edition pages, and requires one bounded, measured query. The composition binding stays deferred and Curation.Published is named as unfinished, not delivered; I did not widen scope into Stage 4.
    4. Rehearsal vs the working installation. The Outcome now says what Pre-#211 delivery: accept the routing backfill, make On the main reading entry, and consolidate main for the storage move #219 does and does not deliver; CP4 (enablement on the external working installation, after Repeatable dictionary migration: prove external setup against the retained working baseline #211) is a linked follow-up with the one-sentence success criterion.
    5. Evidence protection. The demo runs on a disposable database restored from the frozen stage2r_c1, not on stage2r_bfa2. D2 requires process and ignored-file checks and managed archival; D3 requires a per-database inventory row and forbids name-based drops.
    6. The reader on final main. D1 includes the reader PR(s); D5/CP3 repeat the B6 journey and B7 demonstration from the final main revision with Curated opening section, Phase 1: manual fixture, Bierce-first lead, sourced highlights (#156) #202 present.

    Smaller: B2 covers 400/404/410/500 and aliases; the Mars album is marked a fixture; provenance moves into an inspection drawer; A2 states atomicity as the criterion with savepoints as one implementation; A5's reuse of projection evidence is conditional on the final diff; the schema section no longer promises zero columns unconditionally.

    Decisions recorded from the owner (27 September)

  2. holden commented on Sep 27, 2026

    @holden
    MemberAuthor

    Status, 28 September 00:20 (UTC+2): C executed, D2 half done, prompts for the three checkpoints

    C — schema reconciliation, option (i), executed. #210 was audited (findings: precommit 2,335 tests / 0 failures; nothing blocking; three should-fix items) and merged as d31d212 with a merge commit, branch kept. The audit found the one drift the brief anticipated: local_model_configs_shape on devils_dictionary_v2 is the early draft. PR #220 repairs it idempotently and is proven on the draft form, on a fresh database and through rollback; it is applied to v2 only after preservation, per #211. The three should-fix items are #221 (stale dispatch refusal, orphaned ollama runner children on stop, two vacuous benchmark cells). D1's order is now #208 → #216 → #220 → #202 → the reader PR(s).

    D2 — worktrees. Nineteen idle sessions on merged branches were archived through the managed archive; no process holds any of the 16 worktrees now. The directories themselves (about 5.5 GB) remain: the app's cleanup did not delete them, most likely because each holds the untracked data symlink, and this session's permission mode blocks git worktree remove. The owner runs the removal (command list in the session recap). dictionary-word-page-grouping-2c67e4's two dirty files are preserved as a patch under data/worktree-reclaim-2026-09-27/. Left alone on purpose: word-level-tier-shelves-720f8b (its session is pinned), exemplar-layer-build-2-32d57f (detached, unverified), rail-counts-and-stack-cap and 172-final-sweep (a live claude process each), 144-kit-de9e9d and 143-spotify-music-6bb8a8 (branches not merged), the locked agent-af1d…, and the two ~/.codex worktrees. Internal disk: 7 GiB free, from 4.

    Sequencing. A (CP1) and B (CP2) are independent and touch disjoint code; run them in parallel. B is the critical path. D (CP3) is short once both have merged. One session for all of it is possible in the order A → B → D, but B's size makes three sessions the faster and safer shape.

    Prompt A — CP1, backfill acceptance and the merges

    Work in razrfly/dictionary on issue #219, section A only (checkpoint CP1). Before changing anything, read #219, the #194 audit comment it links, the local audit files under data/audits/2026-09-27-stage2-independent-review/ (REPORT.md, independent_backfill_test.exs, adversarial-tests.log), and PRs #208 and #216 at their current heads; verify main, the PR heads, and whether a session is live in the #216 worktree (dictionary-issue-194-stage-1-218d99).
    Deliver A1–A6: P1 (every graph record the walk visited, or that was missing at export and now exists, is validated for currency; a change defers as evidence_changed); P2 (per-record atomic confirmation: decision, page, path and ledger state unchanged after any refusal, the item still refused, the batch continues; cover retired pages, target_has_other_role, target_kind_mismatch, a path taken and a canonical appearing between check and allocate); A3 docs; A4 target_config/1 on #208 plus the stale Stage 1 lines; A5 evidence (focused routing tests and mix precommit on a private MIX_TEST_PARTITION; the backfill rehearsal re-run on a fresh 5433 copy restored from devils_dictionary_stage2r_c1); A6 an independent re-reading against REPORT.md, grade posted on #194.
    Then merge in order with merge commits, branches kept: #208 → #216 (rebased onto main, base retargeted to main) → #220 if still open.
    Constraints: push to a PR branch only if no other session is live there, otherwise stack a branch on it; never write to devils_dictionary_v2; every mix ecto.* and mix test call carries MIX_ENV=test and the private partition; the audit's probes become regression tests; keep independently executed checks separate from author claims; reuse the projection evidence only if the final diff leaves Absorb.*, the materializers, the resolver and the fingerprints untouched, and say so.
    Finish with: what changed, both probes' results, test counts, rehearsal results, the grade, what merged, what remains.

    Prompt B — CP2, On as the main reading entry

    Work in razrfly/dictionary on issue #219, section B (checkpoint CP2), on a branch from current main in your own worktree. Read #219 in full (B1–B7 are the contract), ADR 0004 §4 and §6, lib/devils_dictionary/routing/{resolver,pages,address,policy}.ex, lib/devils_dictionary_web/live/{word_live,entity_live,home_live}.ex and the Word, Thing and Examples components before designing. Run /ui before writing markup.
    Deliver: the public/internal reading mode on Resolver.resolve/2 and Resolver.link/2 (public unchanged and tested to withhold drafts); /on/:slug replacing /define/:slug with every internal link moved in the same PR and no redirect layer; /words/:id/:slug as the identity-carrying route for exact selections; the eight explicit family routes; one link helper; the Subjects section per B3 (curated memberships authoritative, discovered candidates labelled and deduplicated by identity, edition pages included, one bounded query measured on corpus-scale data); B4's slug fixtures with passing outcomes; every B6 test in both modes, ConnTest and LiveView; the ADR §4/§6 amendment and README paragraph.
    Demo (B7): a disposable database on the 5433 scratch cluster restored from devils_dictionary_stage2r_c1 with the corrected tooling and backfilled with the rehearsal review file (run the backfill from #216's branch if it is not merged yet); Mars fixtures allocated in internal mode with a marked rehearsal review; screenshots of search → On Mars → /nature/mars and /subjects/mars, and of each card state, attached to #219. Never write to devils_dictionary_v2 or to devils_dictionary_stage2r_bfa2.
    Constraints: private MIX_TEST_PARTITION; MIX_ENV=test on every ecto task; fixtures marked, never minted into the corpus; the curated opening (#202) and Curation.Published wiring are out of scope and named unfinished; no publication-state changes.
    Finish with: the B2 table as implemented, test counts, the screenshots, what is deferred, and a PR against main.

    Prompt D — CP3, consolidation and the #211 handoff

    Work in razrfly/dictionary on issue #219, section D (checkpoint CP3). Entry condition, verify it: #208, #216, #220 and the reader PR(s) are on main; the #202 decision is recorded. Read #219 D1–D7, #211, and the #193 checkpoint comment.
    Deliver: validate main (compile with warnings as errors; the combined migration path from zero and from v2's current version on a private partition; mix precommit); repeat the B6 journey and B7 demonstration from the final main revision with #202 present; D2, record what was removed and the space recovered (the owner deletes directories; never force-remove); D3, the per-database inventory for both clusters with a disposition column, dropping nothing; D4, record quiescence (4007, the ngrok tunnel and 4017 are the owner's to stop); D5, re-cut the consolidation checkpoint against main and verify local main equals origin/main; D6, docs (README, ADR §4/§6/§8, docs/routing/stage-2/README.md, backfill.md, stage-1-foundation.md) and the #194 status comment with the honest foundation grade; D7, the handoff comment on #211 (code SHA, schema_migrations head, migrations main expects after the move, preserved outstanding work, cleared blockers); open CP4 as a linked issue.
    Constraints: nothing written to devils_dictionary_v2; merge commits; no database deleted; MIX_ENV=test on every ecto task.
    Finish with: the handoff text, what remains outside main, and whether #211 can begin.

  3. holden commented on Sep 27, 2026

    @holden
    MemberAuthor

    D2 correction and ownership note (28 Sep 00:35 +02). The session "Issue #219 routing and aggregate reading" now works from the worktree .claude/worktrees/source-listing-ui-ef5442 (detached at d31d212), so that directory is excluded from the reclaim list; the other fifteen archived worktrees plus rail-sr-only-fix still hold no process and remain for the owner to remove. That session owns A, B, D1 and D5–D7, the #211 handoff and the CP4 issue from here. This audit session is done with #219 apart from #220, which is complete, pushed, CodeRabbit-clean and ready for review and merge (D1 order: #208 → #216 → #220 → #202 → reader PRs). D3's database inventory has not been started by anyone.

  4. holden commented on Sep 27, 2026

    @holden
    MemberAuthor

    Implementation started (28 Sep, 00:45 +02): contract corrections applied, A1–A4 pushed

    The re-audit's three corrections are now in the body (B1, B2's table, B6, B7, D2, D3), each marked "28 Sep re-audit":

    1. Reading-mode eligibility everywhere. Public mode = published only. Internal mode = published + draft (marked). Withdrawn content is withheld in both modes, and lifecycle rules are the same in both. Aliases and equivalent spellings answer with the destination's outcome in the same mode, so a public request never redirects to a draft. The mode comes from trusted configuration (:internal_reading: dev/test only, refused in prod config by a test) or from an authenticated internal contributor or reviewer. No request parameter can set it.
    2. Overview ↔ lexical association by identity. An overview is loaded only at its allocated address. It is presented as the lexical aggregate's treatment only through a supplies_lexical_material membership that names one of the aggregate's lexemes (or senses). An unrelated collision is shown as a separate labelled choice, with its own fixture test.
    3. Cleanup. Unique ignored configuration and evidence are preserved privately before retiring a worktree. The managed archive is used only where the tool manages the worktree. D3 inventories and proposes, and drops nothing without authorization for the exact database and server.

    A1–A4, pushed. Independent re-review (A6) and rehearsal (A5) are next.

    Head What changed Evidence so far
    A1 (P1) #216 f3c2339 Policy.classify/3 lists every record it read, matched or not, and every record it looked for and did not find (dependencies). The backfill defers as evidence_changed unless each is at the exported revision or still absent. The fingerprint and source_pins cover the same set. The audit's Polish probe, plus: newly arrived ancestry, a contradicting unmatched branch
    A2 (P2) #216 f3c2339 The override, page and address are written under one savepoint. A refusal rolls back to it, records refused and the batch continues. available/2 refuses a page that is not active. The retired-page probe, plus: other-role page, a path taken mid-flight and a canonical given mid-flight by a concurrent connection, a kind change mid-batch, and a lock-bypassing writer (the batch rolls back and the rerun resumes). Each compares the record's decisions, pages, paths and ledger rows before and after. 8 of the 9 new tests fail on 0415e5b.
    A3 #216 975919e, 12826de backfill.md (step 1, refusals, limits), README, ADR §8, Stage 1 decision 7, and the Stage 2 README
    A4 #208 7ef322d target_config/1 drops every inherited endpoint key before merging a URL (nil-ing them, because start_link/1 re-merges the configuration), then resolves; same_database?/2 reuses Snapshot.same_database?/2. Also the two stale Stage 1 lines and one runbook sentence. The regression fails on the old code (the URL inherited a dead port)

    On private partition _219a: routing directory 185 tests + 3 doctests, 0 failures; recovery files 24 tests, 0 failures. Full precommit runs on the merged states before each merge. #216 will be brought up to main with a merge commit, not a force-push. Its branch belongs to another session.

  5. holden commented on Sep 28, 2026

    @holden
    MemberAuthor

    #219: delivered on main 16e35ca; three owner steps remain before it closes

    Grade: A−. The code reached every acceptance bar under independent review, and the reader works from main on an isolated dataset. It is not an A for two reasons:

    • the owner steps below are still open, so the finish line's "local main equals origin/main" and quiescence are not yet true;
    • some P3 limits are recorded rather than removed. The largest is that name discovery misses labels whose case folding differs between PostgreSQL and Unicode, such as Turkish İ.

    What works, and where it was shown

    • On is the reading entry. /on/:slug reads every word a slug reaches, and /define is gone. /words/:id/:slug holds one exact word through drawers and reloads.

    • Subjects open at their own addresses, through eight explicit family routes, in public and internal reading modes.

    • Demonstrated from main 04f97bc, with Curated opening section, Phase 1: manual fixture, Bierce-first lead, sourced highlights (#156) #202's curated opening present, on a disposable copy of the corpus. The copy, devils_dictionary_stage2r_219final on the scratch cluster, was restored from the frozen capture with the corrected tooling and verified exact. It was then backfilled with rehearsal reviews, which are not approvals, and given marked Mars fixtures. The record and screenshots cover:

      • search → On mars → /nature/mars (the corpus planet, Q111) and /subjects/mars (the deity fixture), then back and reload;
      • public mode withholding every draft, with no draft address linked anywhere;
      • the five card states;
      • 375 px.

      Every direct request, in both modes, answered as B2 requires.

    • The owner's working installation is not enabled. That is CP4.

    Defects fixed, and the independent reviews

    Checks and limits

    Merged, in D1 order, with merge commits and branches kept

    #208 484d21e → #216 903f82b → #220 64f12cd → #202 e1d60b0 → #225 04f97bc → #226 (docs) 16e35ca.

    Documentation

    • README: routing status table and the On paragraph.
    • ADR 0004: status, §4, §6 and §8.
    • docs/routing/stage-2/README.md.
    • backfill.md: the corrected-code re-run and the limits.
    • stage-1-foundation.md, recovery.md.
    • docs/routing/on-demo/.
    • docs/operations/consolidation-checkpoint-2026-09-28.md, which records:
      • the merges and how each was validated;
      • the migrations main expects after the move;
      • preserved work, worktree dispositions and the database inventory;
      • its predecessor, carried over as that date's record.

    Can #211 begin?

    Yes, once D4 is done. The handoff is on #211. CP4 is #224. Its decisions stay with the owner, after the move: the catch-up, the population rule, the named reviewers, and enabling addresses for the everyday reader.

    What remains for the owner

    1. D4: stop the 4007 server (PID 13180), its ngrok tunnel (PID 14391) and the 4017 preview (PID 32882).
    2. D5: fast-forward the main checkout's main, 66 commits behind origin/main. Do not start its dev server against v2 until Repeatable dictionary migration: prove external setup against the retained working baseline #211's preservation is done: the development page would offer to apply the two pending migrations.
      git -C ~/Code/projects-2026/dictionary pull --ff-only
    3. D2: remove the worktrees below. None holds a process or an untracked file, and all their content is on main (3.6 GB). Then delete their local branches:
      cd ~/Code/projects-2026/dictionary && for w in .claude/worktrees/wf_7a346eac-a42-{1,2,3,4,5,6,7,8,9,10} .claude/worktrees/wf_6372bf64-852-{1,2} .claude/worktrees/wf_dd7e7ea3-b57-{1,2,3} .claude/worktrees/issue-219-backfill-a; do git worktree remove "$w"; done
      cd ~/Code/projects-2026/dictionary && git branch -D a5-final a5-rehearsal a6-delta a6-review claude/219-a-backfill claude/219-a4-recovery claude/219-backfill-acceptance claude/219-demo claude/219-integ
      This session's own worktree (source-listing-ui-ef5442, on claude/219-checkpoint) goes when the session ends. claude/219-on-reader and claude/219-checkpoint are merged and can be deleted with it.
    4. D3: no database was dropped. The checkpoint proposes dispositions for both clusters. Each drop needs your authorization naming the exact database and server.

    Once 1 and 2 are done, #219's finish line holds and the issue can close. I have left it open because the criteria are not yet all true.

  6. holden commented on Sep 28, 2026

    @holden
    MemberAuthor

    Independent closing audit, 28 September 08:55 (+02): A− stands; the issue can close after the three owner steps

    Checked from main 16e35ca in a fresh checkout on a private partition, against this issue's checkpoints and the handoff on #211. Nothing was written to devils_dictionary_v2, no database was dropped, no worktree removed.

    Executed here

    Check Result
    MIX_TEST_PARTITION=_audit219 mix precommit on 16e35ca 19 doctests, 2,540 tests, 0 failures, exit 0; formatter changed nothing
    Router and reader on main /on/:slug; eight explicit family routes; zero /define references left in lib/; Resolver.resolve/2 and decide/5 take mode:; target_config/1 refuses unfollowable URL options and inherits nothing
    P1/P2 regressions on main present in backfill_test.exs: changed unmatched ancestor, contradicting unmatched branch, retired page, other-role page rolled back, kind change mid-batch rolled back, two concurrent runs
    Two servers from main against the retained demo copy stage2r_219final (internal 4231, public 4232), the B2 table over HTTP /on/mars 200 both; /define/mars 404 both; /on/%FF 400 both; /nature/mars and /subjects/mars 200 internal / 404 public; /nature/Mars and /nature/mars/ 301 → canonical internal, 404 public; /works/mars 404; the published Stage 2A rehearsal pages 200 in both modes, including the split page; /entities/:id/:slug 200 both; /words/1831413/mars 404 (an entity id, not a lexeme)
    Page bodies internal /on/mars links /nature/mars and /subjects/mars, marks drafts and fixtures; public /on/mars links no draft address, falls back to /entities/… with "not yet public" and "no public address"; no /define link anywhere
    The demo copy 123 pages, 121 paths, 117 canonical, 291 decisions, 2 backfill runs, 12 published (the Stage 2A rehearsal fixtures carried in c1.dump); Mars the planet (1831413, Q111) at /nature/mars draft; deity fixture 3742817 at /subjects/mars draft; album fixture 3742818 with no address; both fixtures say so in metadata.fixture
    Schema baseline the v2 schema-only copy migrated through 20260927200717 and 20260927220937 (stage2r_219v2m, 26 migrations) differs from the fresh reference only in whitespace inside function bodies (22 diff lines), and carries the merged local_model_configs_shape
    Documents docs/operations/consolidation-checkpoint-2026-09-28.md with a per-database inventory of both clusters (123 databases on 5432, 35 on 5433, each with purpose and a proposed disposition); docs/routing/on-demo/ with the record and 11 screenshots; the evidence directory data/audits/2026-09-28-219/ local and ignored

    Verdict

    A− confirmed for the code and the evidence, for the implementer's own two reasons: the reader has run only on isolated copies, and the recorded P3 limits stand. CP1 and CP2 are met; CP3 is met except for the owner's D4 (quiescence), D5 (the main checkout's local main is 66 commits behind) and D2 (directories). Close this issue once those are done.

    Two notes for D2

    • The twelve local branches the closing comment lists for deletion: six are not merged as refs (a5-final, a5-rehearsal, a6-delta, a6-review, claude/219-demo, claude/219-integ), but each is ahead of main by merge commits only, with zero non-merge commits, so git branch -D loses nothing. Verified.
    • The 16 wf_* and issue-219-backfill-a worktrees hold no process (checked), as stated. Together with the 15 archived on 27 September and rail-sr-only-fix, 32 directories (about 9 GB) await the owner's removal.

    The two open PRs, #218 and #223

    Not part of #219 and not blocking #211. #223 is stacked on #218, so #223's branch is the one designated integration branch the entry checkpoint allows. #218 carries a migration (20260927204902_add_exemplar_composition_items); merging it before the capture would change the handoff's "exactly two pending migrations". Recommendation: leave both open until the move is accepted, then give them their independent final-head audit (the #212 visibility decision is still open per its last comment) and merge #218 → #223 as a recorded post-move step.

  7. holden commented on Sep 28, 2026

    @holden
    MemberAuthor

    D4 is done (see #211): 4007, its tunnel and the 4017 preview are stopped and devils_dictionary_v2 has no clients. D5 and D2 remain, then this issue can close.

  8. holden commented on Sep 28, 2026

    @holden
    MemberAuthor

    D5 is done: the main checkout's local main is 16e35ca, equal to origin/main, clean, with no dev server started against devils_dictionary_v2 (still at 20260927131023, zero connections). With D4 and D5 done the finish line holds; D2's directory removal is the remaining reclaim step and does not block #211.

  9. holden commented on Oct 5, 2026

    @holden
    MemberAuthor

    D2 done; closing (5 October 2026)

    Worktrees. 17 removed today, each checked at the moment of removal: branch or detached head merged into main, no process holding it, nothing dirty beyond the untracked data symlink (the one exception, dictionary-word-page-grouping-2c67e4, had its two-file diff compared byte for byte against the patch kept under data/worktree-reclaim-2026-09-27/ before removal). The 16 workflow worktrees from the 28 September list were already gone. Two more were freed by archiving their idle sessions through the managed archive first ("Exemplar layer build 2", "Dictionary Routes"). git worktree prune run. Internal disk: 13 GiB free, from 7.

    Branches. Merged local branches deleted with git branch -d; the six merge-only scratch branches (a5-*, a6-*, claude/219-demo, claude/219-integ) with -D after confirming zero non-merge commits ahead of main.

    Left in place, on purpose (11 registered): the two ~/.codex worktrees; this audit session's own; four held by pinned sessions (source-listing-ui-ef5442 on the merged claude/219-checkpoint, word-level-tier-shelves-720f8b, dictionary-curated-opening-phase1-f69daa, 212-exemplar-items-b549d7 with #218/#223 open); 172-final-sweep, merged but still the working directory of a live process; and the unmerged 143-spotify-music-6bb8a8, 144-kit-de9e9d and curation-persistence-196 (the superseded consolidation branch), which are the owner's to decide. Unpinning and archiving the three finished pinned sessions would release about 1 GB more.

    Finish line. CP1–CP3 met; D4 and D5 done on 28 September, D2 today. main 16e35ca is accepted and current in the main checkout; the one integration branch is #223 (stacked on #218); the #211 handoff is posted. CP4 is #224. Nothing on devils_dictionary_v2 changed in the meantime: schema 20260927131023, zero connections. #211 may begin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions