Repository navigation
Description
Rewritten 2026-09-24. The first draft measured the gap and asked three questions. This version answers them, adds the mechanism
/define/cowardexposed, and is written to be built: three builds, each with schema, wireframes, contracts and acceptance. The measurements from the first draft are kept below unchanged.
Purpose
Every sense-level shelf — Quotes (live Wikiquote and the public-domain corpus), Artworks, Images — is gated on a sense having a refers_to link to a Wikidata concept and that concept having the page the provider reads. Today that gate passes on about 1 % of English words, and /define/coward shows why the rest is empty: GIFs, music, images, news and texts all match on the word and fill for anything; Quotes matches on the concept and finds nothing. This issue makes the sense-level shelves show on most words without giving up the rule that a match is an identifier, never a name.
Refs #158 (builds 1–6, closed), #174 (the corpus, closed), #164 (creators), #101 (the assessor, which this issue does not need), #74 (the encyclopedia model: entities enter through senses), #69 §5 (the linking ladder).
What is measured
| Count | Share | |
|---|---|---|
| English lexemes | 1,541,669 | |
… with at least one sense that refers_to a Wikidata entity |
15,010 | 1.0 % |
Senses with a current refers_to |
15,986 of 1,989,579 | 0.8 % |
lexeme_entity_candidate rows (word-level, from Wikipedia titles and disambiguation pages) |
76,195 | |
Of the twelve #158 probe words, those with any refers_to |
2 (war, power) | |
Where refers_to comes from (method) |
wordnet_wikidata 13,960 · wordnet_ili 10,236 · wiktionary_qid 2,268 |
|
| What it points at | taxa 18,709 · concepts 6,491 · persons 956 · places 259 · works 41 |
The three shapes of "no Quotes shelf", with a word for each (2026-09-24, dev database, live Wikidata):
| Shape | Word | What the registry has | What Wikiquote has | Fix |
|---|---|---|---|---|
| Wrong hop | coward | noun sense → Q104605901 "cowardly or fearful person", no Wikiquote page; the item says has characteristic (P1552) → Q1401607 cowardice | Cowardice, Q1401607 | Build A: take the step |
| No sense link, a candidate exists | most common nouns | no refers_to; a lexeme_entity_candidate from the lemma's Wikipedia title, some corroborated by a gloss match at 0.85 |
the concept's page | Build B: read the candidate, say so |
| Nothing | situationship | no refers_to, no candidate, no Wikiquote page |
none | honest empty, unchanged |
What exists (read on main @ 0d69346)
- The ladder (
DevilsDictionary.Absorb.Linker, 🔁 MVP-0: absorption-first word graph, walking skeleton (WordNet · Wiktionary · Wikidata · Wikipedia · Bierce, Animals scope) — decisions, schema, pipeline, scorecard #69 §5): six rungs writerefers_to(sense-backed:wiktionary_qid0.95,wordnet_wikidata0.90,wordnet_ili0.85) orlexeme_entity_candidate(word-level:title_match0.70,disambiguation0.40–0.60).corroborate/1raises a title match to 0.85–0.90 when a second independent signal agrees — a taxon name, an existing QID rung, or content words shared between the article's description and a gloss — and records which inmetadata["corroboration"]. So the offline linking pass the first draft proposed already exists; what is missing is that nothing downstream reads its output. - The read (
Discovery.PageEvidence):query/1joins senses →refers_to→ entity → verifiedwikidataidentifier, and nothing else.covers?/1andautomatic_mapping/1on every sense-level provider go through it. A corroborated candidate at 0.90 is invisible to every shelf. - The hop, for one provider: the Met matches a tag QID "equal, or reached in ≤ 2
P31/P279steps" (README, The one rule). Wikiquote and the corpus got the strict form — equal only — with no reason given. - The plaque:
MatchReasonrenders a:queryreason as Search result for "…" and the:quoterow admits[:identity, :attestation]only. A word-level identity match is neither: it is an identifier, reached for the word rather than the sense, and needs its own wording.
Decisions (answered 2026-09-24)
- A fallback on the Quotes shelf: yes, but not a text search — a word-level identity fallback from the corroborated candidates the ladder already writes, labelled as such. No provider searches Wikidata by lemma at read time.
- The same fallback on Artworks and Images: yes, through the same read, because they sit behind the same gate.
- Promotion: gloss corroboration first (the ladder's own rule), the assessor (Discovery relation assessor: lite ML and LLM judging of provider results before persona curation #101) later if the false-positive rate says so.
Build A — the concept hop (Wikiquote provider and the corpus build)
From the sense's item, follow a fixed, short list of properties at most two steps to an item with an enwikiquote sitelink, the way the Met walks P31/P279:
| Property | Reads as | Example |
|---|---|---|
P1552 has characteristic |
the quality a kind of person has | coward → cowardice |
P279 subclass of |
the broader thing | world war → war |
P1269 facet of |
the topic this is an aspect of | |
P31 instance of |
only when the instance is a concept, never for a person or a work |
Stop at the first step that lands on a page; record the path. The mapping's entities entry gains "via": [{"property": "P1552", "qid": "Q1401607"}], the result's match_details["sitelinks"][…] gains "reached": ["P1552"], and the reason on the card reads From Wikiquote's page "Cowardice", the concept a sense of coward has as its characteristic (Q1401607). The corpus build takes the same hop in its selection, so a corpus line filed under cowardice reaches coward's page.
Schema: no rows change shape. discovery_mappings.parameters and discovery_results.match_details are JSON and gain the via/reached keys; priv/quotes/manifests/wikiquote-pd-v1.json gains concept_qids_via per row on its next build (a manifest change is a new version, v2). MatchReason gains the wording; Providers.validate!/0 unchanged. Migration: none.
Cost: one wbgetentities for the sense's items with claims filtered to the four properties (the client already batches), inside the existing sitelinks request stage: zero extra requests when the item has its own page, one when it does not.
-
/define/cowardshows a Quotes shelf from Cowardice, reason naming the hop, no plaque; the corpus's cowardice lines (if any) with it - The hop never lands on a person or a work (the Met rule's own guard), proved by a fixture where
P31points at a human - Two steps at most; a third is refused with a test
- The twelve Quotes research: which sources are reliable, measured on Voltaire and Vonnegut — Wikiquote re-tested through Parsoid, Wiktionary citations already absorbed, the aggregators fail corroboration, a keyless primary-text check (toward a
:quoterow) #158 probe words re-measured: which gain a page by the hop alone -
docs/discovery/README.mdThe one rule table gains the Wikiquote row with its reach
Build B — read the corroborated candidates, and say what they are
PageEvidence learns a second tier. entities/2 returns, first, the sense-backed refers_to items as today; when there are none, the lexeme_entity_candidate items whose current revision has confidence ≥ 0.85 (i.e. corroborated), each marked "level": "word". A provider's covers?/1 is true for either; its mapping records the level; the result's reason carries it; the card and the shelf say it.
The wording, decided here so no provider invents one: a word-level match reads For the word "coward", not a particular sense beneath the reason, and the shelf's byline carries the same sentence once. It is not Search result for… — that phrase stays for :query, which this is not. ContentTypes gains an evidence class :word_identity admitted by :quote, :artwork and :image; conformance asserts it is labelled wherever it appears.
Promotion, so the fallback shrinks over time: a corroborated candidate at ≥ 0.85 whose corroboration is a gloss match against a specific sense is promoted to refers_to on that sense by the ladder's corroborate/1, method corroborated_gloss, confidence as computed — the ladder already knows the sense it matched, it just does not write the sense-backed claim. Promotion is a ladder run, offline, re-runnable; a promoted link makes the fallback unnecessary for that word and the reason drops the sentence.
Schema: assertion_revisions gains rows (method corroborated_gloss, predicate refers_to, subject the sense, metadata["corroboration"] carried from the candidate); predicate_endpoint_rules unchanged (sense → entity exists); discovery_mappings.parameters.entities[…] gains "level"; ContentTypes.evidence gains the atom. Migration: none. The spike measures, before any write, how many of the 76,195 candidates are ≥ 0.85 and how many of those have a sense-specific gloss corroboration.
- Spike posted: candidates ≥ 0.85; of those, sense-specific; probe words covered by B alone and by A + B (Sense-level shelves show on 1 % of words: the concept hop, the word-level tier from corroborated candidates, and promotion — so Quotes, Artworks and Images appear on most pages without a name match #172 (comment): 13,884 ≥ 0.85; the ladder records no sense, 6,230 promotable; probes B alone 4, A + B 5)
-
PageEvidence.entities/2two tiers,levelon every entry, sense-backed always first and never mixed with word-level in one mapping (PR Word-level tier: sense-level shelves read the word's corroborated candidate, labelled; the ladder promotes gloss matches (#172 build B, C) #187,page_evidence_test.exs) - The wording on card and byline, asserted by the multi-source conformance suite for the three rows (PR Word-level tier: sense-level shelves read the word's corroborated candidate, labelled; the ladder promotes gloss matches (#172 build B, C) #187; plus
Conformance.assert_labelled!/3on Met, Commons, Wikiquote) -
corroborate/1promotes sense-specific corroborations torefers_to, re-runnable, counts printed bymix dd.link(PR Word-level tier: sense-level shelves read the word's corroborated candidate, labelled; the ladder promotes gloss matches (#172 build B, C) #187; dev run 6,230 promoted, rerun 0 revisions; since review, the same pass also withdraws a promotion whose evidence lapses) -
/define/cowardafter promotion: the sentence gone, the page unchanged otherwise (coward was already sense-level and never carried the sentence; it is unchanged after promotion, seedocs/discovery/issue-172-build-b-coward-after-promotion-shelf-*.jpg. The promotion evidence is grief and love: word-level before (12 of 12 results"level": "word"each), sense-level after one run under new recipes (0 of 12), indocs/integrations/wikiquote.mdW3/W4 andwikiquote_word_level_live_test.exs)
Build C — the honest empty, made visible
When neither tier yields anything, the page today shows no Quotes shelf at all, which reads as "no quotes exist". Add the shelf's About note for the empty case: No concept this word's senses refer to has a Wikiquote page. One template line, no request, no rows. Optional; do it if build B lands with time to spare.
Wireframes
/define/coward — BEFORE /define/coward — AFTER build A
┌ GIFs ─────────── [GIPHY] ┐ ┌ Quotes ──────────────────────── [Wikiquote] ┐
┌ Music ──────── [Spotify] ┐ │ 👑 Public domain │
┌ Images ──── [Unsplash…] ┐ │ ┌────────────────┐ ┌────────────────┐ │
┌ News ────── [Guardian…] ┐ │ │ "Cowards die │ │ "…the coward │ … │
┌ Texts ────── [PoetryDB] ┐ │ │ many times…" │ │ sneaks to │ │
│ │ Shakespeare ↗ │ │ death…" │ │
(no Quotes shelf; nothing says why) │ │ Julius Caesar │ │ Wilde ↗ · 1898 │ │
│ │ ● Verified │ │ ○ Plausible │ │
│ └────────────────┘ └────────────────┘ │
│ From Wikiquote's page "Cowardice", the │ ← reason names
│ concept a sense of coward has as its │ the hop, no
│ characteristic. │ plaque
└───────────────────────────────────────────────┘
/define/<a word with no sense link> — AFTER build B
┌ Quotes ──────────────────────────────────────────────────── [Wikiquote] ┐
│ For the word "…", not a particular sense. │ ← the byline sentence, once
│ ┌────────────────┐ ┌────────────────┐ │
│ │ "…" │ │ "…" │ │
│ │ … ↗ · 1859 │ │ … │ │
│ │ ○ Plausible │ │ ○ Plausible │ │
│ └────────────────┘ └────────────────┘ │
│ From Wikiquote's page "…", the concept the word "…" names (Q…) — │ ← reason, word-level
│ for the word, not a particular sense. │
└──────────────────────────────────────────────────────────────────────────┘
the read, after both builds
senses ──refers_to──▶ concept ──sitelink──▶ page (tier 1: sense-level, as today)
│
└──P1552/P279/P1269/P31 (≤2)──▶ concept ──sitelink──▶ page (build A: the hop, still tier 1)
lexeme ──lexeme_entity_candidate (≥0.85)──▶ concept ──sitelink──▶ page (build B: tier 2, word-level, labelled)
▲
└── corroborate/1 promotes a sense-specific gloss match to refers_to, and the word leaves tier 2
Contracts
- C1 — the one rule holds. Every match in this issue is a QID reached by a stated relation or a stated ladder rung. No provider searches Wikidata or Wikiquote by lemma at read time; the word-level tier reads what the ladder wrote offline.
- C2 — sense-level first, never mixed. A mapping is built from tier 1 when tier 1 has anything; tier 2 only when it has nothing. A page never shows both levels on one shelf.
- C3 — the hop is data. The property list lives in one module attribute, the path is recorded on the mapping and the result, and the reason prints it. Adding a property is a one-line change with a fixture.
- C4 — the label is not optional. A word-level result that reaches the card without the
:word_identityevidence class fails conformance, the way an unlabelled:querydoes. - C5 — promotion is the ladder's. No provider writes
refers_to.corroborate/1does, offline, andmix dd.linkprints the counts both ways.
Order and prompts
Build A first (one PR, small, fixes coward today), then build B's spike (numbers posted here before code), then build B, then C if cheap. Each PR: gate green, CodeRabbit addressed, owner merges with a merge commit.
Acceptance for this issue
- Build A merged;
/define/cowardshows the Cowardice shelf with the hop named (PR Build A of #172: the concept hop — coward reaches Cowardice #184, 2026-09-24) - Build B spike posted with the candidate counts and the probe-word coverage by A, by B, by both (Sense-level shelves show on 1 % of words: the concept hop, the word-level tier from corroborated candidates, and promotion — so Quotes, Artworks and Images appear on most pages without a name match #172 (comment))
- Build B merged; the twelve probe words re-measured: how many now show a Quotes shelf, and how many of those are word-level (PR Word-level tier: sense-level shelves read the word's corroborated candidate, labelled; the ladder promotes gloss matches (#172 build B, C) #187 merged as b8efc0c, 2026-09-25; 13 probe words: 4 Quotes shelves, 2 word-level after B; 4 and 0 after promotion)
- Promotion run once on the dev database; counts recorded; the README's The one rule names the two tiers and the hop (6,003 animals + 227 emotions; README in PR Word-level tier: sense-level shelves read the word's corroborated candidate, labelled; the ladder promotes gloss matches (#172 build B, C) #187)
- Coverage re-measured after promotion: lexemes with a sense-backed link, before and after (1,849 → 8,053 lexemes, 1,810 → 7,950 pages, measured on a dev DB whose WordNet links had been withdrawn on 2026-09-24; remeasure after PR The ladder's links survive a source re-materialization #188 and the restore)
Activity
[-]Sense-level shelves show on 1 % of words: only 15,010 of 1,541,669 English lexemes have a sense that refers to a Wikidata concept — link more senses offline, a lexeme-level fallback for the Quotes shelf, or both[/-][+]Sense-level shelves show on 1 % of words: the concept hop, the word-level tier from corroborated candidates, and promotion — so Quotes, Artworks and Images appear on most pages without a name match[/+]holden commentedon Sep 24, 2026
Rewritten 2026-09-24 as an implementation issue. The first draft's three questions are answered in the body (a word-level fallback yes, but from the ladder's corroborated candidates and labelled, never a lemma search; the same for Artworks and Images; gloss corroboration before the assessor).
/define/cowardadded a fourth shape the draft missed: its sense does refer to a concept (Q104605901), which has no Wikiquote page but says has characteristic → cowardice (Q1401607), which does — the same two-step reach the Met already has. Three builds with schema, wireframes, contracts and boxes: A the concept hop (small, first), B the two-tier read ofPageEvidenceplus promotion byLinker.corroborate/1(spike first), C the honest empty note. No migration in any of them.holden commentedon Sep 24, 2026
Build A measurement: the hop alone (2026-09-24)
Live Wikidata, senses from the dev registry, walked with
Discovery.ConceptHopas built. The full ledger is indocs/integrations/wikiquote.mdunder Concept hop: ≤39 of 60 live requests, 5 of them for this table.The thirteen probe words
P1552→ Q1401607The hop alone gains one of thirteen. The other ten have no
refers_toat all.A rate: 50 of the registry's 2,165 hop origins
This is a fixed sample: the 50 concept/event QIDs with the lowest
md5(qid).The hop reached:
P279)P279× 2)P1552→P279)P31)Two read as a stretch: fingerprint → Results and Underground Railroad → Confidentiality. Both follow Wikidata's own statements, and the card prints the path.
What the measurement changed
As the issue wrote it,
P31("only when the instance is a concept") walked luthier → profession → Wage, witch doctor → occupation → Labor and aunt → Brotherhood. The rule as built followsP31only when all three hold:P279of its own. Read off a class,P31names a metaclass.P279.coward's item has a
P279, so itsP31is never read. The hop goes byP1552alone.holden commentedon Sep 24, 2026
Build A's five boxes ticked. Everything is in PR #184, which is open and waits on CodeRabbit and the owner's merge.
/define/cowardshows a Quotes shelf from Cowardice, the reason naming the hop, no plaque.test/devils_dictionary_web/live/wikiquote_concept_hop_live_test.exs.docs/discovery/issue-172-build-a-coward-{shelf,reason}-*.jpg.concept_qids_via), and the v2 rebuild is the follow-up below.concept_hop_test.exs, a hop to a human is refused, even with a page, and not walked through, plus the creative-work case.wikiquote_test.exs, a hop to a human is refused, though the human has a page.concept_hop_test.exs, a two-step hop resolves and a third step is refused, and never fetched.docs/discovery/README.md, The one rule, has a Wikiquote row with its reach.Follow-ups:
wikiquote-pd-v2with the hop (mix dd.quotes.corpus.build --reselect --output priv/quotes/manifests/wikiquote-pd-v2.json). This needs awikiquote-pd-v2source row and theManifestkind'ssource, which today names v1.holden commentedon Sep 24, 2026
Audit of PR #184 (build A, the concept hop) — grade A. Ready to merge; not merged, pending the CodeRabbit run the owner is re-triggering.
Independent check: the branch merged locally with current main (after #182, #183, #159), own partition: 16 doctests, 1,974 tests, 0 failures. No conflicts with anything merged since it was opened.
Discovery.ConceptHop: four properties in order with one phrase each, two steps, origins only concepts and events, refused ends and steps by Wikidata's ownP31(human, character, 14 work classes — the registry'sconceptkind is too loose to trust, correctly),P31only from an instance onto a class as the last step — the two live walks that went wrong (luthier → profession → Wage) are in the doc and the guardsfrom,via,reached; the reason renders the hop; a direct sitelink is byte-identical to beforewikiquoterowconcept_qids_viaper row, a pre-hop selection re-runs to the same rows; v2 rebuild left as a follow-up, rightlyOne side effect to know before merging: every existing Wikiquote mapping re-versions once on deploy (the recipe now freezes the rule), which is what un-caches coward's old empty run. Expected and harmless.
The sideways-scroll claim in the PR ("the page scrolls sideways on every word because the source mark's
sr-onlylabel escapes each rail") did not reproduce on the running server: at 375 px no element outside anoverflow-x-autorail extends past the viewport andbody.scrollWidthis 375. It may be a headless-capture artefact (the same one #109 noted, headless Chrome clamping at 500 px). Not fixed here; worth a look by whoever captured it.Build A's five boxes stand ticked. Build B's spike is next.
Merge pull request #184 from razrfly/claude/wikiquote-concept-hop-f9701c
holden commentedon Sep 24, 2026
Build A merged: PR #184 as 0dc8ef8. CodeRabbit's four findings were fixed in 0c0ba87 (a retry allowance per hop step; a folded corpus line names each concept's own page; escaped pipes; the v2 wording), each thread answered and accepted. Gate on the head merged with main: 16 doctests, 1,976 tests, 0 failures (one flake on the first run, green on the rerun). Ticking the first acceptance box; build B's spike is next.
holden commentedon Sep 24, 2026
Build B spike (2026-09-24, before any code)
Dev database
devils_dictionary_v2, read-only. Live Wikidata for the sitelink check only: 23 of 60 requests, ledgered indocs/integrations/wikiquote.mdunder Word-level tier (on the build B branch).The candidates
lexeme_entity_candidaterevisionsgloss_overlap6,613 ·taxon_name6,376 ·qid_agreement895The ladder does not record which sense it matched. The issue says it "already knows the sense it matched, it just does not write the sense-backed claim". It doesn't know.
corroborate_gloss/2is anEXISTSover the lexeme's senses and writes only{"corroboration": "gloss_overlap"}. So promotion has to recompute the match per sense, with the same rule (two shared content words of four letters or more):refers_toof its own (21 excluded)taxon_name(6,376) andqid_agreement(895) are never promoted. A taxon name agrees with the word, and an agreement is already a sense link to that entity. The taxa stay word-level.At 18:41, a WordNet re-materialization's reconcile withdrew all 24,196 of the ladder's WordNet links (
wordnet_wikidata13,960,wordnet_ili10,236) with no longer emitted by its source. The ladder registers the WordNet record as their provenance. That fits the re-materialization #183 asked for. Pages with an active sense link: 14,676 as the ladder wrote them, 1,810 now./define/powerhas lost Q911554. I have not touched it. It's filed as a separate task, and add.linkrerun reinstates the links (there are no reviews to block it). Both states are measured below.Who gains a page
Base = a direct sitelink from a sense's item. A adds the hop. B adds a corroborated candidate's direct sitelink when the page has no sense link. The first number uses sense links as the ladder wrote them; the one in brackets is today's DB.
¹ A noun with a WordNet and a Wiktionary sense,
^[a-z]{3,}$, lowestmd5(lemma).Probe words: grief (Q1026040) and love (Q316) gain a word-level Quotes shelf from B alone. War, power and coward are unchanged. Nepotism and bank are in the culture scope, which has no candidates. The other six are in no scope, so they have nothing.
B is bounded by the ladder's scopes. None of fifty random common nouns has a corroborated candidate, because the ladder has run on animals (25,385 lexemes), emotions (809) and culture (5) and nothing else. Inside those scopes, A + B doubles what A reaches (8 → 16), e.g. prey → Predation, fondness → Affection, jennet → Horses and larva → Animals (the hop from a candidate). Making B's reach wider means running the ladder over another scope. This build doesn't do that.
What I'm building from this
corroborate/1writesrefers_toon the one sense with the most shared words, when it is unique. It skips persons and senses that already have an active link. Methodcorroborated_gloss, 0.85,metadatacarries the candidate'scorroborationplus the sense match (shared_words,candidate_assertion_id). The row has no source-record provenance, so no reconcile can withdraw it the way today's did.holden commentedon Sep 24, 2026
Build B + C measured (dev database, 2026-09-24): PR #187
The promotion run used
Linker.corroborate/1per scope, with the app not started (run rows 196–198). It wrote 6,003 in animals, 227 in emotions and 0 in culture. That is the 6,230 the spike predicted. A second run wrote no revision.The before figure is low because of today's WordNet withdrawal (see the spike). The promoted links carry no source record, so a reconcile cannot withdraw them the same way.
The word-level screenshots are of /define/prey, which is still word-level after promotion (Q170430 → Predation on Quotes, plus 12 Commons depictions). coward has a sense link, so it was never word-level and after promotion it is unchanged. Live requests: 43 of 60 (ledger in
docs/integrations/wikiquote.md).Build C is in the same PR: No concept this word's senses refer to has a Wikiquote page. shows in the About when neither tier names a concept. It makes no request and writes no row.
Boxes ticked except Build B merged, which waits on CodeRabbit and the owner.
Regressions for a late source mapping and a late rival candidate; the…
Merge pull request #187 from razrfly/claude/word-level-tier-shelves-7…
holden commentedon Sep 25, 2026
Build B + C merged: PR #187 as b8efc0c (merge commit, 2026-09-25)
Validation
mix precommiton the head I merged,07c8a17, with a private test partition: 16 doctests, 2,020 tests, 0 failures. That is main's 1,978 plus 42. The branch merged cleanly with main ate779dd9, which had not moved. This repo runs no CI test workflow; the PR checks are CodeRabbit and GitGuardian.Review
CodeRabbit's first run was rate-limited. After one
@coderabbitai reviewit posted two findings, both on promotion. Both were real. Both were fixed in0562398, and CodeRabbit verified each fix against that commit and resolved its thread (it did not run the tests).Two candidates could each pick the same sense as their best, and both were written. Now a sense chosen by more than one candidate is promoted to neither.
A promoted link outlived its evidence. The same pass now withdraws a promoted link (it never deletes one) when:
mix dd.linkreports these asunpromoted.Two more bugs came out of the tests for those fixes:
<>, which let a mapping with no method slip through. It isIS DISTINCT FROMnow.07c8a17adds regression tests for a source mapping and for a rival candidate that each arrive after a promotion. It also corrects the measurement wording. CodeRabbit's run on that tests-and-docs push was rate-limited, and I did not ask again.After the fixes I re-ran promotion on the dev database (run rows 202–204): 0 promoted, 0 withdrawn. Every one of the 6,230 still stands.
What is measured (dev database, 2026-09-24)
Every number here was taken while the WordNet links were withdrawn (see the spike comment), so none of it is the issue's baseline.
"level": "word". Each became a new sense-level recipe on the same page, with 0 of 12 word-level. The ledger is W3/W4 indocs/integrations/wikiquote.md, andwikiquote_word_level_live_test.exsreplays it.Screenshots
docs/discovery/issue-172-build-b-*-2026-09-24.jpg, at 375 and 1024, light and dark. At every size the page's scroll width equals the window's, so nothing scrolls sideways.On prey's Images shelf, the word-level Commons files sit beside stock-photo search results that are labelled as searches. A search is not a level, so no shelf mixes the two levels (C2 holds).
Limitations, stated plainly
dd.linkrerun restores the WordNet links. That restore is not done, and it is not part of this work.wikiquote-pd-v2corpus rebuild with the hop.Every acceptance box is ticked with its evidence, so I'm closing this issue. The WordNet repair and remeasurement live in #188 and its follow-up.
coderabbitai commentedon Sep 25, 2026
🧩 Analysis chain
🏁 Script executed:
Length of output: 10846
@holden, the repository inspection completed after my previous reply. I was wrong to say I could not verify the referenced files.docs/integrations/wikiquote.mdrecords the W3/W4 transition for grief and love and states that the coverage figures were measured while the WordNet links were withdrawn.wikiquote_word_level_live_test.exscovers the word-level label, its removal after promotion, and Build C’s empty note. I inspected those files; I did not run the tests.You are interacting with an AI system.
holden commentedon Sep 27, 2026
Completion audit and one final bounded sweep
The implementation is merged and this issue is already closed: Build A shipped in #184; Builds B and C shipped in #187. The latter's final recorded gate was 2,020 tests, zero failures, with both actionable CodeRabbit findings verified and resolved. The source-backed discovery mechanism, word-level labels, promotion lifecycle, and empty-state explanation are delivered.
The remaining gap is operational evidence and the deferred corpus artifact, not another redesign of the shelves:
This should fit one bounded completion sweep if the corpus inputs and existing request budget are sufficient. If not, report the concrete blocker rather than silently skipping it or expanding the budget.
The original phrase “appear on most pages” is not established by the delivered measurements. The spike found no candidates for fifty sampled common nouns outside the ladder's limited coverage. Expanding candidate coverage and evaluating match quality need explicit, measured follow-up work; they are not something this final verification sweep can promise. Completion should mean verified delivery of Builds A–C, with broader coverage tracked honestly rather than inferred from checked boxes.
This audit checked GitHub records and relevant code/documentation; it did not itself run promotion, rebuild the corpus, query today's database, or rerun tests.
Keep the #172 final sweep's scripts, queries and outputs in the repo
holden commentedon Sep 27, 2026
Final sweep complete (2026-09-27)
PR #215 is open, awaiting the owner's merge (merge commit).
8e695ad;mix precommitgives 19 doctests, 2,270 tests, 0 failures.Promotion after the #188 restore.
Coverage on the restored database. The 2026-09-24 figures (1,849 → 8,053 lexemes, 4 probe shelves) are historical: they were taken while the WordNet links were withdrawn.
These are pages eligible for a sense-level shelf, not pages that retrieve content.
Probe words. 5 of 13 show a Quotes shelf, all sense-level:
P1552→ Cowardice.The other 8 (nepotism, family, solitude, justice, bank, pop art, situationship, narcissism) have no evidence in either tier. power's link is right for its sense, a very wealthy or powerful businessperson, but that one minor sense supplies the whole shelf. That problem is #214's.
Presentation, checked on the connected page:
docs/discovery/issue-172-sweep-*-2026-09-27.jpg, 375 and 1024, light and dark (20 files).A bug found on bunny. A red-linked author was credited as Leonardo Fibonacci?action=edit&redlink=1. It is fixed in #215 with a regression test on the captured Grief page, and bunny's refreshed run (400) credits Leonardo Fibonacci.
wikiquote-pd-v2was not built. There are two blockers:Tracked in #213, with two decisions owed to the owner: which version the Quotes shelf reads, and the build's budget and pace.
"On most pages" is not established. #214 owns coverage and match quality against a defined denominator and a fixed sample.
Where the method lives:
docs/spikes/2026-09-27-issue-172-sweep/holds the scripts, queries, captured outputs, high-water marks, a README with the run order and a reversal procedure, and the Sense-level shelf coverage and match quality: the measurement instrument (denominator, fixed sample, precision reviews) after #172 and #217 #214 starting point./Volumes/LLM Models/dictionary/recovery/2026-09-27-issue-172-sweep/current_links.csv, SHA-2567bab1e87789d14c43dbe68d7f6b8f6c78fc2461a573411b44777521ea448fa39, verified after the move.Requests. This sweep used 16 Wikiquote requests of the 600-an-hour budget. The other providers' runs, 71 requests in all, count against their own budgets.
Merge pull request #215 from razrfly/claude/172-final-sweep