Repository navigation
Description
Purpose
Decide where the dictionary gets quotations — Voltaire on toleration, Vonnegut on war — that we can stand behind, how two or more sources corroborate a quotation so a falsified or misattributed one is caught rather than repeated, and how the result lands in the discovery system as a shelf. This is a research issue in the shape of #100 and #134: every source was called from this machine today, the responses were parsed, and the numbers below are what came back. No adapter is implemented or claimed here. The recommendation and the build order were settled with the owner on 2026-09-23; see Recommendation and Builds.
Refs #65 (the quote layer with provenance scoring — this issue supplies the ingestion half it left as a sketch), #100 (named Wikiquote theme pages as wave 1 and the aggregators as tier 4), #101 (the assessor's "verbatim extraction" rule for Wikiquote items), #105 (exemplars: a quotation is content/quotation, and it already noted that Wiktionary's {"text","ref","type"} examples are absorbed and unrendered), #116 (many sources, one shelf — the rule a Quotes shelf follows), #144 (Provider.Helpers, retention rules, the operator's view), #152 (one badge per card, one mark per shelf).
Research date: 2026-09-22. Facts are labelled live (a request from this session, identifying User-Agent, parsed), documented (read on the source's own pages today) or verify (recalled or second-hand; confirm before relying on it). 100 requests in total, ledger at the bottom. No key-holding endpoint was called with a credential; nothing was written to the database.
The test case
Two authors and the discovery probe set. Voltaire (Q9068, public domain, French with English translations) and Kurt Vonnegut (Q49074, died 2007, in copyright, two early stories on Gutenberg) stand for the two halves of the corroboration problem. The words are #86's ten (nepotism, war, grief, love, power, family, solitude, justice, bank, Pop Art) plus situationship and narcissism.
Five quotations were traced end to end, chosen because each fails in a different way on the open web:
| Quotation | Circulates as | What the sources say today |
|---|---|---|
| "I disapprove of what you say, but I will defend to the death your right to say it" | Voltaire | Wikiquote's Voltaire page files it under Misattributed: Evelyn Beatrice Hall, The Friends of Voltaire (1906), summarising his attitude (live) |
| "No snowflake in an avalanche ever feels responsible" | Voltaire (returned as Voltaire by the Quotable mirror today) | Wikiquote Misattributed: Stanisław Jerzy Lec, More Unkempt Thoughts (1964); Quote Investigator has a 2022 post on it (live) |
| "We must cultivate our garden" | Voltaire, Candide | Gutenberg #19942 full text, line 4121, verbatim; Wikisource Candide chapter pages; an Internet Archive text hit on Candide & Other Stories (live) |
| "Wear sunscreen" (the MIT commencement speech) | Vonnegut | Wikiquote Misattributed: Mary Schmich, Chicago Tribune, 1 June 1997 (live) |
| "So it goes" | Vonnegut, Slaughterhouse-Five | Wikiquote under the Slaughterhouse-Five (1969) heading; not verifiable in any keyless full text (the novel is in copyright; Internet Archive's lending copies are not text-indexed, 0 hits with creator:vonnegut) (live) |
The pattern the rest of the issue rests on: the misattributions are caught by a register, the true attributions are confirmed by a primary text, and the in-copyright true attributions are the gap.
Finding 1: Wikiquote, re-tested — the pages are structured; the search and the wikitext are where the horror is
The owner's experience is that Wikiquote's API output is horrible to extract from. Measured today, that is true of two of the three ways in, and false of the third.
The wikitext route is the bad one. action=parse&prop=wikitext on Voltaire returns 136 kB of nested */** bullets with templates, refs and HTML tags interleaved; nothing short of a wikitext parser gets a clean (quotation, citation) pair out of it (live). That is presumably what earlier attempts used.
The Parsoid HTML route is the good one. GET https://en.wikiquote.org/api/rest_v1/page/html/<title> returns semantic HTML where a quotation is a top-level <li>, its citation is the nested <li> beneath it, and the work and year are the <h3> above it. A 120-line stdlib parser (no lxml, no mwparserfromhell) written for this issue extracts:
| Page | Quotations | Carry a citation | Sit under a dated work heading | Either | Misattributed / Disputed / Attributed rows kept aside | Median length |
|---|---|---|---|---|---|---|
| Kurt Vonnegut | 333 | 173 | 296 | 326 (7 bare) | 3 misattributed | 186 chars |
| Voltaire | 116 | 106 | 42 | 116 | 83 (mostly Attributed: an 1902 anthology cited by page, plus Disputed and Misattributed) | 103 |
| Hillary Clinton (a search hit, as a control) | 322 | 139 | 319 | 322 | — | — |
(live; parser and counts in the probe log.) The Misattributed and Disputed sections are not noise to strip — they are the provenance register #65 wants, already written, with the true author and year in the nested citation.
Theme pages (the concept's enwikiquote sitelink, the route #100 proposed) for the probe words, same parser (live):
| Word | Sitelink | Quotations | Cited | Citation carries a year | Dated before 1900 / 1900–49 / 1950–99 / 2000+ |
|---|---|---|---|---|---|
| war | War | 730 | 701 | 542 | 141 / 199 / 104 / 98 |
| love | Love | 1,610 | 1,587 | 1,063 | 336 / 233 / 340 / 154 |
| justice | Justice | 226 | 212 | 145 | — |
| power | Power | 203 | 196 | 140 | 40 / 20 / 55 / 25 |
| solitude | Solitude | 132 | 131 | 84 | — |
| grief | Grief | 79 | 79 | 46 | 33 / 3 / 1 / 9 |
| family | Family | 46 | 46 | 38 | — |
| narcissism | Narcissism | 18 | 18 | 16 | 0 / 1 / 4 / 11 |
| nepotism | Nepotism | 3 | 3 | 3 | a Psychology Today column, a HuffPost Canada op-ed, and Bierce 1911 |
| bank | Bank → redirects to Banking | — | — | — | the river sense is gone: the term_unverified problem in one redirect |
| situationship | none (404) | 0 | — | — | — |
So: theme pages are 96–100 % cited, and the citation year is right there to band 👑/📚/📱 by era rather than by source. The thinness is per word (nepotism: three; and the three are two journalists and Bierce), and abstract Middle-tier words skew recent (narcissism: 11 of 16 dated 2000+). The quality complaint is real on thin pages and on search, not on the big pages.
The full-text search route is the other bad one. list=search for "nepotism" reports 67 hits; the top five pages, parsed, yield two quotations that actually contain the word (a Harvard University quote and an Elizabeth Olsen quote) — the other hits are navboxes and lead paragraphs (live). Search proposes candidates; it never decides, which is the README's one rule anyway. "situationship": 2 hits, neither a quotation.
Operational facts (live unless marked): 55 Wikimedia requests in this session at 230–900 ms (one SPARQL count 502; no 429 from any Wikimedia host) (contrast the burst 429 in #100 — pace at the existing Wikidata client's 200 ms). Site statistics: 70,925 articles, 94 M words. Licence CC BY-SA 4.0, already accepted for Wiktionary and Commons. The monthly dump enwikiquote-latest-pages-articles.xml.bz2 is 217 MB, last written 2026-09-01 — a corpus-sized object, see build 6. Wikiquote has a written sourcing policy (Wikiquote:Sourcing, documented) and a Misquotations page: 68 entries under Misquoted or misattributed and Commonly misquoted, 91 under Unsourced, unverified, or other best guesses (live).
Wikidata is the crosswalk, not a quotations database: Q198 → War, Q161165 → Nepotism, Q9068 → Voltaire by sitelink (live). Its P1683 "quotation" property is used on Stockholm literary street signs, not famous sayings; its lexeme usage examples (P5831) hold none for war; the count query answered 502 (live).
Finding 2: Wiktionary's own quotations are the dictionary-native source, and we already hold them
Every English Wiktionary entry carries dated, cited quotations under each sense line (quote-book, quote-journal, RQ: templates), the OED's method applied by a second editor community with different sourcing rules from Wikiquote's. Counts in the English section today (live):
| Word | Quotation lines | Templated citations | Plain usage examples (ux) |
|---|---|---|---|
| war | 40 | 23 | 9 |
| love | 19 | 24 | 13 |
| bank | 19 | 18 | 8 |
| power | 17 | 35 | 13 |
| family | 10 | 12 | 16 |
| situationship | 8 | 5 | 0 |
| justice | 5 | 5 | 6 |
| solitude | 5 | 5 | 0 |
| grief | 3 | 3 | 6 |
| nepotism | 2 | 2 | 1 |
| narcissism | 1 | 1 | 0 |
Three things make this the first source to render, not the second:
- It is sense-level by construction. A quotation sits under the definition line it illustrates. Nothing else in this issue can say which sense a quotation belongs to without Discovery relation assessor: lite ML and LLM judging of provider results before persona curation #101's assessor.
- The structure is machine-readable without a wikitext parser. The Parsoid HTML (
/api/rest_v1/page/html/nepotism) carries the template parameters as JSON indata-mw—year,journal,volume,page,passage— and marks the rendered piecescited-sourceande-quotation cited-passage(live). Citations:war (theCitations:namespace) adds a timeline: Mary Shelley, Frankenstein, 1818 (live). - It is already in the database. The Kaikki absorb keeps
text,refandtypeon every sense example (lib/devils_dictionary/absorb/sources/wiktionary.ex,examples/1), and Exemplars: people, works and media as attributed examples of a meaning — nomination, the why, votes, ranking and the reverse view on a person's page #105 recorded that nothing renders them.typedistinguishes a quotation from an invented usage example andrefis the rendered citation (verify the Kaikki field values on a live record before Phase 1).
These are attestations in the README's sense — uses "war" at a 1864 Sherman letter — which is the :text row's evidence class, and they cost no request.
Finding 3: the aggregators, measured — none is a source
| Service | Today | What a record carries | Verdict |
|---|---|---|---|
Quotable (api.quotable.io) |
Connection fails outright, with and without certificate checking (the expired certificate of 2024 was never renewed) (live) | — | Dead |
Quotable community mirror (api.quotable.kurokeita.dev) |
Answers; Swagger UI at the root; author=Voltaire returned 8 (live) |
content, author name, tags, a Wikipedia link for the author — no work, no year, no locator | 2 of the 8 fail corroboration on sight: "No snowflake in an avalanche" (Lec, 1964) and "Meditation is the dissolution of thoughts…" (0 hits anywhere on Wikiquote). Data licence unstated in the upstream README (live) |
| ZenQuotes | Keyless random works; keyword=war silently returns 50 random quotations, 0 containing "war" — keyword filtering is a paid feature and the free endpoint does not say no (live) |
q, a, c, h | Random-only; attribution link required (documented); no source field |
| API Ninjas quotes | Missing API Key (live); category filtering is premium (verify) |
quote, author, category | Keyed, no source field |
| STANDS4 quotes.net | Invalid Token (live); 100/day free (verify) |
quote, author | Keyed, no source field |
| TheySaidSo quotes.rest | 401 (live), paid | — | — |
| Quote Garden | This service has been suspended (live) | — | Dead |
| Forismatic | 522 after 19 s (live) | — | Dead |
| DummyJSON | Mock data; /quotes/search is not even a route (live) |
— | Not a source |
| Goodreads | API removed (404 on the quotes search) (live); robots blocks GPTBot and CCBot | quote, author name | Scrape-only, forbidden |
| BrainyQuote | No API; robots Content-Signal: search=yes, ai-train=no, use=reference, /search_results disallowed (live) |
quote, author name | Scrape-only, forbidden |
The shape of the failure is the same everywhere: quotation + a name, never a work, a year or a page. That missing field is precisely the one corroboration needs, so an aggregator cannot be corroborated — it can only be checked against a register, where the free search mirror misattributed 2 of 8 in the first sample. #65 put these at tier 5. They are not a tier; they are the thing the layer exists to be better than.
Finding 4: corroboration — the primary-text check works keyless for the dead, and is patchy for the living
The corroboration question is does the quotation appear verbatim in the work the citation names. Measured (live):
| Check | Status today | Use |
|---|---|---|
Project Gutenberg full text (/cache/epub/<id>/pg<id>.txt) |
Candide (#19942) in 974 ms; "cultivate our garden" at line 4121, "best of all possible worlds" ×3. Vonnegut: only 2 B R 0 2 B and The Big Trip Up Yonder exist ("Everything was perfectly swell." line 74) | The primary check for public-domain authors; a line number is a locator |
| Gutendex (the community JSON API) | 4 of 4 failed (three 40-second timeouts, then a 301 after 38 s) — as in #100 | Use the 21 MB pg_catalog.csv (last modified 2026-09-20) offline instead |
Wikisource full-text search (list=search) |
"best of all possible worlds": 89 hits, chapter-level (Candide/Chapter 1, 3, 6); Author:Voltaire lists about 20 works | Second primary check, keyless, same client and licence as Wikiquote |
Internet Archive advancedsearch.php |
Does full text through its text: field, keyless: "cultivate our garden" → 8 items including Candide & Other Stories; "so it goes" → 989 items but 0 in mediatype:texts with creator:vonnegut (lending-library copies are not text-indexed). The separate FTS endpoint answered 502 |
Third check for scanned editions; not for in-copyright books |
Google Books volumes?q="…" |
429: Queries per day exhausted for the shared keyless project (consumer project_number:624717413613) on both attempts |
The only keyless-ish check for in-copyright authors (snippets in searchInfo.textSnippet, documented); needs a free API key, 1,000/day (verify). An owner decision |
Open Library search/inside.json |
3 of 3 failed (Unable to query search engine, one 500 after 30 s) | Not reliable today |
| HathiTrust full-text search | 403, Cloudflare challenge page | Not scriptable |
| Quote Investigator | It is a WordPress site: wp-json/wp/v2/posts?search= answers keyless with title, link and date. "snowflake avalanche" → the 2022 post; "insanity" → 5 posts; "voltaire" → 5. robots.txt allows everything. No licence statement on the site → all rights reserved |
The misattribution register. Store title, URL, date only (the titles are literally Quote Origin: ), never the body; ask Garson O'Toole by email first, as #136 did for Urban Dictionary |
| Wikiquote Misattributed / Disputed sections and the Misquotations page | See Finding 1 | The free register, CC BY-SA, already parsed by the same code |
The corroboration model this supports, with #65's score table as the arithmetic:
- Claim: a Wikiquote citation (work, year, locator) or a Wiktionary
ref. Two independent editor communities agreeing is the first corroboration. - Primary text: the quotation is a verbatim substring of the cited work on Gutenberg or Wikisource (public domain), or of an Internet Archive text; the hit's locator is stored. Verified.
- Snippet: for in-copyright works, a Google Books snippet containing the quotation with the cited title (key). Plausible.
- Register: a hit in Wikiquote's Misattributed/Disputed sections, the Misquotations page or a Quote Investigator title → Disputed or Apocryphal, with the true author carried as the provenance note.
- Nothing found → Plausible at best, never Verified; an aggregator-only quotation is Apocryphal by construction.
The five test cases above each resolve under this model; the one that stays Plausible is "So it goes", which is the honest answer without a Google Books key.
What the kit already has, and what is missing
Has (read from main @ 5b491bf): the :text and :news rows in ContentTypes (text-first card, w-44 sm:w-52 column, evidence: [:attestation]); PoetryDB as the worked example of a text provider (two-stage retrieve, the 404-in-200 empty rule, attestation reasons with a locator); Provider.Helpers (word_pattern/1, whole_word?/2, year/1, drop_hidden/1, clamp_label/1); mix dd.provider.new; the Wikidata client with sitelinks and 200 ms pacing; retention as a rule with expires_at real (#144); the one attribution_mark/0 shape (#151); one badge per card and the #page-sources stack (#152); the absorbed Wiktionary examples; content_kind: :quotation in SourceIdentity.Entry with fixtures (#100 noted it; still unfed); the tiers :aristocracy | :middle | :plebs.
Missing:
- A
:quoterow inContentTypes: heading Quotes, text-first,attribution: :credited(the CC BY-SA credit names Wikiquote or Wiktionary; the author line is the card's own),evidence: [:identity, :attestation]— identity when the sense's QID sitelink names the theme page, attestation when the quotation itself contains the word. A quotation card wants the quotation as its body and the attribution as its title line, the reverse of the:textcard; that is one template per Word page height and shelf templates: fold the prose, one template per content type, the world above the fold — measured on /define/love, phased #111's rule, not a branch. - A provenance field on the result (Verified / Plausible / Disputed / Apocryphal from Quote layer with provenance scoring #65) and a badge for it, distinct from the source badge of The word page names its sources 200 times: one badge per card, one mark per shelf, one stack in the rail #152.
- A verifier: a background pass, not a shelf concern, that runs the corroboration model over stored quotations and writes the provenance. It is the first thing in the system that calls a source about an item rather than for candidates, so it needs its own budget row.
- Author identity, measured on the database 2026-09-23. A quotation does not reach its author's entry today, and the reasons are specific. A result earns a clickable
/entities/…link only when its provider proposes an identity ofobject_kind: :entity;SourceIdentity.resolve/1answersobject_kind_not_implementedfor anything else, so acontent/quotationcannot be resolved at all yet. An identity entry may carry a relationship (role,target_identifiers,certainty: :verified | :candidate), but resolution validates it and then never acts on it; PoetryDB and Open Library store the author asauthor_display_name, a string, and the card's author line is plain text. The registry has the parts: 6,674personentities, every one with a Wikidata QID; theauthored_bypredicate withcontent/quotation → entity/personalready an allowed endpoint (43,739authored_byrows exist, for definitions);person_details. Bierce is seeded with a verified QID, work and edition; Voltaire and Vonnegut are not in the registry. Build 2 (Creator identity for every provider: a result credits a person by identifier, the pipeline resolves or mints the person, the card links the entry — built once, before any durable quotation is written (build 2 of #158) #164) is the design, and by the owner's rule it is provider-neutral and lands before any durable quote write. Why Voltaire is missing at all is Why Voltaire is not in the registry: people enter through senses, not through credits — 6,674 persons are word-referents; decide on-demand minting, a people seed, or both #165. - A quotation's identity.
Shelf.dedup/2folds items on identifiers and image URLs, and the README's one rule says text is never identity. For a quotation the text is the object: two sources citing the same line share no identifier today and will never fold, so "two sources agree" cannot be computed. Open question 6.
Recommendation (2026-09-23)
The owner's verdict on Wikiquote was half right, and the wrong half matters. The quotations on its pages are well cited; the routes most people use to reach them, the wikitext endpoint and full-text search, are the horror. Read through the Parsoid HTML endpoint, Wikiquote is the best structured, licensed quotation source on the open internet. It must still never be trusted alone, which is the owner's actual requirement: mix the sources and trust none of them on its own.
The design that follows from the measurements:
- Wiktionary's absorbed quotations open the Quotes shelf. They are in the database with a citation and a type on every sense example, and nothing renders them. Sense-level by construction, zero requests, one PR.
- Wikiquote theme pages are the second source on that shelf, with its own register applied from day one. Resolve the sense's QID to its
enwikiquotesitelink, parse the Parsoid HTML, keep quotation, citation, work and year. The Misattributed and Disputed sections come out of the same parse and are the first corroboration check for free. Never wikitext; never search as evidence. - A verifier as its own background job, not a shelf concern. Public-domain authors: the quotation is a verbatim substring of the cited work on Gutenberg, Wikisource or the Internet Archive. The register: Wikiquote's misattribution sections, its Misquotations page, Quote Investigator's post titles. Every quotation carries a badge — Verified, Plausible, Disputed, Apocryphal — and Verified requires two independent agreements (a Wikiquote citation plus a primary-text hit; or Wiktionary and Wikiquote citing the same work).
- The aggregators are excluded entirely, not kept as a bottom tier. They carry no work, year or page, so they cannot be checked, and the one free search mirror misattributed 2 of 8 Voltaire quotations. Noise without signal.
- The aristocracy corpus last: the monthly dump filtered to public-domain, primary-text-verified quotations, committed as a checksummed file the way the Met highlights are.
Two things the readiness check on 2026-09-23 added: a creator-identity build (build 2, #164) ahead of any durable quote write, because today no result from any provider — artwork, poem, book or quotation — reaches its creator's entry, and the owner wants the fix made once, neutrally, for all of them; and a fingerprint decision (open question 6), because two sources cannot be seen to agree on a line until the line has an identifier.
Three small yeses that would help: a free Google Books API key (the only route to corroborating in-copyright authors like Vonnegut); an email to Quote Investigator asking to index titles and links, as #136 asked Urban Dictionary; banding the shelf by citation year so a 1713 Addison line and a 2023 director's quip do not sit as equals.
The honest limit: a true quotation from a living or recently dead author will often top out at Plausible. That is the correct answer, and showing it plainly is what makes the page better than Instagram. A misattribution shown as misattributed, with the real author named, is itself good content.
Builds, in order (decided 2026-09-23)
Six builds, each its own PR against the checks in docs/discovery/README.md. Builds 1 and 2 touch different code and run in parallel worktrees. The four decisions the order needed were taken on 2026-09-23: the fingerprint ADR, a Google Books key, the Quote Investigator email and era banding by citation year are all yes. Nothing durable for quotations is written before build 2 closes.
Build 1 — Wiktionary quotations on the word page, from data we hold. Ready to start. No provider, no request: read the absorbed sense examples whose type is a quotation and that carry a ref, and render them under the sense they illustrate (they are sense-scoped like #133's relation chips, so the WordNet-card question the owner raised applies here too). This opens the Quotes shelf: the :quote row in ContentTypes, the quotation-first card template, one badge per card (#152). Provenance badge: Plausible for every item (one cited claim, unverified), which is honest and is what Phase 3 upgrades.
- Kaikki
type/refvalues verified on a live record and documented (PR Build 1: Wiktionary's absorbed quotations under the sense they illustrate, on the shared quotation card (#158) #173:typeis"quotation"/"example"/ absent, 616,220 / 96,139 / 45,433 on current revisions, 615,886 / 6 / 17,341 of them with aref;refis the wiki citation verbatim ending in the colon that introduces the passage; the quotations are already onsense_revisions.examples, so the payload is never read) -
:quoterow added: heading Quotes, text-first,attribution: :credited,evidence: [:identity, :attestation](added by build 4, PR Wikiquote theme pages on a Quotes shelf, the register applied from the first run (#158 builds 4a and 4) #169, because build 1 had not started;w-44 sm:w-64, five lines, and aquote_cardbuild 1 can reuse) - Rendered under the sense, quotation body first, citation line second, provenance badge distinct from the source badge (PR Build 1: Wiktionary's absorbed quotations under the sense they illustrate, on the shared quotation card (#158) #173:
Word.quotations/1inside the definition row, onQuotation.card/1extracted from build 4'squote_card; two open, the rest behind N more quotations for this sense; folded on the fingerprint at display time; citation is text, nothing minted, nothing written) - Measured on
/define/war(40 lines) and/define/narcissism(1) at 375 px (PR Build 1: Wiktionary's absorbed quotations under the sense they illustrate, on the shared quotation card (#158) #173: war, nepotism and narcissism at 375 and 1024, light and dark,scrollWidth == innerWidthat both; war renders 24, not 40 — the issue's rule excludes 9 untyped-but-cited lines, Sherman's letter among them, and 6 usage examples; decision owed below)
Build 2 — creator identity for every provider, before anything durable is written. Child issue: #164. Done: #164 closed by PR #166 (2999cd8), residuals in #167 (4c3462f); audited A− then A on #164. The owner's rule (2026-09-23): this is not a quotes fix. A Met artwork by Picasso, a PoetryDB poem, an Open Library book and a Wikiquote line all name a creator who may already be an entry, and today every one of them stores that creator as a display string. So the fix lives in shared code, is neutral about the kind of thing being credited, and lands before build 4, because a quotation persisted without its creator relationship is a row someone has to backfill.
The shape, all in the kit and none in a provider: (1) an identity entry's relationships become real — SourceIdentity.resolve/1 acts on a relationship whose target_identifiers names a Wikidata QID by matching it against the persons held (6,674 today, all with QIDs) or minting the person on demand from Wikidata the way Catalog.people/0 seeds Bierce (label, dates, QID verified against Wikidata, person_details), then writing the assertion the relationship's role names with its certainty as the assertion's confidence; (2) resolve/1 accepts object_kind: :content (today object_kind_not_implemented), because a quotation or a passage is content that has a creator too; (3) the reader's creator line — the artist line on an artwork card, the author line on a text card, the attribution line on a quote card — links to /entities/<id>/<slug> when the relationship resolved and stays text when it did not; (4) a provider's only job is to put the QID it reads (a Wikiquote author sitelink, a Met constituentULAN/Wikidata crosswalk, an Open Library author key crosswalked through P648, PoetryDB's manifest QID) into the relationship — never a name, so no label matching anywhere. The predicate is authored_by for works and content (its endpoints already allow entity/work → entity/person and content/quotation → entity/person); a kind that needs another role (a film's director) adds an endpoint row, not a branch. Several creators cost several rows. The reverse view — a person's page listing what they made or said — is #105's and falls out of the assertions.
Filed as #164; the people question it raised (why Voltaire is not in the registry) is #165.
- Resolution acts on relationships: a Voltaire quotation from the fixture resolves to a minted
personwith Q9068, and a second one reuses it, minting nothing - The same code path links a Met artwork to its artist and a PoetryDB poem to its poet from their existing crosswalks — three providers, one change, proved by the multi-source conformance suite
- Bierce's Nepotism line on the Wikiquote theme page resolves to the seeded Ambrose Bierce (object 1): the QID match, never the label (PR Wikiquote theme pages on a Quotes shelf, the register applied from the first run (#158 builds 4a and 4) #169: the Wikiquote conformance creator case,
:candidate, through the linked page'swikibase_item) - A result with no creator identifier keeps a text creator line and writes no assertion
-
content_kind: :quotationentries resolve; theauthored_byendpoint rules are exercised by tests, not assumed - The build 1 Wiktionary quotations get the same treatment where the
refyields a QID; where it cannot, they stay text - No durable quote write (build 4) merges before Creator identity for every provider: a result credits a person by identifier, the pipeline resolves or mints the person, the card links the entry — built once, before any durable quotation is written (build 2 of #158) #164 closes
Build 3 — the quotation fingerprint (open question 6, decided yes). A small PR: an ADR under docs/adr/ recording the one written exception to association is identity, never text; the quotation_fingerprint namespace and its normalisation (case-folded, punctuation and whitespace collapsed, curly quotes straightened, a leading or trailing ellipsis dropped, never stemmed — as built in ADR 0003: NFC, dashes straightened too, apostrophes removed and other marks a word break, an interior ellipsis kept as a word, % & # kept); the build 1 renderer and every quote provider carrying it in identifiers; and the Shelf.dedup/2 test that folds one line from two sources into one card with two sources named. Before build 4 so the second source folds from its first day.
- ADR merged, naming the exception and its limits (ADR 0003, PR A quotation's words are its identifier: the fingerprint, ADR 0003, and one line from two sources as one subject (#158 build 3) #168, merged 2026-09-23 as 195d9fd)
- Two fixtures with the same line and different punctuation produce one fingerprint; "cultivate our garden" and "cultivate one's garden" produce two (PR A quotation's words are its identifier: the fingerprint, ADR 0003, and one line from two sources as one subject (#158 build 3) #168)
- One card, two sources, for a line held by two providers (as built: neither Wiktionary-as-provider nor Wikiquote exists yet, so two fake quotation providers stand in, driven through
Discovery.execute_run/1;quotation_fingerprint_fold_test.exs, PR A quotation's words are its identifier: the fingerprint, ADR 0003, and one line from two sources as one subject (#158 build 3) #168)
Build 4a — the P0 probe, committed. Fixtures from the 2026-09-22 responses: a populated theme page (Grief, 64 kB), a thin one (Nepotism), a redirect (Bank → Banking), a missing page (Situationship, 404), an author page with Misattributed (Voltaire), and a Wiktionary entry (nepotism). The stdlib parser from the probe log ported to Elixir as the fixture's test.
- Fixtures captured with
mix dd.fixtures.capture, sanitised, under 100 kB each (PR Wikiquote theme pages on a Quotes shelf, the register applied from the first run (#158 builds 4a and 4) #169; Voltaire and Vonnegut gzipped to ~55 kB, since their words alone are ~150 kB and dropping any would change the counts; the 429 is synthetic and says so) - The parser's counts asserted on the fixtures (Voltaire 116 / 83 aside; Nepotism 3) (116 is his Quotes section; the page also holds 62 lines about him)
- A redirect is followed once and a missing page is a successful empty (the
{"status":404}rule, Wikiquote edition) (Bank answers 307 to/w/rest.php/v1/page/Banking/html; Situationship a real 404 with a JSON body)
Build 4 — Wikiquote theme pages as the second source on the shelf, register included. The first durable quote writes; depends on builds 2 and 3. mix dd.provider.new wikiquote --archetype discovery --content-type quote --transport get. Target: the sense's QID → enwikiquote sitelink (identity, sense-level, honest without the plaque). Candidates: the Parsoid HTML's top-level list items. Keep: every item whose nested citation or <h3> gives a work or a year, with the year carried in metadata for the era banding. Register: the Misattributed, Disputed and Unsourced rows are extracted in the same pass and stored as provenance, never as shelf items; a kept quotation whose text matches a register row anywhere on Wikiquote is badged Disputed or Apocryphal with the true author as its note, from this phase on. Retention durable (CC BY-SA). Tier :middle for the source row; era per item. Wikiquote is then the second source on a shelf Wiktionary opened, which is #116's rule applied. Neither wikitext nor list=search is called.
- Sitelink resolution reuses the Wikidata client; a page with no sitelink declines at coverage, spending nothing (as built: a page with no
refers_toQID declines at coverage and spends nothing; a QID with no Wikiquote sitelink costs the one Wikidata request that says so, then caches empty, because the registry does not holdenwikiquotesitelinks) - Quotation, citation, work, year extracted verbatim (any extracted string not a substring of the page text is discarded — Discovery relation assessor: lite ML and LLM judging of provider results before persona curation #101's rule) (asserted across all five captured pages)
- The Voltaire and Vonnegut register rows from the test case come out of the fixtures and the two misattributions badge correctly (never
authored_by,misattributed_towith the register's sentence verbatim,contradictsevidence on another source's credit) - Response body capped (a theme page can be 1.45 MB); redirect, missing and throttled (429 +
Retry-After) responses each a fixture (2 MB cap, refused whole asresponse_too_large) - Multi-source conformance passes with Wiktionary and Wikiquote on one shelf (as built: Wiktionary is not a provider yet, so a Wiktionary fixture provider stands in; Bierce's line folds to one card naming both, via
Shelf.fold/3) - Era banding by citation year on the shelf, decided yes on 2026-09-23 (👑 before 1931 · 📚 to 1999 · 📱 since 2000 · undated)
Build 5 — the verifier and the two-agreement rule. Depends on builds 2 and 3: without a quotation fingerprint there is no way to say two sources cite the same line. A background pass per stored quotation with its own budget rows, listed in the operator's view (#144 Phase 4). Primary text: Gutenberg full text by catalog match (the offline pg_catalog.csv, then one .txt fetch per work, cached), Wikisource search, Internet Archive text: search. Register: Wikiquote sections and Misquotations (already parsed), Quote Investigator titles by permission. Google Books snippets with the key the owner agreed to supply (2026-09-23); Quote Investigator titles once the permission email is answered. Writes the #65 score and the four-way badge; Verified needs two independent agreements, Plausible is one cited claim, Disputed is a register hit with a conflicting author, Apocryphal is a register hit with no cited claim at all.
- The five test cases resolve from fixtures: two Voltaire misattributions Disputed, "cultivate our garden" Verified, "wear sunscreen" Disputed, "So it goes" Plausible without a Google Books key (PR Build 5: the quotation verifier (#158) #171,
verifier_test.exs; as built: Wikiquote now reads "Let us cultivate…", so Verified is the provider's citation plus Candide, Gutenberg #19942, line 4121; "So it goes" is two citations, the second a passage on Vonnegut's page that contains it) - A quotation with no citation and no primary hit is never shown as Verified (PR Build 5: the quotation verifier (#158) #171: Verified needs a primary finding by rule,
badge_test.exs; "So it goes", with two citations and no primary text, stays Plausible) - Budget row per verification source; the operator's view lists them (PR Build 5: the quotation verifier (#158) #171:
gutenberg,wikisource,internet-archive,quote-investigator,google-booksinsource_policiesand as source rows;/ops/discovery→ Quotation verifier, one row per checker with budget used/limit and why it is off,verification_live_test.exs) - Re-verification is a refresh rule (The kit after nine providers: enforce the contract, share the helpers, make retention real, bring the first generation in line, and show the operator the freshness (phased) #144), not a rerun of the shelf (PR Build 5: the quotation verifier (#158) #171:
verification_runs.refresh_after,VerifyWorkerat :07/:37; 30 days after success, a day after failure, the retry time after a deferral. First pass for both authors 13 requests, the second 0 requests and 0 writes)
Build 6 — the aristocracy layer as a corpus. The 217 MB Wikiquote dump, filtered to quotations whose cited work is in the public domain (US: published before 1931 as of 2026, verify the cut-off the project wants) and that build 5 verifies against a primary text, committed as a checksummed selection in the corpus archetype, tier :aristocracy. This is the 👑 quotes shelf the audit found missing, built the way met-highlights-v1 was: a file, not a query.
Schema and wireframes, per build
Read against devils_dictionary_v2 on 2026-09-23. The rule that held for #164 holds here: the #74 model already has the tables, so each build is described by the rows it writes and the one or two indexes it may need. Build 2's schema is in #164 and is not repeated.
Build 1 — Wiktionary quotations rendered
Schema: nothing. The quotations already exist as examples[*] with type: "quotation" and ref inside source_record_revisions.payload of the wiktionary source (617,792 of them, 617,458 with a ref). Each sense the word page shows carries sense_revisions.source_record_revision_id, so the read is: the sense's current revision → its source record revision → that sense's examples in the payload. No materialisation, no new rows, no discovery_results, no content_items (that is build 4, through #164). The :quote row in ContentTypes and the card template are code. The provenance badge is derived (Plausible for every item) and stored nowhere.
One thing to check and record in the PR: sense_revisions.examples is a column of its own. If the absorb already copies the quotation examples into it, the read is one table shorter; if it holds only ux examples, the payload read above stands.
/define/war (375 px)
┌──────────────────────────────────────────────┐
│ war /wɔː/ │
│ │
│ 1. Organized, large-scale conflict … │ ← the sense (Wiktionary card, as today)
│ ┌────────────────────────────────────────┐│
│ │ "War is cruelty, and you cannot refine ││ ← quotation body first
│ │ it; and those who brought war into ││
│ │ our Country deserve all the curses…" ││
│ │ 1864, William T. Sherman, letter to ││ ← the `ref`, verbatim, second
│ │ the mayor of Atlanta ││
│ │ [Wiktionary] ○ Plausible ││ ← source badge (#152) + provenance badge
│ └────────────────────────────────────────┘│
│ ┌────────────────────────────────────────┐│
│ │ "…any warre, they ioyne also vnto our ││
│ │ enemies…" ││
│ │ 1611, King James Version, Exodus 1:10 ││
│ │ [Wiktionary] ○ Plausible ││
│ └────────────────────────────────────────┘│
│ 38 more quotations for this sense ▸ │ ← disclosure; the page does not grow 40 cards
│ │
│ 2. A campaign against something … │
│ (1 quotation ▸) │
└──────────────────────────────────────────────┘
Sense-scoped, so it sits inside the definition row the way #133's relation chips do, not on the culture rail. The author line is text in this build; #164 links it.
Build 3 — the quotation fingerprint
Schema: one namespace value, no migration. external_identifiers gains rows with namespace: "quotation_fingerprint", external_id = the normalised text's SHA-256 (hex, 64 chars; the raw normalised text stays in content_revisions.body), status: "verified". The existing unique index on (namespace, external_id) WHERE status = 'verified' is what makes two sources' copies of one line resolve to one content_items row. Providers put the fingerprint into identifiers beside their own id; build 1's renderer computes it for display-time folding only and writes nothing. The ADR under docs/adr/ records: what is normalised (case, whitespace, punctuation, curly quotes, leading or trailing ellipsis), what is not (stemming, translation), that a translation is a different fingerprint and a different subject linked by excerpt_of or a later translation_of, and that a fingerprint collision across genuinely different lines has never been observed and would show as a wrong fold on a card, which is visible.
before build 3 after build 3
┌────────────────────────┐ ┌────────────────────┐ ┌────────────────────────────────────┐
│ "We must cultivate our │ │ "we must cultivate │ │ "We must cultivate our garden." │
│ garden." │ │ our garden" │ │ Voltaire ↗ · Candide (1759) │
│ Voltaire · Candide │ │ Candide, 1759 │ │ [Wikiquote] [Wiktionary] ● Verified│ ← two badges,
│ [Wikiquote] Plausible │ │ [Wiktionary] Plaus.│ │ │ one card,
└────────────────────────┘ └────────────────────┘ └────────────────────────────────────┘ badge from build 5
two subjects, two cards, no agreement visible one subject: sha256("we must cultivate our garden")
Build 4 — the Wikiquote provider with the register
Schema: rows only, plus one predicate definition.
| Table | Rows |
|---|---|
sources |
one row wikiquote: tier: :middle, kind: :quotations (verify the kind enum; :dictionary and :encyclopedia exist), access: :api, license: "CC BY-SA 4.0", url_template: "https://en.wikiquote.org/wiki/{title}", logo for the mark (#152) |
config :devils_dictionary, :source_policies |
"wikiquote" => [request_budget_limit: …, request_budget_window_seconds: …] with the 200 ms pacing; a body cap of 2 MB (Love is 1.45 MB) |
discovery_runs, discovery_results, source_records, source_record_revisions |
as for every provider: one record per kept quotation, revision payload = the Parsoid <li> text, citation, work, year, section, page revision id; preview_metadata = title (the quotation), author, work, year, source_url, attribution, creators (#164), and era: "aristocracy" or "middle" or "plebs" from the citation year for the banding decided yes |
content_items + content_revisions |
through #164's content path: content_kind: "quotation", body, canonical_url, year, rights_metadata = the CC BY-SA names |
external_identifiers |
wikiquote_item (page title + section + list position, the provider's stable id) and quotation_fingerprint (build 3) |
predicates + predicate_endpoint_rules |
new predicate misattributed_to seeded from core.json: forward "often misattributed to", reverse "Misattributed", endpoints content/quotation → entity/person, content/passage → entity/person (#164 C4) |
assertions + assertion_revisions |
authored_by to the resolved author (#164); misattributed_to for every Misattributed / Disputed row whose named person resolves, method: "wikiquote_register", rationale = the register's own sentence, verbatim |
assertion_evidence |
one contradicts row per register hit on any existing authored_by to that person, locator = the Wikiquote section anchor, source_record_revision_id = the register record |
reconciliation_cases |
unresolved_creator per #164 |
Migration: none. The predicate is a seed row. If the era value proves to be read on every shelf render, a (run_id, (preview_metadata->>'era')) index is the one candidate; not needed for the first hundred words.
Quotes shelf on /define/war, banded by citation year (decided yes)
┌ Quotes ─────────────────────────────────────────────── [Wikiquote] [Wiktionary] ──┐
│ 👑 before the public-domain line │
│ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │
│ │ "My voice is │ │ "They sent forth │ │ "War is cruelty, │ … │
│ │ still for war." │ │ men to battle…" │ │ and you cannot…"│ │
│ │ Addison ↗ · Cato │ │ Aeschylus ↗ │ │ Sherman ↗ · 1864 │ │
│ │ 1713 │ │ Agamemnon │ │ [Wikiquote] │ │
│ │ [Wikiquote] │ │ [Wikiquote] │ │ [Wiktionary] │ ← one line, two │
│ │ ● Verified │ │ ○ Plausible │ │ ● Verified │ sources (build 3) │
│ └──────────────────┘ └──────────────────┘ └──────────────────┘ │
│ 📚 1900 – 1999 │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ "The arms race │ │ … │ │
│ │ is a race…" │ │ │ │
│ │ Amis ↗ · 1987 │ │ │ │
│ │ ○ Plausible │ │ │ │
│ └──────────────────┘ └──────────────────┘ │
│ 📱 2000 – (collapsed, 98 ▸) │
│ │
│ Misattributed ▸ "War is hell" is Sherman's in substance, not these words … │ ← register rows,
└─────────────────────────────────────────────────────────────────────────────────────┘ never as cards
The band is the item's citation year; the source badge is the source row's tier as before. A register row renders as a disclosure line under the shelf, and on the person's page under Misattributed, never as a card with a link that reads as credit.
Build 5 — the verifier
Schema: rows in the evidence table, source rows for the checkers, and one open question.
| Table | Rows |
|---|---|
sources |
one row per checker so its records have provenance and A1 can grade it: gutenberg (access: :download, public domain), wikisource (CC BY-SA), internet-archive (per item), quote-investigator (access: :api, all rights reserved, titles and links only, active: false until the permission email is answered), google-books (access: :api, keyed, snippets never stored — only the volume id and the fact of a match). #165's point stands: these are sources of evidence, and A1 needs a rule for grading a source that is never absorbed |
source_records + source_record_revisions |
one record per fetched work text (Gutenberg pg<id>.txt, cached), per Wikisource search hit, per IA item, per QI post (title, link, date — never the body), per Google Books volume id |
assertion_evidence |
the verifier's whole output: on the quotation's authored_by revision, evidence_role: :supports with locator = "line 4121" / "Candide/Chapter 30" / the IA identifier / the Google Books volume id, source_record_revision_id = the record above, attribution_text = the licence sentence when one is owed; evidence_role: :contradicts for a register hit (QI title match, Wikiquote Misquotations), which review/3 then turns into a rejected revision and a misattributed_to |
assertion_revisions |
confidence raised or lowered by the verifier in a new revision with method: "verifier" and metadata: %{"checks": [...], "verifier_version": …}; #164 C3 guarantees a provider refresh never overwrites it |
content_revisions.metadata |
"provenance": %{"badge": "verified", "score": 0.91, "agreements": 2, "computed_at": …} — the badge the card reads, recomputed when evidence or confidence changes; derived, never authoritative |
source_policies |
one budget row per checker; Google Books 1,000/day against the key |
Decided by the spike (PR #171, docs/integrations/verifier.md): a new table. discovery_request_attempts.run_id was NOT NULL → discovery_runs, so nothing could be budgeted outside a discovery run. verification_runs is per person (subject_object_id, status, times, request_count, error_code, refresh_after, summary), not per checker, because one pass over a person's page and texts serves every line credited to them. The ledger gained verification_run_id beside a now-nullable run_id, exactly one set. The badge lives on content_items.metadata["provenance"], not content_revisions.metadata: revisions are immutable. Checkers built: Gutenberg by Wikidata P50 + P2034 (not pg_catalog.csv), the credited person's own Wikiquote page, Misquotations (agreement with a misattribution only). Measured out: Wikisource, IA text:. Waiting: QI, Google Books.
Open question for build 5's own spike: verification runs need a run record for the operator's view. discovery_runs is shaped for provider pages (mapping, cursor, page); import_runs for absorbs. Reusing either is a stretch; a small verification_runs table (id, checker_source_id, subject_object_id, status, started_at, completed_at, request_count, error_code) may be the honest answer. This is the one new table in the whole plan and it is not decided; the spike decides it with numbers, not this issue.
what the verifier writes for one line
content/quotation "We must cultivate our garden." ──authored_by──▶ Voltaire (Q9068)
│ revision 2 method: verifier confidence 0.91
│ │
│ ├─ evidence supports gutenberg #19942, line 4121
│ ├─ evidence supports wikisource Candide/Chapter 30
│ └─ evidence supports wikiquote (the citation itself)
│
└─ content_revisions.metadata.provenance = {badge: verified, agreements: 2 (+1 primary)}
content/quotation "No snowflake in an avalanche…" ──misattributed_to──▶ Voltaire
│ rationale: Stanisław Jerzy Lec, More Unkempt Thoughts (1964)
│ evidence contradicts quote-investigator 2022/04/26/snowflake
└──authored_by──▶ Stanisław Jerzy Lec (Q…) confidence 0.5 (candidate) → raised when a primary text is found
badge rule Verified ≥ 2 independent agreements, one of them a primary text or a snippet
Plausible 1 cited claim, nothing against it
Disputed a contradicts row exists and a cited claim exists
Apocryphal a contradicts row exists and no cited claim at all
Build 6 — the aristocracy corpus
Schema: a source row and a manifest file; the seeder is the schema. sources gains wikiquote-pd-v1 (tier: :aristocracy, access: :dump, license: "CC BY-SA 4.0", config.dump_url = the monthly enwikiquote-latest-pages-articles.xml.bz2, config.dump_checksum); priv/quotes/manifests/wikiquote-pd-v1.json lists every selected line with its fingerprint, author QID, work, year, and the build 5 evidence row it passed on; the seeder writes content_items, authored_by and the evidence rows with archetype: :corpus on the results, exactly as met-highlights-v1 does for artworks. One naming debt to pay in this build: DevilsDictionary.Artworks.Corpus.Seeder is the corpus seeder for everything and should move to DevilsDictionary.Corpus.Seeder before a second kind of corpus lands under an Artworks namespace.
👑 shelf item note (build 6) the reader's distinction, as today
┌──────────────────────┐
│ "The bravest are │
│ surely those who…" │
│ Thucydides ↗ · c. 431│
│ [Wikiquote] ● Verified│
│ held locally │ ← archetype: :corpus → "held locally", sorts after live results on the same shelf
└──────────────────────┘
How the builds share rows
build 1 sense_revisions ──▶ source_record_revisions.payload.examples (read only)
build 2 #164: relationships ──▶ persons, authored_by (every provider)
build 3 external_identifiers[quotation_fingerprint] ──▶ one content_items row per line
build 4 sources[wikiquote] ──▶ discovery_results ──▶ content_items ──▶ authored_by / misattributed_to
build 5 sources[checkers] ──▶ assertion_evidence ──▶ confidence ──▶ content_items.metadata.provenance (+ verification_runs)
build 6 sources[wikiquote-pd-v1] + manifest ──▶ the same rows, archetype corpus
Every arrow lands on a table that exists today, plus build 5's verification_runs (decided by its spike, PR #171).
Open questions for the owner
Wikiquote itself.Answered 2026-09-23: the rejection is revised. Wikiquote is a claim source read through Parsoid theme pages with its register applied, never the sole source, never via wikitext or search. Build 4 proceeds.- A Google Books API key — yes (2026-09-23). Free, 1,000 queries/day; the only route to corroborating in-copyright authors like Vonnegut. Used in build 5.
- Quote Investigator by permission — yes (2026-09-23). An email to Garson O'Toole asking to index post titles and links (never bodies), the way Urban Dictionary: one browser-fetched 📱 definition per word page, attributed, never stored, permission requested first (from #134) #136 asked Urban Dictionary; the owner sends it, build 5 waits for the answer.
- Era banding — yes (2026-09-23). The Quotes shelf bands by item year (👑 public domain, 📚 to 1999, 📱 after), keeping the source-row tier for the badge. Lands in build 4.
- Author pages (Voltaire's 116) versus theme pages (War's 730): a word page targets a concept, so theme pages are build 4. Author pages become useful when a person page exists (Exemplars: people, works and media as attributed examples of a meaning — nomination, the why, votes, ranking and the reverse view on a person's page #105's reverse view). Recommended: confirm that order.
- A quotation's fingerprint as its identity — yes (2026-09-23), build 3. The one rule says association is identity, never text, and it is right for films and artworks. A quotation has no identifier but its own words. Proposal: a namespace
quotation_fingerprintwhose external id is a normalised form of the text (case-folded, punctuation and whitespace collapsed, curly quotes straightened, a leading or trailing ellipsis dropped; not stemmed, because "cultivate our garden" and "cultivate one's garden" are different claims), carried inidentifiersby every quote provider and by the build 1 renderer.Shelf.dedup/2then folds the same line from Wiktionary and Wikiquote into one card with two sources, and the verifier's "two independent agreements" is a count of distinct sources on one fingerprint plus a primary-text hit. The rule stays intact: the fingerprint is an identifier the provider publishes, not a search. Recorded as an ADR underdocs/adr/so the exception is written down.
Probe log
2026-09-22, 101 requests from one development machine, identifying User-Agent, public endpoints only, nothing stored. Status codes (the first 100; the 101st was a Wikidata entity search confirming Q49074, 200): 75 × 200, 5 × connection failure (Quotable ×2, Gutendex ×3), 4 × 404, 3 × 429 (Google Books ×2, one retry), 2 × 502 (Wikidata SPARQL count, Internet Archive FTS), 2 × 400, 2 × 307, 2 × 301, and one each of 522 (Forismatic), 503 (Quote Garden), 500 (Open Library), 403 (HathiTrust), 401 (TheySaidSo). Keys used: none. Largest responses: Wikiquote Love 1.45 MB, War 729 kB, Hillary Clinton 502 kB — a theme page can be large, and Phase 2's transport must cap the body.
Wikiquote parser used for every count (stdlib html.parser, Parsoid HTML): a top-level <li> under a section heading is a quotation; a nested <li> is its citation; the <h2>/<h3> chain is its section; <sup class="reference"> and <style> are skipped; items under External links, See also and References are dropped; items shorter than 15 characters are dropped. "Cited" means at least one nested citation; "dated" means a four-digit year 1500–2029 in the citation. The era buckets take the earliest year in the citation.
Activity
holden commentedon Sep 23, 2026
Body updated 2026-09-23: added a Recommendation section (mix the sources, trust none alone; Wiktionary first, Wikiquote via Parsoid with its register from day one, a verifier with a two-agreement rule for Verified, aggregators excluded, the corpus last) and rewrote Phases to that order. Open question 1 is answered: the Wikiquote rejection is revised. Questions 2–5 now carry a recommendation each and still wait on the owner.
holden commentedon Sep 23, 2026
Body updated 2026-09-23 after a readiness check against the live database (6,674 persons with QIDs; 617,792 Wiktionary quotations held, 617,458 with a citation;
authored_byalready allowscontent/quotation → entity/person;SourceIdentity.resolve/1refuses content objects and ignores relationships; Voltaire and Vonnegut are not in the registry — the earlier line saying Voltaire was is corrected).Added: Phase 1b — creator identity for every provider, neutral about what is being credited (a Picasso artwork, a poem, a book, a quotation), built once in shared code and landing before any durable quote write, per the owner; large enough to be its own child issue. Open question 6 — a normalised quotation fingerprint as an identifier namespace, the one written exception to association is identity, never text, recommended yes with an ADR. Phase 3 now depends on both.
holden commentedon Sep 23, 2026
Decided 2026-09-23: the phases are renumbered into six builds in execution order — 1 Wiktionary render (ready), 2 creator identity for every provider (#164, ready, parallel with 1), 3 the quotation fingerprint ADR, 4 the Wikiquote provider with register and era banding (first durable writes, after 2 and 3), 5 the verifier, 6 the corpus. Open questions 2, 3, 4 and 6 are all yes. Why Voltaire is not in the registry is its own issue, #165.
holden commentedon Sep 23, 2026
Added Schema and wireframes, per build (builds 1, 3, 4, 5, 6; build 2's are in #164), read against the live database: build 1 writes nothing (reads the sense's source record revision payload); build 3 is one
external_identifiersnamespace whose existing verified-unique index does the folding; build 4 is rows plus one seeded predicatemisattributed_to; build 5 writes its whole output asassertion_evidencerows and verifier revisions, with the run record as the one undecided table in the plan; build 6 is a source row and a manifest, plus moving the corpus seeder out of the Artworks namespace. Wireframes: the sense-scoped quotation on /define/war at 375 px, the fold before/after, the era-banded shelf with the register as a disclosure line, what the verifier writes for a true and a misattributed line, the corpus note. #164 was separately revised after a review (Contracts C1–C7).holden commentedon Sep 23, 2026
#164 (build 2, creator identity) merged in #166, so build 4 is no longer gated on it. Outside
entry_path/2and the creator line, the onlyculture.exchange is an Evidence link beside the link-out for content objects.holden commentedon Sep 23, 2026
Build 3, the quotation fingerprint: PR #168 (open; it's the owner's to merge). CodeRabbit reviewed it once, after its rate limit cleared; both findings are fixed and answered on their threads.
mix precommit: 16 doctests, 1,775 tests, 0 failures.Boxes
ADR merged. ADR 0003 (
docs/adr/0003-quotation-fingerprint.md) is written and in A quotation's words are its identifier: the fingerprint, ADR 0003, and one line from two sources as one subject (#158 build 3) #168. I left this box unticked until the owner merges.Same line, different punctuation, one fingerprint; "cultivate our garden" and "cultivate one's garden" two.
fingerprint_test.exscovers the Quotes research: which sources are reliable, measured on Voltaire and Vonnegut — Wikiquote re-tested through Parsoid, Wiktionary citations already absorbed, the aggregators fail corroboration, a keyless primary-text check (toward a:quoterow) #158 line punctuated four ways (one fingerprint) and our/one's, garden/gardens and a French translation (all distinct). There are 16 doctests, one per rule, and one pinned SHA-256.One card, two sources.
quotation_fingerprint_fold_test.exsruns two providers throughDiscovery.execute_run/1, with the line punctuated differently by each. The results:content_itemsrow by fingerprint and writes no item and no revision.authored_byto Voltaire is a second assertion on the same subject.EntityPageshows one row with two badges.Shelf.dedup/2gives one item, and the newShelf.fold/3names both sources.Culture.section, given both states, renders one card.It also proves that a new wording is an alternate revision, and that a correction onto another item's wording opens the existing
external_identifier_conflictcase rather than folding.Build 1 hook. Build 1 has not merged. Its renderer should call
DevilsDictionary.Quotations.Fingerprint.fingerprint/1on each line to fold at display time and write nothing; the module doc and ADR 0003 say the same. None of build 1 was built.Files touched outside the brief
test/support/fake_quote_provider_base.ex(new): the fake quote provider moved into a__using__macro so two providers can share one shape.FakeQuoteDiscoveryProvideris now just its options.test/support/fake_quote_mirror_discovery_provider.ex(new): the second source the fold is proved with.lib/devils_dictionary/discovery/shelf.ex:fold/3. The brief asks for "one card with two sources", anddedup/2discards the second source, so nothing could name it. Drawing those names on the card is left to the card work.What the code taught that the issue had wrong
don't=dont), every other mark is a word break, dashes are straightened first, and%,&and#are kept because they read as words.content_kind: :quotation; definitions and articles are still compared byte for byte. This came from CodeRabbit's review.Merge pull request #168 from razrfly/claude/quotation-fingerprint
holden commentedon Sep 23, 2026
Audit of PR #168 (build 3, the fingerprint) — grade A
Independent check on a fresh worktree of
main@195d9fd, own test partition:mix precommitgreen, 16 doctests, 1,775 tests, 0 failures, matching the builder's number. Every hunk read against build 3's section and open question 6.external_identifier_conflict, not a fold; rules frozen by version), consequences for builds 1, 4 and 5Quotations.Fingerprint% & #kept, whitespace; a doctest per rule, one pinned SHA-256,nilfor a line with no words;identifier/1carriesnormalisation_version: 1add_alternate_text/2compares fingerprints for:quotationonly, bytes for everything else; no migration, the existing verified-unique index does the foldDiscovery.execute_run/1: onecontent_itemsrow, twoauthored_byon it, one person-page row with two badges, one card; a corrected punctuation is the same text, a new wording an alternate, a wording that is another item's is a conflict case2e7dd02with a doctest and a test each, answered on the threadsResiduals, none blocking, all for build 4 to know:
Shelf.fold/3is defined and tested but nothing inlib/calls it yet: no card names two sources until a quote card exists. Build 4 (or build 1, whichever lands first) is where it gets used.:quotecontent-type row still does not exist (build 1 has not started), so the fixture providers declarecontent_types: [:text]. Build 4 must add the row and the quotation-first card if build 1 has not merged by then, and say so.:passagecontent is compared byte for byte. The ADR scopes the exception to quotations; a passage provider (Wikisource, later) would need the ADR extended, not the code quietly widened.add_alternate_text/2normalises every held revision body in Elixir on each match. Fine at "few per item"; worth a look if a popular line ever collects dozens of variants.Ticked build 3's remaining box in the body (ADR merged, 195d9fd). Build 3 is complete.
Note: PR #159 (the #156 Phase 0 sketches) is still open; only #167 and #168 merged.
holden commentedon Sep 23, 2026
Builds 4a and 4: PR #169 (open; it's the owner's to merge). CodeRabbit reviewed it; all five findings were valid and are fixed in
fdb8e54, with a reply on each thread.mix precommit: 16 doctests, 1,825 tests, 0 failures (1,775 before). I ticked all nine build 4a and build 4 boxes above, plus two earlier ones this PR proves: build 1's:quoterow (added here) and build 2's Bierce-on-Nepotism box.Evidence
mix dd.fixtures.capture --source wikiquote: 7 live requests; the task never starts the app.wikiquote_parser_test.exsasserts Voltaire 116 + 83 (15 misattributed / 5 disputed / 63 attributed), Nepotism 3 and Grief 79, and that every extracted string is a substring of the page.wikiquote_conformance_test.exsruns the shared suite over those pages, including Creator identity for every provider: a result credits a person by identifier, the pipeline resolves or mints the person, the card links the entry — built once, before any durable quotation is written (build 2 of #158) #164's creator case (Bierce, Q191050,:candidate).wikiquote_test.exs(16):misattributed_towith the verbatim sentence, neverauthored_by, never a card, and under Misattributed on the person page.contradictsevidence on a Wiktionary credit for the same line; a disputed badge with its note.Shelf.fold/3's first caller.docs/integrations/wikiquote.md./define/warand/define/nepotismat 375 and 1024, light and dark, committed underdocs/discovery/issue-158-build4-*and shown in the PR. They are static renders, with no second server on the shared database.Files touched outside the brief
discovery/transport.ex,discovery/provider.ex:body: :html, the optionalabsent_status?/1, andresponse_too_large. Without them the transport can't read a page or treat a 404 as empty.discovery/match_reason.ex: asitelinkreason kind. The wiki's name travels as data, so shared code names no provider.absorb/clients/wikidata.ex:sitelink_params/2andsitelink_titles/2, so the provider's budgeted transport makes the concept-sitelink request.source_identity/entry.ex,creators.ex: a relationship'srationalereaches its assertion, through revise and reinstate. A register relationship attachescontradictsevidence.mix/tasks/dd.fixtures.capture.ex: thewikiquotesource. The task now starts the app only for absorb sources, and!opts[:force]replaces anotthat raised on nil.test/support/discovery/conformance.ex,fixture.ex:creator_case/1may declarecertainty:.test/support/fake_quote_provider_base.ex,fake_wiktionary_quote_provider.ex: acontent_typesoption, and Wiktionary's stand-in for the multi-source case.content_types_test.exs::quoteclamps at five lines, and the test says why.docs/discovery/source-marks.mdandpriv/static/images/sources/wikiquote.png: the Commons mark, as The word page names its sources 200 times: one badge per card, one mark per shelf, one stack in the rail #152 did the others.What the code taught that the issue got wrong or left open
enwikiquotesitelinks, so a QID without a Wikiquote page costs one Wikidata request and then caches empty.refers_toQID in the dev registry today. The other ten decline before any request. The live run probed them through their Wikipedia articles' QIDs. Power refers to business magnate, and the shelf follows it faithfully./w/rest.php/v1/page/Banking/html), notrest_v1.prop=pageprops&ppprop=wikibase_item&redirects=1gets the same item through the same sitelink and follows redirects. That recovered four credits live.sources.kindhas no:quotations, so the row is:encyclopedia.misattributed_toseed. I didn't apply them. Wikiquote needs both before it runs there.Merge pull request #169 from razrfly/claude/wikiquote-4a
holden commentedon Sep 24, 2026
Audit of PR #169 (builds 4a and 4, the Wikiquote provider) — grade A−
Independent check on a fresh worktree of
main@a76866c, own test partition:mix precommitgreen, 16 doctests, 1,825 tests, 0 failures, matching the builder's number. Every hunk read against builds 4a and 4, #164 C4/C6, ADR 0003 and the README's provider contract. CodeRabbit reviewed it before merge; all five findings were real and are fixed infdb8e54with a test each (the deferral 4-tuple, the alias-page credit through Wikiquote's own page properties, the rationale on revise and reinstate, the whole-parse fixture guard, thenot opts[:force]crash).Parseris pure, Floki over Parsoid HTML, the six rules from the probe log each a test; "everything extracted is a substring of the page" is a test; Voltaire 116 + 83, Nepotism 3, Grief 79 assertedenwikiquotesitelink throughwbgetentities, identity evidence with a new:sitelinkreason kind; Parsoid HTML only, never wikitext or search; one redirect followed; 404 is an honest empty via the newabsent_status?/1callback; 2 MB cap refuses whole; 429 defers withRetry-After; 200 ms pacing and a 600/hour budget rowcontentmap, CC BY-SA rights, locator and revision id carried; era from the citation year banded 👑/📚/📱 on the shelf as decidedpageprops(redirects followed), never by name;:verifiedunder an author page's own cited work,:candidateon theme pages. See residual 1misattributed_towith the register's sentence verbatim as rationale,register: truesoauthored_byis impossible,contradictsevidence on any existing credit; a kept line matching a register row is badged Disputed or Apocryphal with the noteShelf.fold/3gets its first caller and two sources fold to one card naming both; screenshots at 375 and 1024, light and dark, committed as files this timedocs/integrations/wikiquote.mdwith the posture, a 72-request ledger and the per-word live run; README and adding-a-provider updated; nine "what the code taught the issue" notes, all correct on inspectionResiduals, none blocking:
authored_byendpoint rule, orP31says it is not a creator), but each such line now opens anunresolved_creatorcase after Three residuals from the #164 audit: a version bump is not a new credit, a refused endpoint opens a case, no Wikidata spend for a run that will not publish #167, so the operator's queue will fill with works. Worth a rule: prefer a linked page whose Wikiquote item is a human, which the samepagepropscall could tell us with one more property.:quoterow) #158 matches the shelf.sources.kindis:encyclopediafor want of a:quotationsvalue. A one-line enum addition when the next quote source lands.body: :html,absent_status?/1,response_too_large), a:sitelinkmatch reason,sitelink_params/2andsitelink_titles/2on the Wikidata client, rationale plumbing in creators and entry, the fixtures task. All listed in the report with a sentence each, which is what Creator identity for every provider: credit by identifier, mint by QID, link from the assertion (#164) #166's report failed to do.The finding that matters is not in the code. Only 2 of the 12 probe words have a sense that refers to a Wikidata concept in the development registry, so the shelf built here declines at coverage on the other ten and shows nothing. Measured today across the whole registry: 15,010 of 1,541,669 English lexemes carry a
refers_toon any sense (about 1 %), 15,986 senses of 1,989,579, and the links come from WordNet's Wikidata mapping, WordNet's ILI and Wiktionary QIDs, pointing mostly at taxa (18,709) and concepts (6,491). Every sense-level provider (the Met, Commons, Wikiquote) is gated on the same 1 %. Filed as #172.Live state after this audit: the pending migration and the predicate seed are now applied to
devils_dictionary_v2(index present,misattributed_toseeded with its endpoint rules). The main checkout's server on port 4007, started 2026-09-23 13:29, answers 500 on every page becauseconfig/config.exschanged under it (builds 2–4 added config) and Phoenix's reloader refuses to compile until it is restarted. Restart it and open/define/war: that page's sense refers to Q198, so it is one of the two words that will show a Quotes shelf.Name the untyped-but-cited lines beside the quotation rule, because S…
holden commentedon Sep 24, 2026
Build 1: PR #173 (open; the owner's to merge).
mix precommiton its own partition: 16 doctests, 1,845 tests, 0 failures (1,825 before; 20 new). Ticked build 1's three remaining boxes above with the evidence inline.Evidence
sense_revisions.examplesalready carries every Wiktionary example withtext,refandtype— the absorb copies them onto the revision — so the read is the senses query the page already runs plus one column, never the payload.typetakes three values on current revisions:"quotation"616,220 (615,886 with aref),"example"96,139 (6 with aref), absent 45,433 (17,341 with aref). Arefis the wiki citation verbatim, ending in the colon that introduces the passage (…→OCLC, page 52:); the page prints it without that one character and keepsrefwhole.Word.quotations/1, inside the definition row after the gloss and before Word page grouping: one row per source, one related-words block, the origin on screen (follow-up to #131) #133's chips), onDevilsDictionaryWeb.Quotation.card/1— build 4'squote_cardextracted, not copied, with the caption and footer as slots;Culture.quote_cardnow wraps it. Quotation first, citation second as text (no link in the caption, asserted; nothing minted from aref, no names matched), one Wiktionary badge per card and a separate Plausible badge on every one. Two open per sense, the rest behind N more quotations for this sense (WordPage.quotation_cap/0); folded byFingerprint.fingerprint/1at display time. Nothing written.scrollWidth == innerWidtheverywhere; war 24 cards, nepotism 2, narcissism 1. Thirteen PNGs underdocs/discovery/issue-158-build1-*, shown in the PR, including war at 375 with the first sense's disclosure open.word_page_quotations_test.exs(7, the data contract),word_quotations_test.exs(6, the page),quotation_test.exs(7, the shared card).Files touched outside the brief
lib/devils_dictionary_web/components/quotation.ex(new): the shared quotation card, so build 1 reuses build 4's card without a branch.lib/devils_dictionary_web/components/culture.ex:quote_cardwraps the shared card; the badge ids on a rail card becameculture-quote-<id>-source-<slug>andculture-quote-<id>-provenance(no test targeted the old ones; the words keepculture-entry-title-<id>).test/support/word_fixtures.ex:sense!/4takesexamples:.docs/discovery/README.md: the:quoterow no longer lists Wiktionary's absorbed quotations as a future shelf source; they render under the sense.What the code taught that the issue got wrong or left open
examplescolumn holds the quotations; the payload is never read."quotation"+ref) renders the 24; the wireframe's "38 more" is "8 more" on the first sense.{"text": …, "ref": "1864 Sept. 12, William Tecumseh Sherman, letter to the mayor of Atlanta & al."}, notype), as is his 1879 "war is hell". They read as quotations Kaikki did not label. I followed the rule as written and did not widen it. Decision owed: admit an untyped example that carries aref(17,341 across the dump). It is one clause inWordPage.quotation?/1and the tests already cover the case.refyields a QID" stays open by design: build 1 keeps the citation as text, and arefyields no QID without matching names, which Creator identity for every provider: a result credits a person by identifier, the pipeline resolves or mints the person, the card links the entry — built once, before any durable quotation is written (build 2 of #158) #164 forbids.A sample sense carries an empty quotations map, so the sense contract…
holden commentedon Sep 24, 2026
Build 5 — the verifier: PR #171 (open, CodeRabbit addressed; not merged)
Boxes ticked in the issue body, with evidence:
test/devils_dictionary/quotations/verifier_test.exs):badge_test.exs), and "So it goes", with two citations and no primary text, stays Plausible.gutenberg,wikisource,internet-archive,quote-investigatorandgoogle-bookseach have a source row and asource_policiesbudget./ops/discovery→ Quotation verifier lists them with budget used/limit and why each off one is off.verification_runs.refresh_after, andVerifyWorkeron cron at:07/:37. The first pass for both authors costs 13 requests; the second, after the clock runs out, makes 0 requests and 0 writes.mix precommit: 16 doctests, 1,845 tests, 0 failures.CodeRabbit. Five findings on the first review, all real, all fixed in 9cba5a9 and confirmed resolved:
activeanddiscovery_retry_afternow gate the verifier, and a verifier 429 backs off the discovery provider too.One test finding on the re-review, fixed in b801d82.
Files touched outside the brief
discovery/request_attempt.ex: the ledger row can belong to a verification run (verification_run_id, andrun_idno longer required).discovery/budget.ex:claim_shared/4takes{:verification, id}; the newdefer_source/3sets a source's backoff by id, anddefer_provider/3now delegates to it.discovery/policy.ex: the five checker slugs are shared budgets.sources/catalog.ex: the checker source rows (Quotations.Checkers.source_catalog/0) join the catalog.discovery.exanddiscovery/result.ex:display_itemsreads the item'sprovenanceinto a virtual field, so the card sees the verifier's badge.components/culture.ex: the quote card prefers the verifier's badge, adds a Verified style, and gives the tooltip an honest count.discovery/status.exandlive/discovery_live.ex: the operator's Quotation verifier section.config/config.exsandconfig/test.exs: the budgets, the cron entry,:verification, and:verification_req_options.20260924005117_add_verification_runs: the one new table, plus the ledger column and its check.What the code taught that the issue got wrong
P50+P2034, notpg_catalog.csv. A catalog match would be a match on the author's name; the identifier route is one SPARQL call. Voltaire has only 4 works linked this way, Vonnegut 2.text:were measured out (spike: 0 of 3 Wikisource hits attributable; IA items carry no author identifier). QI and Google Books have inactive rows, waiting on the permission email and the key.:contradictsonauthored_by. It names no one by identifier and usually names the true author, so as built it only agrees with amisattributed_to. The author's own page register still contradicts, because that page is reached by the person's QID.content_revisions.metadata. Revisions are immutable; a review cites them. It lives oncontent_items.metadata["provenance"], and it is the item's: computed from every claim on the line, so two people's passes agree.verification_runswas decided becausediscovery_request_attempts.run_idwasNOT NULL→discovery_runs.:quoterow) #158's rule. The badge follows the rule, and the score is recorded beside it. Quote layer with provenance scoring #65 is also internally inconsistent: QI is +0.40 in one table and +0.25 in the other.Follow-ups and environment
Budget.prune_attemptsjoinsdiscovery_runs, so verification ledger rows are never pruned, and nothing deletes oldverification_runsyet.devils_dictionary_v2) still lacks Creator identity for every provider: credit by identifier, mint by QID, link from the assertion (#164) #166's migration and themisattributed_toseed. I did not apply them. It will also need this PR's migration before the cron runs there.holden commentedon Sep 24, 2026
Audit of PR #173 (build 1, the Wiktionary quotations under each sense) — grade A
Independent check on the branch
claude/158-build1@1f49ce3, own test partition:mix precommitgreen, 16 doctests, 1,845 tests, 0 failures, matching the builder's number. Every hunk read against build 1's section and wireframe; the war/375 screenshot with the disclosure open inspected.sense_revisions.examplesalready carriestext/ref/type;WordPage.senses/1selects one more column and reads no payload — one table shorter than the issue's schema note, and the issue is corrected. Verified live:typeis"quotation"(616,220),"example"(96,139) or absent (45,433)quotation?/1is the one place that decides: typed"quotation"with a non-emptyref. Folded on the fingerprint at display time, first as filed kept, capped at two with the rest behind a<details>that counts them; nothing written anywhereCulture.quote_cardcould not take an absorbed example without a branch, so the card was extracted toDevilsDictionaryWeb.Quotation.card/1with caption and footer as slots and the shelf now wraps it — one place a quotation's look is decided, which is what the brief asked for if reuse needed a branchscrollWidth == innerWidthResiduals, none blocking:
/define/warrenders 24 quotations, not the 40 the wireframe counted, because 9 cited lines carry notypein Kaikki's data — Sherman's 1864 letter to Atlanta, the wireframe's own lead, is one of them. Admitting an untyped example that has aref(17,341 dump-wide) is one clause inquotation?/1. I would admit them: a citation is the evidence, the type is Kaikki's bookkeeping.→OCLC,[…],→ISBNappear in the caption (visible in the screenshot: "London: […] Robert Barker, […], →OCLC, Exodus 1:10"). Verbatim was the rule; a follow-up could strip the arrow tokens, which are Wiktionary's own link furniture and not the citation.culture-quote-<id>-source-<slug>/-provenance); nothing tested the old ones, so nothing broke, but Wikiquote theme pages on a Quotes shelf, the register applied from the first run (#158 builds 4a and 4) #169's report named the old ids.Build 1 is complete. With it merged, #158's builds 1 to 4 are done; build 5 is PR #171 and build 6 is unstarted.
Merge pull request #173 from razrfly/claude/158-build1
Build 5: the quotation verifier — checks by identifier, the two-agree…
The Verified badge and its honest title on the one quotation card, af…
Merge pull request #171 from razrfly/claude/verifier-build5
holden commentedon Sep 24, 2026
Build 5 merged: #171 →
801fd15, rebased onto #173 first.Quotation.card/1, so Verified (emerald), theagreementsattr andagreements_title/1now live there.Culture.quote_cardpasses the verifier's badge through, andquotation_test.exscovers the title wording.mix precommitgave 16 doctests, 1,866 tests, 0 failures.holden commentedon Sep 24, 2026
Audit of PR #171 (build 5, the verifier) — grade A−
Independent check on
main@801fd15, own test partition:mix precommitgreen, 16 doctests, 1,866 tests, 0 failures. CodeRabbit did review this one (six findings: batch isolation, Misquotations scoped to the person it names, the item badge from every claim on the item, inactive/backed-off sources before spending, the tooltip, a vacuous test); every thread is answered "fixed in 9cba5a9/b801d82" and each fix has a test I found.P50+P2034on the credited author's QID; the author's own Wikiquote page through its sitelink; Misquotations as a register only, never against a credit. Wikisource and Internet Archive not built, by measurement: neither ties a text to an author by an identifier, and counting a hit would have meant matching a name. The right call, and the issue text should say so (it does not yet)verification_runsplusdiscovery_request_attempts.verification_run_idbeside a nullablerun_idwith a one-of check, because attempts required a discovery run and a verification has no mapping. Exactly the decision #158 left to the spike, decided with the reason written downBadge.compute/2is pure: Verified needs two distinct sources including a primary and nothing against; a register that names who did say it is a cited claim; a contradiction outranks agreement; the #65 score recorded beside it and landed asconfidence:supportson the misattribution. See residual 1/ops/discoverygains Quotation verifier with a row per checker and why it is offResiduals, none blocking:
content_items.metadata, notcontent_revisions.metadataas Quotes research: which sources are reliable, measured on Voltaire and Vonnegut — Wikiquote re-tested through Parsoid, Wiktionary citations already absorbed, the aggregators fail corroboration, a keyless primary-text check (toward a:quoterow) #158's schema table said. Defensible (it is a property of the line, not of one wording), but the schema table should be corrected so the next reader does not look in the wrong place.active: falseand will stay that way until you act:google-booksneedsGOOGLE_BOOKS_API_KEYin.env(decision 2, yes on the 23rd, not yet done) andquote-investigatorneeds the permission email (decision 3, same). Until then in-copyright authors top out at Plausible, as the issue predicted.VerifyWorkerat :07 and :37. Nothing has been verified on the dev database, because the server on 4007 has been refusing pages since yesterday.Misquotationscounts every row as register by construction, including Unsourced (93 of 150), which is a weaker claim than Misattributed; the code treats them the same. Fine for a disclosure, worth a word in the doc.Dev database, after this audit: the
verification_runsmigration is applied, andSources.Catalog.seed!/0has been run so the checker rows exist (gutenbergactive;google-books,internet-archive,quote-investigator,wikisourceinactive, as designed). Together with yesterday's index and predicate seed, the database now has everything builds 2–5 need. The server still needs its restart.holden commentedon Sep 24, 2026
Closing. Builds 1–5 are merged and audited (build 1 A, #173; build 2 A− → A, #166/#167; build 3 A, #168; build 4 A−, #169; build 5 A−, #171). Main @ 801fd15, 1,866 tests green. Every acceptance box in this issue is ticked with evidence, and the dev database carries every migration and seed the builds need.
Build 6, the public-domain corpus, is now its own issue: #174 — with the three decisions it needs.
What this issue leaves open elsewhere: #172 (the sense-to-concept coverage gate: the shelves show on 1 % of words until it is answered), #165 (people who are not word-referents), the Google Books key and the Quote Investigator email (decisions 2 and 3 here, said yes, not yet done), and the untyped-but-cited Kaikki lines (build 1's finding; one clause in
WordPage.quotation?/1once decided). #65 is closed as superseded by this issue.Two corrections to this body that the builds taught: build 5 writes the provenance on
content_items.metadata, notcontent_revisions.metadata; and Wikisource and the Internet Archive were not built as checkers, by measurement (neither ties a text to an author by an identifier). The verifier doc (docs/integrations/verifier.md) has the numbers.holden commentedon Sep 24, 2026
The audits' residuals are PR #179: the README states where the badge lives (
content_items.metadata), the verifier doc says Unsourced is register, and a Wikiquote credit is now the first linked person (P31=Q5; there is no pageprop for it). The untyped Kaikki lines wait on your yes.Merge pull request #179 from razrfly/claude/quotes-residuals-158