Repository navigation
Description
Rewritten 2026-09-27 after the linking reassessment. The earlier body proposed rebuilding the public-domain quote corpus as
wikiquote-pd-v2by reselecting from every concept the registry now holds: over 3,000 requests to verify lines for pages nobody has opened. That is bulk prepopulation of a dictionary that is meant to grow through use, and it is withdrawn. What remains is a decision about how the verified tier grows at all. The earlier blockers (a corpus hard-wired to v1, a builder with no budget) are kept below as facts, because either answer to the decision needs one of them fixed.
What the corpus is, and what it is for
wikiquote-pd-v1 is 1,053 quotations over 197 concepts, each Verified against a pre-1931 Gutenberg text by build 5's checks (#171), committed as a manifest and seeded as source records with a quotation fingerprint as identity (ADR 0003). On the Quotes shelf its lines carry Verified; live Wikiquote lines carry Plausible. Its value is provenance pinned to a primary text, a stable set that survives a database reset (restored, not regenerated), and lines that are there when Wikiquote is not.
Live discovery is what serves a page: a visit admits one Wikiquote run for the page's concept (or the concept the hop reaches), the result is cached under its refresh and retention policy, and a rule change re-versions the recipe. The corpus is a verified tier on top of that, never a substitute for it.
Why not rebuild
- A reselect starts from 10,862 concepts and 3,410 hop origins (v1: 6,972 and 2,165) and would verify lines for concepts no page has reached. The cost is real (>3,000 requests; v1's ledger was 2,936) and the benefit lands on pages nobody opens.
- The visit-driven lifecycle already benefits every visited page from the hop with no rebuild, which is what Sense-level shelves show on 1 % of words: the concept hop, the word-level tier from corroborated candidates, and promotion — so Quotes, Artworks and Images appear on most pages without a name match #172 delivered.
- The corpus's selection rule is frozen at build time; each rule change would mean another rebuild. Verification on demand follows the rules as they are when a page is visited.
The decision owed by the owner
How does the Verified tier grow?
- By use (recommended). When a page's live Wikiquote run returns lines, a bounded background job runs build 5's verification on those lines under a request budget (Gutenberg paced as today) and records the ones that pass as verified source records. Verified lines then show as Verified on the next read; a page visited by nobody stays Plausible-only. Nothing is reselected, and the set grows exactly as fast as the site is used.
- By rebuild. Keep the manifest model and rebuild periodically with a budget. Predictable, reproducible, and mostly spent on unvisited words.
- Both. v1 stays as the committed seed; new verified lines are added by use and periodically folded into a committed manifest so a reset restores them.
Under 1 or 3, a second decision: are the verified lines a committed manifest (reproducible after a reset, one more file to maintain) or database rows only (restored from a backup like routing and curation state, #205's rule)? Recommended: rows under a wikiquote-pd-live source, folded into a committed manifest at each release.
What is needed under either answer (kept from the earlier body)
- The corpus module hard-codes the v1 slug and manifest path (
Quotations.Corpus@slug,@manifest_path); the manifest kind names v1; the catalog registers only v1; the shelf reads only v1. A second source row, whichever way it fills, needs the corpus to take its version from the manifest or the source row. - The builder and the verifier pace requests but have no budget. Any of the three answers needs one, with a per-stage ledger, like every discovery provider has.
Acceptance for this issue
- The owner answers the two decisions in a comment.
- A child implementation issue in the Explore next discovery providers: quotes, music, books, news, images, games, nature and social APIs #100 spike shape (source, budget, count, manifest or rows, tier) is opened for the chosen answer, with Sense-level shelf coverage and match quality: the measurement instrument (denominator, fixed sample, precision reviews) after #172 and #217 #214's relevant column as its measurement.
-
priv/quotes/manifests/wikiquote-pd-v1.jsonstays byte-identical throughout.
Activity
[-]wikiquote-pd-v2: make the quotes corpus version-aware, then rebuild it with the concept hop inside a request budget[/-][+]Grow the verified quote tier by use, not by rebuild: the decision that replaces the wikiquote-pd-v2 reselect[/+]holden commentedon Oct 21, 2026
Folded into #193, the curation programme's one open tracker, on the owner's word (21 October 2026). The specification here stands; reopen when the work starts. Status is in the consolidation comment on #193.