Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ This file is the **living record** of the project: what it is, what it is not, t

Automatic cultural discovery and the direct-browser GIPHY integration are documented in [`docs/integrations/giphy.md`](docs/integrations/giphy.md). Durable cross-provider film identity and its adapter/backfill contract are documented in [`docs/integrations/source-identity.md`](docs/integrations/source-identity.md).

## Current delivery β€” 11 September 2026
## Current delivery β€” 17 September 2026

The discovery layer landed on 12–13 September in four PRs: automatic CineGraph film discovery on definition pages ([#88](https://github.com/razrfly/dictionary/issues/88) / [PR #89](https://github.com/razrfly/dictionary/pull/89)), the shared cached-discovery lifecycle with GIPHY as a browser-only shelf ([#92](https://github.com/razrfly/dictionary/issues/92) / [PR #94](https://github.com/razrfly/dictionary/pull/94)), durable source identity for discovered films ([#93](https://github.com/razrfly/dictionary/issues/93) / [PR #95](https://github.com/razrfly/dictionary/pull/95)) and the Wikidata-linked Artsy artwork catalog ([#86](https://github.com/razrfly/dictionary/issues/86) / [PR #98](https://github.com/razrfly/dictionary/pull/98)). The 17 September triage ([#102](https://github.com/razrfly/dictionary/issues/102)) grades it **B**: only CineGraph runs through the discovery pipeline (`Discovery.Providers.server_providers/0` returns `["cinegraph"]`), Artsy is a seeded catalog on its own client, GIPHY has no recorded live verification, and three shared-code assumptions β€” POST-only transport, cursor-only pagination, a `[:film, :gif]` allowlist β€” stand between the Met and a cheap second provider. #102 sequences what comes next: harden the provider contract, then the Met ([#99](https://github.com/razrfly/dictionary/issues/99)), then one text provider ([#100](https://github.com/razrfly/dictionary/issues/100)), then the category-grouped UI and [#96](https://github.com/razrfly/dictionary/issues/96), then the assessor ([#101](https://github.com/razrfly/dictionary/issues/101)) and personas ([#97](https://github.com/razrfly/dictionary/issues/97)). The development database is `devils_dictionary_v2`; `devils_dictionary_dev` is the pre-#74 baseline and has no discovery tables.

The #82 reader milestone is delivered through [PR #83](https://github.com/razrfly/dictionary/pull/83): 99.50% definition coverage, typed word/entity discovery, connected person/work/edition pages and bounded pagination. Final review validation passes 774 tests. Public contributions remain gated; the original participatory MVP is not complete.

Expand Down Expand Up @@ -150,6 +152,11 @@ MVP-0 was done when every scorecard row passed, and it does: **animals 36 / 36 g
| **#74 P3** one connected flow (`0dbb752`): `/words/:id/:slug` and `/entities/:id/:slug` canonical, `/define/:slug` a resolver that *offers the choice* where a slug names more than one lemma, `/connections/:id` readable from either endpoint, and `/connect` the authenticated composer whose relation list is read off `predicate_endpoint_rules`. The acceptance walk is real clicks: nepotism β†’ Bierce's definition β†’ Bierce β†’ his works β†’ the edition β†’ back | βœ… 2026-09-09 |
| **#79 W1** the corpus spike and the budgets ([evidence](docs/rebuild/w1-2026-09-10/README.md)): `mix dd.absorb wiktionary --sample-every N` samples by hash of the headword, not by position, because the dump is in page order and `--limit` takes a biased head. Two near-disjoint samples (N=30, N=31, **94,034 records**) put the full pass at **~15 min, ~3 GB, projected DB ~9.5 GB**, and coverage at **12.7 % β†’ low-to-mid 90s**; the census corrects #79's guess β€” **1,487,639** English records, not ~1.3 M. The spike also found the defect the three test scopes had hidden: `origin_key` is `varchar(255)` and Wiktionary's `coordinate_terms` names a whole series in one target (`A-shaped - … - Z-shaped`, 315 bytes), aborting the entire materialize transaction β€” 1 key in 94,034 records, fatal every time. All four builders now share a bounded key (readable head + digest tail): deterministic, inspectable, **no migration**. 725 tests | βœ… 2026-09-10 |
| **#79 N1** the prototype surface stripped (#77 Β§1, Β§2): the navigation was `Animals Β· Health Β· Imports` β€” one test population and two consoles standing in for the product's structure β€” and it is now a logo and a theme toggle. **24 sites.** The three consoles moved to `/ops/{scopes/:slug,health,imports}`, with the old paths kept as **302s** (not 301s: `/ops` is a first guess, and a browser caches a permanent redirect near-forever). **15 production `\\ "animals"` defaults removed** and the argument made required β€” `Health`'s moduledoc now names which rows are whole-corpus and which are one population's β€” and all four `--scope` flags refuse a missing selection, `dd.rebuild` **before** it builds a plan, because its stages 8–9 go to the API for hours. No selection is a state the consoles render, not a value they invent. Public copy lost the internal names: the homepage reads *1,541,668 words indexed Β· 282,098 with at least one definition Β· 6 sources so far*, and `/define/ambrose-bierce` reads *One source so far Β· Open English WordNet 2025* instead of *not in Animals or Culture or Emotions*. Found while there: `mix dd.health --scope emotions` computed its footnote off a second, Animals-scoped ledger; Animals' twelve Wiktionary category exclusions were compiled into a command that accepts any scope, and are now `priv/scopes/animals.json`'s; and the kit's buttons had no disabled styling, so the absorb button that refuses an unscoped API import still offered the hand cursor. **Animals stays as ordinary content** β€” a test deletes the scope row and walks the app. 752 tests (from 725); animals 43/44, culture 41/42, emotions 42/43, X3 failing identically on `main`; U4 re-attested at 375 px, all twelve **0** | βœ… 2026-09-10 |
| **#88** automatic CineGraph discovery on definition pages ([PR #89](https://github.com/razrfly/dictionary/pull/89)): `discovery_mappings` / `discovery_runs` / `discovery_results`, the `Discovery.Provider` behaviour, a DB-coordinated rolling request budget, positive and negative caches with a freshness policy, the run lease and cleanup worker, `mix dd.discovery` and `mix dd.discovery.check`; **live-verified** against `cinegraph.org` ([handoff](docs/discovery/issue-88-handoff-2026-09-12.md)); audited B, repaired, re-audited **A+** ([re-audit](docs/audits/2026-09-12-issue-88-reaudit.md)); 875 tests | βœ… 2026-09-12 |
| **#92** unified cached discovery ([PR #94](https://github.com/razrfly/dictionary/pull/94)): per-source freshness overrides and shared quotas for the pipeline; GIPHY restored as a **browser-only, transient** shelf (`assets/js/giphy_shelf.mjs`) because GIPHY has not granted caching approval β€” no live verification recorded, and its `node:test` file is not wired to anything | βœ… 2026-09-12 |
| **#93** durable source identity for discovered films ([PR #95](https://github.com/razrfly/dictionary/pull/95)): the CineGraph / TMDb / IMDb / Wikidata crosswalk into `external_identifiers`, local film entries with backlinks, legacy-result backfill ([contract](docs/integrations/source-identity.md)); the identity adapter is duck-typed, so a new provider gets it without wiring | βœ… 2026-09-12 |
| **#86** the Wikidata-linked Artsy artwork catalog ([PR #98](https://github.com/razrfly/dictionary/pull/98)): `Artsy.Client` + `RequestCoordinator`, the `Artworks` context and seeder, `/artworks`, `mix dd.artworks.seed` / `mix dd.artsy.withdraw`, four manifests and the meaning β†’ gene mappings; **0 migrations**; ≀194 live requests across the feasibility probe, the pilot and two audit-repair rounds (C+ β†’ B β†’ repaired; [repair](docs/audits/2026-09-12-issue86-repair.md), [re-audit repair](docs/audits/2026-09-13-issue86-reaudit-repair.md)); 9 artworks seeded; the definition page reads the local catalog only, so Artsy's retirement is survivable. Its provider module has no `retrieve/4` β€” the catalog runs outside the discovery pipeline | βœ… 2026-09-13 |
| **Discovery triage** ([#102](https://github.com/razrfly/dictionary/issues/102)): #73, #74, #75, #79, #86 and #93 closed with evidence comments; the layer graded **B**; the order fixed β€” provider-contract hardening, the Met, one text provider, the category-grouped UI, then the assessor and personas | βœ… 2026-09-17 |

---

Expand All @@ -174,7 +181,7 @@ Everything below is designed for in the MVP-0 schema and adds tables rather than

**Media**
- [ ] Evidence wall (#67): paste a link, unfurl it, attach it to a word
- [ ] Media providers as sources: YouTube, TikTok, Instagram, X, Giphy, Unsplash, Spotify/Genius, TMDb (via Cinegraph), news
- [~] Media providers as sources β€” **landed**: films via CineGraph (#88), an Artsy artwork catalog (#86), GIPHY browser-only (#92); **next, in [#102](https://github.com/razrfly/dictionary/issues/102)'s order**: the Met (#99), one text provider (#100), then YouTube, TikTok, Instagram, X, Unsplash, Spotify/Genius, news
- [ ] Images on every word and thing (Wikipedia thumbnails, Wikidata P18, Commons)

**Product**
Expand Down