diff --git a/README.md b/README.md index fd9a6197..fd3aedb4 100644 --- a/README.md +++ b/README.md @@ -358,6 +358,11 @@ mise install # erlang 28.1, elixir 1.19.0-otp-28, mix setup # deps, empty db, assets ``` +The server must be **PostgreSQL 15 or newer**, in development, CI and production +alike. The curation migration uses column-list referential actions +(`ON DELETE SET NULL (column)`), which older versions reject. On an older server it +stops with an explicit error rather than failing partway through. + That leaves a working app with an **empty** database. Two things are deliberately not in git and have to arrive another way: diff --git a/docs/curation/persistence-slice-1.md b/docs/curation/persistence-slice-1.md new file mode 100644 index 00000000..d7257711 --- /dev/null +++ b/docs/curation/persistence-slice-1.md @@ -0,0 +1,313 @@ +# Curation persistence, slice 1: default configuration and manual compositions + +First delivery slice of [#196](https://github.com/razrfly/dictionary/issues/196) +and [#201](https://github.com/razrfly/dictionary/issues/201) under +[#193](https://github.com/razrfly/dictionary/issues/193). + +This slice adds the configuration and profile identities that curation is frozen +against. It also adds the durable records every published composition rests on: +exact-reference versions, their eligibility, and publication receipts with the +authority that authorized each one. + +**Publication has two authorities, and only one exists yet.** See +[Publication authority](#publication-authority). + +* **Routine curation is meant to publish on a panel decision.** The configured + agent panel reaches consensus on a version, the version passes the evidence and + policy checks, and the publication service publishes it. There is no + per-selection human sign-off. That path needs genuine decision records from panel + runs (#197), which do not exist, so it is **unavailable**. Nothing here fabricates + a run, vote or decision to stand in for one. +* **The operator path is what this slice implements**, as an optional mechanism: + + provision composition → manual version → operator review → operator publication + + It is for manual compositions made before any model exists, and later for + overrides, corrections and withdrawals. It is not the gate every selection must + pass. + +This is not the whole of #196 or #201. Runs, candidates, ballots, round responses, +co-proposers, leases, schedulers, budgets and public history are later slices, listed +under [Deferred](#deferred). + +**PostgreSQL 15 or newer is required.** The item references use column-list +referential actions (`ON DELETE SET NULL (column)`), which arrived in 15. The +migration checks the server version first and stops with an explicit error on an +older one. + +## Ownership + +| Owner | Owns | This slice | +|---|---|---| +| #196 / #201 (this slice) | Configuration identities and versions, profile identities and versions, rosters, compositions, their versions, items, operator reviews and publication receipts | Creates them | +| #194 / PR #205 | Pages, paths, page memberships, On bodies, classification, page publication and indexability, **and the page ↔ composition binding table** | Untouched. The binding migration follows this one, per #205's recorded contract (`page_composition_bindings.composition_id` → `editorial_compositions.id`; the language must equal the page locale; the scope must be compatible). | +| #190 | Bot claim writes | None are made here. An item may reference an assertion revision only if it is already current, active and publicly visible (`Claims.visible/2`). | +| #195 / #197 | Model runtime, panel execution, **and the panel-decision publication authority** | Absent. A configuration version is **manual-only** until a later migration adds a validated model configuration. | +| #203 / #204 | History projection and the method page | Absent. The records here are what #203 will project. | + +## Reconciliations with the #196 draft + +The draft table in #196 assumes a model exists. Taking it as written would force this +slice to fabricate that model, which the task forbids. These are the deliberate +differences: + +1. **No `local_model_configs` table, and no `local_model_config_id` column.** A + configuration version with no model is *manual-only*. Panel readiness is always + `false`, and `resolve_default(purpose: :panel)` answers `{:unavailable, :panel_not_available}`. + The migration that adds a real, validated model configuration adds the column and + the panel check. No digest is invented to satisfy a foreign key. +2. **Profiles exist as identities, not dossiers.** The five inspired-by identities are + seeded as `proposed`, with no version, no bot actor and no subject entity. A profile + becomes `admitted` only through `Profiles.admit/4`: a human reviewer, a version whose + dossier cites sources, deceased-status evidence and a reason. The database refuses an + `admitted` row without these. No version is seeded, because no dossier has been + reviewed; nothing is quoted or asserted about the five writers. +3. **The global-default version 1 has an empty roster.** It carries the Bierce-first + lead policy and the three-highlight limit, which is all manual composition needs. + It references no profile versions, because none exist to reference. The five-member, + 4/5, two-round roster is a later version, activated once admitted profile versions + and a model configuration exist. +4. **Seeding does not activate anything.** `mix dd.curation.seed` creates the + configuration identity and version 1 in state `draft`, with no current version, so + the default resolves as `{:unavailable, :no_active_version}` until an operator runs + `Configurations.activate/4`. That is an audited human action with a receipt. A seed + or a migration never promotes a configuration, a profile or a composition. + Configuration administration stays a human act; that is a separate policy choice + from per-selection publication. +5. **Composition freshness and refresh columns are omitted.** These are + `last_completed_at`, `refresh_after`, `next_retry_at`, the refresh generation and the + desired input fingerprint. They are #198's, and nothing in a manual path writes them. + Each composition's clocks here are its own receipts: version, review and publication + timestamps. +6. **`origin` is `manual` only**, and there is no `originating_run_id` until runs exist. + A manual version never gets a run, participant, ballot or nomination. +7. **The catalog-work reference form.** A work shown from a committed catalog manifest, + such as `wikidata-famous-v1`, has no source-record revision row. The item stores the + manifest name, checksum and row identity. This is the exact committed revision; no + content revision is invented. A work with a real source-record revision references + it instead. Either way the evidence must identify the selected object (K7). +8. **Publication authority is explicit and versioned.** The draft's receipt has a + required `authorizing_review_id`, which would make a human review the only possible + authority. Here every receipt names its `authority_kind`. Only `operator` is valid + now, and the panel decision is the intended routine authority, added with #197. + +## Publication authority + +A publication receipt records **what authorized it** (`authority_kind`). The commit it +authorizes is the same for every authority (`Publications`): lock the composition, then +check the expected pointer, that the version still stands (scope, configuration version, +a clean evaluation to its own eligibility fingerprint), and the authority. Only then +does it write the receipt and move the pointer, in one transaction. + +### `operator` (available) + +A reviewer publishes a version whose **latest** operator review is an acceptance of its +fingerprint (`Publications.publish/3`), or withdraws a publication +(`Publications.withdraw/3`). The database refuses an operator receipt from anyone but a +human reviewer, one citing a review that is not the version's latest, or one that did +not accept, or one of another fingerprint. It is optional, and it is used for: + +* manual compositions, before any model exists; +* overrides and corrections of panel output, once there is some; +* withdrawal at any time. + +### `panel_decision` (intended routine path, not available) + +A routine selection publishes because the configured panel decided on it, not because a +person signed it off. The database refuses this authority today +(`authority_kind IN ('operator')`, and the authority trigger), because the decision +records it needs do not exist. **The integration that adds it (#197, with #195's model +configuration) must implement all of the following. Nothing less authorizes a +publication:** + +1. **A genuine decision record.** It is a finalized, immutable run result of the + composition's own configuration **version**: + - the actual participants are that version's roster (admitted profile versions, + weight 1); + - every ballot and per-round response status is recorded; + - a missing response is a status, never an abstain or a support; + - the selection trace is deterministic (`selection-v1`, #203), and support counts + can be recomputed from the ballots; + - the configured consensus (4/5 support) is met for the lead and each highlight. + + The composition version it selected carries `origin = 'panel'` and + `originating_run_id`. Its items carry `selection_origin = 'panel_recommendation'`. + Model notes are written by the profile's `bot` actor, never a person. +2. **Receipt shape.** `authority_kind = 'panel_decision'` with a required + `authorizing_decision_id`, and no review. A composite key ties the decision to the + published version, and so to its composition and configuration. The actor is an + accountable non-human principal for the curation pipeline: never a persona bot, and + never a person's account. +3. **Database checks at insert and commit.** They are the counterparts of this slice's + operator checks: + - the decision is finalized and belongs to the published version; + - its run's configuration version is the version's; + - quorum was met; + - its eligibility fingerprint equals the version's and the one published; + - the version's latest operator review, if any, is not a rejection or a withdrawal + (an operator veto stands); + - the receipt chain and pointer checks apply unchanged. +4. **Service.** `Publications` publishes on a decision through the same commit + (pointer, scope, configuration version, eligibility, idempotency keyed on the + decision). A background job may switch the pointer only this way, and only with a + finalized decision. +5. **Tests.** Required before the path is called ready: + - a **missing** decision is refused; + - a **forged** one is refused: not finalized, recomputed counts that disagree, + another version's decision, or a person's review presented as a decision; + - a **stale** one is refused: a fingerprint or configuration version that has since + moved; + - a **cross-configuration** decision is refused; + - an operator rejection vetoes publication; + - a panel decision never accepts a semantic claim. + +What does not change for either authority: + +* semantic claims are reviewed on their own (#190, #105). A panel decision, like an + operator acceptance, never accepts a claim; +* source eligibility, rights, withdrawal and read-time withholding; +* actor provenance on every version, note and receipt; +* receipts are append-only, and approval never transfers to a new version. + +## Invariants → constraint → test + +The migrations are `20260926233642_create_curation_foundation` and +`20260927094256_repair_curation_integrity`. The services live under +`DevilsDictionary.Curation`. Test modules are under `test/devils_dictionary/curation/`: + +| Module | Covers | +|---|---| +| `ConfigurationsTest` | C | +| `CompositionsTest` | K | +| `PublicationsTest` | R | +| `PublicationRaceTest` | the races, unboxed on committed connections | +| `AuditFindingsTest` | the regressions from the independent audit, one `describe` per finding | + +**How the tests reach COMMIT.** A sandboxed test never commits, so the deferred +triggers would never fire there. There are two answers: + +* sandboxed tests end their writes with `settle!/0` (`SET CONSTRAINTS ALL IMMEDIATE`, + then `ALL DEFERRED`), and assert refusals with `refused/1`, which does the same inside + a savepoint. No constraint in the schema is `DEFERRABLE INITIALLY IMMEDIATE`, so this + restores the initial state exactly; +* `PublicationRaceTest` repeats every deferred check at a real COMMIT, and runs the + races on separate connections. + +### Configurations and profiles (#201) + +| # | Invariant | Enforced by | Test | +|---|---|---|---| +| C1 | Only `system` ownership exists; at most one **enabled** global default | check `ownership_kind = 'system'`; partial unique index on `role` where `role = 'global_default' AND state = 'enabled'`; `activate/4` refuses `:another_default_enabled` | "only system ownership exists and one global default is enabled" | +| C2 | A current version belongs to its configuration | composite FK `(current_version_id, id)` → `curation_configuration_versions (id, configuration_id)` | "a pointer to another configuration's version is refused" | +| C3 | The pointer and the state move only with a receipt, and each receipt continues the last | append-only `curation_configuration_activations` (`activate` or `disable`); deferred trigger: pointer and state equal the latest receipt's, the latest receipt's `previous_version_id` equals the one before it, and a configuration with no receipt is a draft with no pointer | "a pointer moved without a receipt fails at commit"; "every deferred check refuses at a real COMMIT" | +| C4 | Versions and rosters are immutable; a roster is fixed at creation | update/delete triggers; `roster_hash` recomputed from the seats at commit | "versions and rosters cannot be edited, removed or extended" | +| C5 | A seat's profile version belongs to its profile; one slot and one profile per version; weight 1; only an admitted version | composite FK `(profile_version_id, profile_id)`; unique `(version, profile)` and `(version, slot)`; check `voting_weight = 1`; `create_version/3` refuses `:profile_not_admitted` | "a seat's profile version is its profile's…"; "a roster seat must be an admitted profile's admitted version" | +| C6 | An admitted profile has one admitted version citing sources and deceased-status evidence, a bot principal, a human reviewer, a time and a reason | check on `curator_profiles`; admission trigger (reviewer `user` actor, `bot` actor, non-empty refs, a new version needs a new admission); `Profiles.admit/4` | "needs a reviewer, a sourced version with deceased-status evidence, and a bot principal" | +| C7 | Default resolution is server-side and default-only; missing, draft, disabled, unready and panel are explicit | `Configurations.resolve_default/1`, which takes no configuration argument | "answers the default or an explicit unavailable, never another configuration"; "a panel is never available, and a version this build cannot run is unready" | +| C8 | Activation is a reviewer's, idempotent, checks the pointer the caller saw, and publishes nothing | `activate/4` and `disable/3`: reviewer role rechecked under the account row lock, configuration row lock, `:expected`, idempotency replay; human-actor trigger on receipts | "activation is a reviewer's audited, idempotent act and publishes nothing"; "a revoked reviewer is refused…"; `Dd.Curation.SeedTest` | + +### Compositions (#196) + +| # | Invariant | Enforced by | Test | +|---|---|---|---| +| K1 | Identity is `(scope_kind, scope_signature, language, configuration)`; two configurations keep independent histories, pointers and clocks | partial unique index where `state = 'active'`; per-composition version numbers, parents and receipts; `provision/3` idempotent under an advisory lock | "the identity is (kind, signature, language, configuration)…"; "two configurations over one scope keep independent histories, pointers and clocks"; "two provisions of one scope make one composition" (unboxed) | +| K2 | The signature is the memberships', never a spelling, and every scope has a receipt | deferred trigger: the signature equals `sha256` of the sorted members and the latest scope-change receipt, and receipts chain; `provision/3` takes lexeme ids | "a scope change is explicit and audited; a signature that does not match fails at commit" | +| K3 | Overlapping or split scopes are reconciled, not guessed; a lexeme scope has one member | `provision/3` refuses `{:overlapping_scope, ids}`; `change_scope/4` audited; deferred trigger counts members; memberships `ON DELETE RESTRICT` | "provisioning takes registry ids, refuses overlap and a mixed language" | +| K4 | A version's configuration version is its configuration's; a parent is from its composition; made only against the current scope of an active composition | composite FKs; insert trigger on versions; `create_version/3` requires an enabled configuration and `:expected_parent` | "cross-configuration, cross-composition and cross-object links are refused"; "two authors cannot both write the next version"; "a draft or disabled configuration has no compositions made under it" | +| K5 | Versions and items are immutable; an arrangement is fixed at creation; one version per arrangement **per configuration version** | triggers; `arrangement_hash` (length-prefixed fields) recomputed from the items at commit; unique `(composition, configuration_version, arrangement_hash, eligibility_fingerprint)`; the fingerprint includes the configuration version | "a version cannot be edited, removed or have items added"; finding 3 "an unchanged arrangement is reissued under the new version, and needs its own review" | +| K6 | At most one lead and three highlights; the lead is a content item at position 1 | checks; unique `(version, role, position)`; the configuration version's `max_highlights` | "a fourth highlight or a second lead cannot be stored" | +| K7 | An exact revision belongs to its item's object, and is the current, active one; **a work's evidence is of that work** | composite FKs `(content_revision_id, item_object_id)` and `(sense_revision_id, item_object_id)`; `Eligibility`: superseded, withdrawn, words hash, catalog checksum and row, a catalog pin matching the object's verified identifier, a source record that materialized the object | "exact references are checked against the registry when the version is made"; finding 4 (three tests) | +| K8 | Exactly one intended meaning, on the scope | insert trigger (one of sense revision or lexeme); `Eligibility` (`:meaning_off_scope`) | "exact references are checked…" | +| K9 | Manual versions have manual origin, a reason, a human author and no run; **a note is its authenticated author's** | checks `origin = 'manual'` and `selection_origin = 'manual'`; human-actor trigger on `created_by_actor_id`; a note spec is its text alone; `note_author_actor_id` must be the version's author (a `bot` for a model note), under that actor's own label | "a version is manual, human, reasoned, exact, and invents no history"; finding 5 (three tests) | +| K10 | Bierce first, among entries that may lead | `Curation.LeadRule` at creation, review, publication and read. A definition is on the page only through a `defines` claim that passes `Claims.visible(:public)`. A priority entry that may not be displayed neither leads nor blocks | "an applicable Bierce entry leads, and a version without it is refused"; "where Bierce has no entry…"; finding 2 (four tests) | + +### Review and publication (#196) + +| # | Invariant | Enforced by | Test | +|---|---|---|---| +| R1 | Only reviewers make operator decisions; decisions are append-only and idempotent; an acceptance is of the version's own fingerprint | human-reviewer trigger; update/delete triggers; unique `idempotency_key`; acceptance trigger; `Reviews.decide/4` rechecks the role under lock and refuses a version that no longer stands | "a bot or a non-reviewer cannot review"; "decisions are append-only and idempotent"; "an acceptance is refused once the version no longer stands" | +| R2 | Approval is not publication, and not a claim or page decision | separate tables and services | "approval publishes nothing and accepts no claim or page" | +| R3 | The pointer moves only with a receipt, **and the receipt names a valid authority**: for `operator`, a reviewer and the version's latest review, an acceptance of the published fingerprint; any other authority is refused | composite FKs on receipts; `authority_kind` check; authority trigger; deferred trigger (pointer = latest receipt) | "the database refuses a pointer without a receipt…"; "every deferred check refuses at a real COMMIT"; finding 6 (five tests: missing, panel, forged, stale, cross-configuration) | +| R4 | Publication is atomic, checks the expected pointer, scope, configuration version and fingerprint, and is idempotent | `Publications` commit, in one transaction | "needs an accepted review, then writes a receipt…"; "is idempotent, checks the expected pointer…"; "stale fingerprints, a withdrawn approval and a changed configuration refuse" | +| R5 | Concurrent publications never overwrite each other, even past the service | row lock and expected pointer in the service; in the database, each receipt must continue the one before it, so the second COMMIT from one pointer fails | "two reviewers publishing from the same pointer: exactly one wins…"; "writers that skip the service's lock still cannot both commit from one pointer" (both unboxed) | +| R6 | Read time withholds and never substitutes | `Published.current/1`: a version-level problem withholds all of it (retired, configuration changed, scope changed, approval withdrawn); an item-level one withholds that item with its reason; a later Bierce entry, or a rejected defining claim, withholds a lead | the R6 tests in `PublicationsTest`; findings 1 and 2 | +| R7 | Source deletion is never blocked, never revives an item, and keeps nothing prohibited | every source reference `ON DELETE SET NULL (column)`, including the item's object and claim; the item guard permits only that nulling; `required_references` records at insert what the item was made with, so a nulled one withholds it (`:claim_deleted`, `:object_deleted`, …); items hold ids, hashes and locators | "a deleted source revision tombstones the item, and nothing prohibited is kept"; "an item's references can be nulled by a deletion, and never repointed"; finding 1 (two tests) | + +### Changed while implementing + +* **Disabling is a receipt too.** `curation_configuration_activations.action` is + `activate` or `disable`, and the commit check compares the state as well as the + pointer. A configuration's states are `draft`, `enabled` and `disabled`; retirement + is left for later. +* **Receipts chain.** A receipt's `previous_*` must equal the one before it. This is what + makes a race fail in the database, not only in the service. +* **An item's object reference nulls on delete.** It is nullable only for a catalog + work, which may have no registry object. +* **A claim reference must be publicly visible.** It must be current, active and + visible under `Claims.visible(:public)`. The earlier wording, "accepted", would have + hidden every imported claim, which has no review. +* **A lexeme is not deleted from under a scope** (memberships and `meaning_lexeme_id` + are `RESTRICT`). Lexemes hold no source text, so R7 does not apply. + +### Repaired after the independent audit (`b8211ed`) + +`20260927094256_repair_curation_integrity` is a separate migration, so every database +that applied the first one is repaired by `ecto.migrate`. It is reversible: rollback +restores the previous triggers, functions and index. + +1. **Deleted claim evidence revived an item.** A deleted claim and "no claim" were + both `nil`. Each item now records `required_references` at insert (written by the + database, not the caller), and a nulled required reference withholds it. +2. **A rejected `defines` claim still authorized a lead.** `LeadRule` now reads the + relationship through `Claims.visible(:public)` for both priority and fallback leads, + and an applicable priority entry must itself be eligible to lead. +3. **A configuration upgrade blocked reissuing an unchanged selection.** The + deduplication key and the fingerprint now include the configuration version. The + reissued version needs its own review. +4. **An unrelated object could carry a catalog pin.** A pinned object must carry the + row's identity as a verified external identifier, and a source-record work must be + the object its record materialized. A catalog pin with no object remains a + catalog-only work. +5. **A caller could name a note's author.** A note spec is its text alone. The author + is the acting principal, recorded as `note_author_actor_id` under its own label, + and the database refuses anything else. + +The publication contract was also reconciled with the intended product (see +[Publication authority](#publication-authority)), and PostgreSQL 15+ is documented and +checked. + +### Not written by this slice + +No `curation_runs`, participants, candidates, ballots, nominations, decisions, +attempts or round responses; no `local_model_configs`; no page bindings. No +`assertions`, `assertion_revisions` or `assertion_reviews` row is written by any +service here, which a test asserts. + +## Deferred + +These are later scoped slices, each owned by its issue: + +- **#196 / #197:** runs, participants, candidates, proposals, ballots, round responses, + nominations and attempts. Five admitted profiles, weight 1, fixed 4/5, two rounds, + ten normal generations plus at most one repair; a missing response is never an + abstain ballot. **The panel-decision publication authority**, as specified above. +- **#195:** `local_model_configs`, a real validated model, and panel-mode configuration + versions. +- **#198:** refresh admission, freshness clocks, leases and the one global inference + budget. +- **#194:** the page ↔ composition binding, and the ADR 0004 default/alternate wording + amendment. +- **#156 Phase 2:** mapping `Published.current/1` onto the Phase 1 view model. PR #202 + is not merged, so this slice does not depend on it. Production selection stays + disabled; no page reads compositions yet. +- **#203 / #204:** the history projection, profile pages and the method page. +- Later stages of #201: domain defaults and personal views. + +## Operating notes + +- `mix dd.curation.seed` is idempotent. It creates the `global-default` configuration + (draft, no current version) and the five proposed profiles. It activates nothing and + approves nothing. +- Registry object ids exist only in the database, so curation state is **restored, not + regenerated**, like routing state (#205's recovery procedure). +- PostgreSQL 15 or newer, in development, CI and production. diff --git a/lib/devils_dictionary/claims/contributions.ex b/lib/devils_dictionary/claims/contributions.ex index a8fab935..90650290 100644 --- a/lib/devils_dictionary/claims/contributions.ex +++ b/lib/devils_dictionary/claims/contributions.ex @@ -939,6 +939,14 @@ defmodule DevilsDictionary.Claims.Contributions do end end + @doc """ + The `user` actor of an account, created on first use. It is what every + human decision is recorded against, here and in curation (#196). Call + inside a transaction: the account row is locked while the actor is found or + made. + """ + def account_actor!(%User{} = user), do: actor!(user) + defp actor!(user) do # Serializes first-use actor creation for this account. Repo.one!(from u in User, where: u.id == ^user.id, lock: "FOR UPDATE") diff --git a/lib/devils_dictionary/curation/authority.ex b/lib/devils_dictionary/curation/authority.ex new file mode 100644 index 00000000..a4651f13 --- /dev/null +++ b/lib/devils_dictionary/curation/authority.ex @@ -0,0 +1,43 @@ +defmodule DevilsDictionary.Curation.Authority do + @moduledoc """ + Who may act on curation, checked the way `Contributions.review/6` checks a + claim review. + + The role is read from the account row under `FOR UPDATE`, so a revocation + that commits first wins. The actor is the account's own `user` actor. A bot + actor is never returned, and a bot never acts as an operator (R1). The + database checks the same rule again on every decision and receipt it + stores. + + This is operator authority only. A panel decision is a different + authority, not a person acting (`Publications`). + + * `:reviewer`: reviews, activates, admits, and publishes or withdraws on + operator authority. The account's `reviewer` flag. + * `:author`: provisions compositions, changes scopes, and writes manual + versions. A reviewer or an internal contributor. + + Call inside a transaction. + """ + + import Ecto.Query + + alias DevilsDictionary.Accounts.User + alias DevilsDictionary.Claims.Contributions + alias DevilsDictionary.Repo + + @doc "The acting account's actor, or `{:error, :unauthorized}`." + def actor(scope, role) when role in [:reviewer, :author] do + with %{user: %{id: id}} <- scope, + %User{} = user <- Repo.one(from u in User, where: u.id == ^id, lock: "FOR UPDATE"), + true <- permitted?(user, role) do + {:ok, Contributions.account_actor!(user)} + else + _ -> {:error, :unauthorized} + end + end + + defp permitted?(%User{reviewer: true}, _role), do: true + defp permitted?(%User{internal_contributor: true}, :author), do: true + defp permitted?(_user, _role), do: false +end diff --git a/lib/devils_dictionary/curation/composition.ex b/lib/devils_dictionary/curation/composition.ex new file mode 100644 index 00000000..4147b75e --- /dev/null +++ b/lib/devils_dictionary/curation/composition.ex @@ -0,0 +1,28 @@ +defmodule DevilsDictionary.Curation.Composition do + @moduledoc """ + A configuration-scoped editorial composition (#196). + + Its identity is `(scope_kind, scope_signature, language_tag, + curation_configuration_id)`. That is the configuration, never its version. + Two configurations over one scope are two compositions, with independent + histories, pointers and clocks (K1). + + `scope_signature` is the SHA-256 of its sorted memberships, never a spelling + or a slug (K2). `current_published_version_id` moves only with a publication + receipt (R3). + """ + use Ecto.Schema + + schema "editorial_compositions" do + field :curation_configuration_id, :id + field :scope_kind, Ecto.Enum, values: [:lexical_page, :lexeme] + field :language_tag, :string + field :scope_signature, :string + field :state, Ecto.Enum, values: [:active, :retired], default: :active + field :current_published_version_id, :id + field :lock_version, :integer, default: 1 + field :created_by_actor_id, :id + + timestamps(type: :utc_datetime_usec) + end +end diff --git a/lib/devils_dictionary/curation/composition_item.ex b/lib/devils_dictionary/curation/composition_item.ex new file mode 100644 index 00000000..7aa560af --- /dev/null +++ b/lib/devils_dictionary/curation/composition_item.ex @@ -0,0 +1,53 @@ +defmodule DevilsDictionary.Curation.CompositionItem do + @moduledoc """ + One lead or highlight of a composition version, as exact references. + + An item stores ids, hashes and locators, never source text (R7): + + * `:content`: a content revision of `item_object_id`; + * `:sense_quotation`: a sense revision, the `quotation:N` locator of one + example, and the SHA-256 of its words; + * `:work`: a source-record revision, or a committed catalog manifest + pinned by name, checksum and row identity. + + When a referenced source row is deleted, the database nulls the reference + and the reader withholds the item. Nothing else about an item ever changes. + + `required_references` is written by the database at insert: the reference + columns the item was made with. It is what tells a claim or object that a + deletion removed apart from one the item never had, so a deletion can + never make a withheld item eligible again. + + A note is attributed to `note_author_actor_id`, the actor who wrote it, + under that actor's own label. The database checks both; a caller never + supplies attribution. + """ + use Ecto.Schema + + schema "editorial_composition_items" do + field :composition_version_id, :id + field :role, Ecto.Enum, values: [:lead, :highlight] + field :position, :integer + field :item_kind, Ecto.Enum, values: [:content, :sense_quotation, :work] + field :item_object_id, :id + field :content_revision_id, :id + field :sense_revision_id, :id + field :source_record_revision_id, :id + field :catalog_manifest, :string + field :catalog_checksum, :string + field :catalog_identity, :string + field :locator, :string + field :words_sha256, :string + field :meaning_sense_revision_id, :id + field :meaning_lexeme_id, :id + field :assertion_revision_id, :id + field :selection_origin, Ecto.Enum, values: [:manual], default: :manual + field :note, :string + field :note_author_kind, Ecto.Enum, values: [:human, :model] + field :note_author_label, :string + field :note_author_actor_id, :id + field :required_references, {:array, :string}, read_after_writes: true + + timestamps(type: :utc_datetime_usec, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/composition_membership.ex b/lib/devils_dictionary/curation/composition_membership.ex new file mode 100644 index 00000000..d35aa677 --- /dev/null +++ b/lib/devils_dictionary/curation/composition_membership.ex @@ -0,0 +1,12 @@ +defmodule DevilsDictionary.Curation.CompositionMembership do + @moduledoc "One registry object in a composition's scope. In this slice, only lexemes." + use Ecto.Schema + + schema "editorial_composition_memberships" do + field :composition_id, :id + field :object_id, :id + field :role, Ecto.Enum, values: [:lexeme], default: :lexeme + + timestamps(type: :utc_datetime_usec, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/composition_publication.ex b/lib/devils_dictionary/curation/composition_publication.ex new file mode 100644 index 00000000..c4152e85 --- /dev/null +++ b/lib/devils_dictionary/curation/composition_publication.ex @@ -0,0 +1,30 @@ +defmodule DevilsDictionary.Curation.CompositionPublication do + @moduledoc """ + The authoritative publication receipt (R3, R4). It records the pointer + moving from one version to another, or to none, in the same transaction, + with the review that authorized it and the fingerprint it published. + Append-only. + + `authority_kind` names what authorized it. In this slice that is always + `:operator`: a reviewer, and for a publication the version's latest review, + an acceptance. The panel-decision authority + (`docs/curation/persistence-slice-1.md`, "Publication authority") needs + decision records that do not exist yet, and the database refuses it. + """ + use Ecto.Schema + + schema "editorial_composition_publications" do + field :composition_id, :id + field :action, Ecto.Enum, values: [:publish, :withdraw] + field :authority_kind, Ecto.Enum, values: [:operator] + field :previous_version_id, :id + field :published_version_id, :id + field :authorizing_review_id, :id + field :actor_id, :id + field :reason, :string + field :eligibility_fingerprint, :string + field :idempotency_key, :string + + timestamps(type: :utc_datetime_usec, inserted_at: :committed_at, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/composition_review.ex b/lib/devils_dictionary/curation/composition_review.ex new file mode 100644 index 00000000..533b78a9 --- /dev/null +++ b/lib/devils_dictionary/curation/composition_review.ex @@ -0,0 +1,26 @@ +defmodule DevilsDictionary.Curation.CompositionReview do + @moduledoc """ + A human presentation decision about one composition version (R1, R2). + Append-only; the latest decision is the version's state. + + Accepting a presentation is not publishing it, and it accepts no claim or + page. It writes no pointer, no receipt, no assertion review and no page row. + """ + use Ecto.Schema + + @decisions [:accepted, :rejected, :withdrawn, :needs_review] + + schema "editorial_composition_reviews" do + field :composition_id, :id + field :composition_version_id, :id + field :reviewer_actor_id, :id + field :decision, Ecto.Enum, values: @decisions + field :reason, :string + field :reviewed_eligibility_fingerprint, :string + field :idempotency_key, :string + + timestamps(type: :utc_datetime_usec, updated_at: false) + end + + def decisions, do: @decisions +end diff --git a/lib/devils_dictionary/curation/composition_version.ex b/lib/devils_dictionary/curation/composition_version.ex new file mode 100644 index 00000000..37ab6280 --- /dev/null +++ b/lib/devils_dictionary/curation/composition_version.ex @@ -0,0 +1,36 @@ +defmodule DevilsDictionary.Curation.CompositionVersion do + @moduledoc """ + One immutable arrangement of a composition (#196): at most one lead and + three highlights, frozen against a configuration version and a scope. + + Manual only in this slice (K9). `origin` is `manual`, the author is a human + account, and there is no run, participant or ballot. The database checks + `arrangement_hash` against the items at commit (K5). + + `eligibility_fingerprint` records the items' eligibility when the version + was made. A review accepts that fingerprint and a publication publishes it + (R4). + """ + use Ecto.Schema + + alias DevilsDictionary.Types.JsonValue + + schema "editorial_composition_versions" do + field :composition_id, :id + field :curation_configuration_id, :id + field :configuration_version_id, :id + field :version, :integer + field :parent_version_id, :id + field :origin, Ecto.Enum, values: [:manual], default: :manual + field :change_reason, :string + field :created_by_actor_id, :id + field :scope_signature, :string + field :scope_members, JsonValue + field :resolution, :map + field :lead_policy, Ecto.Enum, values: [:bierce_first_v1] + field :arrangement_hash, :string + field :eligibility_fingerprint, :string + + timestamps(type: :utc_datetime_usec, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/compositions.ex b/lib/devils_dictionary/curation/compositions.ex new file mode 100644 index 00000000..d9566e8b --- /dev/null +++ b/lib/devils_dictionary/curation/compositions.ex @@ -0,0 +1,448 @@ +defmodule DevilsDictionary.Curation.Compositions do + @moduledoc """ + Configuration-scoped compositions and their manual versions (#196). + + **Identity (K1–K3).** A composition is `(scope_kind, scope_signature, + language_tag, configuration)`. The configuration version is not part of + it. `provision/3` takes registry lexeme ids, never a spelling or a slug. + The signature is the SHA-256 of the sorted members, and the database + recomputes it at commit. + + Provisioning is idempotent for an identical scope. It refuses a scope that + *overlaps* another active composition of the same kind, language and + configuration (`{:overlapping_scope, ids}`): a split or a merge is + reconciled explicitly with `change_scope/4`, never guessed. Both run under + a transaction-scoped advisory lock on (configuration, language), so two + provisions cannot race past the overlap check. + + **Manual versions (K4–K10).** `create_version/3` freezes an arrangement: + + * at most one lead and up to the configuration version's highlight limit; + * each item as exact references (`Eligibility`); + * the Bierce-first rule (`LeadRule`); + * the author, their reason, and the parent version it replaces. + + A version gets no run, no participant and no ballot: it is manual work and + says so. Writing a version publishes nothing. + """ + + import Ecto.Query + + alias DevilsDictionary.Curation.{ + Authority, + Composition, + CompositionItem, + CompositionMembership, + CompositionVersion, + Configurations, + Digest, + Eligibility, + ScopeChange, + Transaction + } + + alias DevilsDictionary.Registry.Lexeme + alias DevilsDictionary.Repo + + # ── identity ────────────────────────────────────────────────────────────── + + @doc """ + Provisions the composition for a scope under an enabled configuration, as + an author. `attrs`: + + * `:scope_kind`: `:lexeme` (exactly one member) or `:lexical_page`; + * `:lexeme_ids`: registry lexeme object ids, all in `:language_tag`; + * `:language_tag`; + * `:reason`. + + Answers the existing composition for an identical scope. + """ + def provision(scope, configuration_id, attrs) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :author), + {:ok, reason} <- Transaction.required(Map.to_list(attrs), :reason), + {:ok, configuration, _version} <- Configurations.current(configuration_id), + {:ok, ids} <- members(attrs.scope_kind, attrs.lexeme_ids, attrs.language_tag) do + lock_scope_space(configuration.id, attrs.language_tag) + signature = signature(ids) + + case active(configuration.id, attrs.scope_kind, attrs.language_tag, signature) do + %Composition{} = existing -> + {:ok, existing} + + nil -> + with :ok <- no_overlap(configuration.id, attrs, ids, nil) do + composition = + Repo.insert!(%Composition{ + curation_configuration_id: configuration.id, + scope_kind: attrs.scope_kind, + language_tag: attrs.language_tag, + scope_signature: signature, + created_by_actor_id: actor.id + }) + + put_members!(composition, [], ids) + scope_receipt!(composition, nil, signature, ids, actor, reason) + {:ok, composition} + end + end + end + end) + end + + @doc """ + Changes a composition's scope to `lexeme_ids`, as an author, with a + scope-change receipt. Existing versions keep the scope they were made + against, so a published version whose scope has since changed is withheld + on read (`:scope_changed`) until a version is made for the new scope and + published. + """ + def change_scope(scope, composition_id, lexeme_ids, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :author), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, c} <- lock_active(composition_id), + {:ok, ids} <- members(c.scope_kind, lexeme_ids, c.language_tag) do + lock_scope_space(c.curation_configuration_id, c.language_tag) + signature = signature(ids) + + with :ok <- Transaction.check(signature != c.scope_signature, :scope_unchanged), + :ok <- + no_overlap( + c.curation_configuration_id, + %{scope_kind: c.scope_kind, language_tag: c.language_tag}, + ids, + c.id + ) do + put_members!(c, member_ids(c.id), ids) + scope_receipt!(c, c.scope_signature, signature, ids, actor, reason) + + c + |> Ecto.Changeset.change(scope_signature: signature, lock_version: c.lock_version + 1) + |> Repo.update() + end + end + end) + end + + @doc "The member lexeme ids of a composition, sorted." + def member_ids(composition_id) do + Repo.all( + from m in CompositionMembership, + where: m.composition_id == ^composition_id, + order_by: m.object_id, + select: m.object_id + ) + end + + @doc "The active composition for a scope identity, or `nil`." + def active(configuration_id, scope_kind, language_tag, signature) do + Repo.one( + from c in Composition, + where: + c.curation_configuration_id == ^configuration_id and c.scope_kind == ^scope_kind and + c.language_tag == ^language_tag and c.scope_signature == ^signature and + c.state == :active + ) + end + + @doc "The signature of a scope of lexeme ids." + def signature(lexeme_ids), do: Digest.scope_signature(Enum.map(lexeme_ids, &{:lexeme, &1})) + + defp members(kind, ids, language) when is_list(ids) do + ids = ids |> Enum.uniq() |> Enum.sort() + + found = + Repo.all( + from l in Lexeme, where: l.object_id in ^ids, select: {l.object_id, l.language_tag} + ) + + cond do + ids == [] -> + {:error, :empty_scope} + + kind == :lexeme and length(ids) != 1 -> + {:error, :lexeme_scope_has_one_member} + + length(found) != length(ids) -> + {:error, {:not_lexemes, ids -- Enum.map(found, &elem(&1, 0))}} + + Enum.any?(found, &(elem(&1, 1) != language)) -> + {:error, :language_mismatch} + + true -> + {:ok, ids} + end + end + + defp members(_kind, _ids, _language), do: {:error, :empty_scope} + + defp no_overlap(configuration_id, attrs, ids, self_id) do + query = + from m in CompositionMembership, + join: c in Composition, + on: c.id == m.composition_id, + where: + c.curation_configuration_id == ^configuration_id and + c.scope_kind == ^attrs.scope_kind and c.language_tag == ^attrs.language_tag and + c.state == :active and m.object_id in ^ids, + distinct: true, + select: c.id + + query = if self_id, do: where(query, [_m, c], c.id != ^self_id), else: query + overlapping = Repo.all(query) + + Transaction.check(overlapping == [], {:overlapping_scope, Enum.sort(overlapping)}) + end + + defp put_members!(composition, current, wanted) do + removed = current -- wanted + + if removed != [] do + Repo.delete_all( + from m in CompositionMembership, + where: m.composition_id == ^composition.id and m.object_id in ^removed + ) + end + + for id <- wanted -- current do + Repo.insert!(%CompositionMembership{composition_id: composition.id, object_id: id}) + end + end + + defp scope_receipt!(composition, previous, signature, ids, actor, reason) do + Repo.insert!(%ScopeChange{ + composition_id: composition.id, + previous_signature: previous, + scope_signature: signature, + members: Enum.map(ids, &["lexeme", &1]), + actor_id: actor.id, + reason: reason + }) + end + + defp lock_scope_space(configuration_id, language) do + Repo.query!("SELECT pg_advisory_xact_lock(hashtext($1))", [ + "curation-scope:#{configuration_id}:#{language}" + ]) + end + + # ── versions ────────────────────────────────────────────────────────────── + + @doc """ + Writes a manual version of a composition, as an author. `attrs`: + + * `:lead`: one item spec, or `nil`; + * `:highlights`: a list of item specs, in order; + * `:reason`; + * `:expected_parent`: the latest version id the author saw, or `nil`. + Required, so two authors cannot both write "version 2". + + An item spec is a map with `:kind` (`:content`, `:sense_quotation` or + `:work`), `:object_id`, its exact reference, and `:meaning` + (`{:lexeme, id}` or `{:sense_revision, id}`): + + * `:content`: `:content_revision_id`; + * `:sense_quotation`: `:sense_revision_id`, `:locator` + (`"quotation:N"`), `:words_sha256`; + * `:work`: `:source_record_revision_id`, or `:catalog` as + `%{manifest, checksum, identity}`. + + An item may also carry `:assertion_revision_id`, and `:note` as `%{text: + text}`: the author's own words, attributed to the authenticated actor. A + note spec naming its own author or kind is refused + (`:note_attribution_not_an_input`). + + The same arrangement under the same configuration version is refused as + `{:duplicate_version, id}`. Under a newer configuration version it is a new + version, with no approval of its own until it is reviewed. + """ + def create_version(scope, composition_id, attrs) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :author), + {:ok, reason} <- Transaction.required(Map.to_list(attrs), :reason), + {:ok, c} <- lock_active(composition_id), + {:ok, _configuration, cv} <- Configurations.current(c.curation_configuration_id), + latest = latest_version(c.id), + :ok <- expected_parent(latest, Map.fetch!(attrs, :expected_parent)), + highlights = Map.get(attrs, :highlights, []), + :ok <- + Transaction.check( + length(highlights) <= cv.max_highlights, + {:too_many_highlights, cv.max_highlights} + ), + {:ok, items} <- build_items(Map.get(attrs, :lead), highlights, actor) do + members = member_ids(c.id) + evaluation = Eligibility.evaluate(items, members, c.scope_signature, cv.id) + arrangement = Digest.arrangement_hash(items) + + with {:ok, rule} <- evaluation.lead_rule, + :ok <- eligible(evaluation), + :ok <- not_duplicate(c.id, cv.id, arrangement, evaluation.fingerprint) do + version = + Repo.insert!(%CompositionVersion{ + composition_id: c.id, + curation_configuration_id: c.curation_configuration_id, + configuration_version_id: cv.id, + version: if(latest, do: latest.version + 1, else: 1), + parent_version_id: latest && latest.id, + change_reason: reason, + created_by_actor_id: actor.id, + scope_signature: c.scope_signature, + scope_members: Enum.map(members, &["lexeme", &1]), + resolution: %{ + "lead_rule" => to_string(rule), + "priority_leads" => evaluation.applicable, + "language_tag" => c.language_tag, + "configuration_version" => cv.version + }, + lead_policy: cv.lead_policy, + arrangement_hash: arrangement, + eligibility_fingerprint: evaluation.fingerprint + }) + + for item <- items, do: Repo.insert!(%{item | composition_version_id: version.id}) + + {:ok, version} + end + end + end) + end + + @doc "A version's items, in `(role, position)` order." + def items(version_id) do + Repo.all( + from i in CompositionItem, + where: i.composition_version_id == ^version_id, + order_by: [i.role, i.position] + ) + end + + @doc "The newest version of a composition, or `nil`." + def latest_version(composition_id) do + Repo.one( + from v in CompositionVersion, + where: v.composition_id == ^composition_id, + order_by: [desc: v.version], + limit: 1 + ) + end + + defp expected_parent(nil, nil), do: :ok + defp expected_parent(%CompositionVersion{id: id}, id), do: :ok + defp expected_parent(latest, _seen), do: {:error, {:stale_parent, latest && latest.id}} + + defp eligible(%{ok?: true}), do: :ok + + defp eligible(%{results: results}), + do: {:error, {:ineligible, Enum.reject(results, &(elem(&1, 2) == :ok))}} + + # The same arrangement under the same configuration version is the same + # version. Under a newer configuration version it is a new one, which needs + # its own review: approval never transfers. + defp not_duplicate(composition_id, configuration_version_id, arrangement, fingerprint) do + case Repo.one( + from v in CompositionVersion, + where: + v.composition_id == ^composition_id and + v.configuration_version_id == ^configuration_version_id and + v.arrangement_hash == ^arrangement and v.eligibility_fingerprint == ^fingerprint, + select: v.id + ) do + nil -> :ok + id -> {:error, {:duplicate_version, id}} + end + end + + defp build_items(lead, highlights, actor) do + specs = + if(lead, do: [{:lead, 1, lead}], else: []) ++ + (highlights |> Enum.with_index(1) |> Enum.map(fn {spec, n} -> {:highlight, n, spec} end)) + + Enum.reduce_while(specs, {:ok, []}, fn {role, position, spec}, {:ok, acc} -> + case item(role, position, spec, actor) do + {:ok, item} -> {:cont, {:ok, acc ++ [item]}} + {:error, reason} -> {:halt, {:error, {reason, role, position}}} + end + end) + end + + defp item(role, position, %{kind: kind} = spec, actor) + when kind in [:content, :sense_quotation, :work] do + with {:ok, meaning} <- meaning(spec[:meaning]), + {:ok, note} <- note(spec[:note], actor) do + catalog = spec[:catalog] || %{} + + {:ok, + struct( + CompositionItem, + %{ + role: role, + position: position, + item_kind: kind, + item_object_id: spec[:object_id], + content_revision_id: spec[:content_revision_id], + sense_revision_id: spec[:sense_revision_id], + source_record_revision_id: spec[:source_record_revision_id], + catalog_manifest: catalog[:manifest], + catalog_checksum: catalog[:checksum], + catalog_identity: catalog[:identity] && to_string(catalog[:identity]), + locator: spec[:locator], + words_sha256: spec[:words_sha256], + assertion_revision_id: spec[:assertion_revision_id], + selection_origin: :manual + } + |> Map.merge(meaning) + |> Map.merge(note) + )} + end + end + + defp item(_role, _position, _spec, _actor), do: {:error, :unknown_item_kind} + + defp meaning({:lexeme, id}) when is_integer(id), + do: {:ok, %{meaning_lexeme_id: id, meaning_sense_revision_id: nil}} + + defp meaning({:sense_revision, id}) when is_integer(id), + do: {:ok, %{meaning_sense_revision_id: id, meaning_lexeme_id: nil}} + + defp meaning(_meaning), do: {:error, :meaning_required} + + # A manual version's note is its author's own words, attributed to the + # authenticated actor and nobody else. A note spec is its text alone: + # attribution is never an input, so a caller naming an author, or a kind, + # is refused rather than quietly overridden. A model note has no place in + # manual work. The database checks the same (`note_author_actor_id`). + defp note(nil, _actor), + do: + {:ok, + %{note: nil, note_author_kind: nil, note_author_label: nil, note_author_actor_id: nil}} + + defp note(%{text: text} = note, actor) when is_binary(text) and map_size(note) == 1 do + if String.trim(text) == "" do + {:error, :note_empty} + else + {:ok, + %{ + note: text, + note_author_kind: :human, + note_author_label: author_label(actor), + note_author_actor_id: actor.id + }} + end + end + + defp note(%{text: _text}, _actor), do: {:error, :note_attribution_not_an_input} + defp note(_note, _actor), do: {:error, :note_invalid} + + @doc "The label an actor's notes are attributed under: its own, or its account number." + def author_label(actor), do: actor.label || "Account ##{actor.id}" + + defp lock_active(id) do + case Repo.one(from c in Composition, where: c.id == ^id, lock: "FOR UPDATE") do + nil -> {:error, :not_found} + %Composition{state: :active} = c -> {:ok, c} + %Composition{} -> {:error, :composition_retired} + end + end +end diff --git a/lib/devils_dictionary/curation/configuration.ex b/lib/devils_dictionary/curation/configuration.ex new file mode 100644 index 00000000..e55e89e7 --- /dev/null +++ b/lib/devils_dictionary/curation/configuration.ex @@ -0,0 +1,33 @@ +defmodule DevilsDictionary.Curation.Configuration do + @moduledoc """ + A system-owned curation configuration identity (#201). + + The identity is stable. What it means at any moment is its current + immutable version, and the pointer moves only with an activation receipt + (C3). `role` separates the one global default from internal test + configurations; there is no domain or personal ownership yet. + + `state` is `draft` until a reviewer activates a version, then `enabled` or + `disabled`. The database checks both halves: a draft has no current + version, and an enabled configuration always has one. + """ + use Ecto.Schema + + @roles [:global_default, :internal_test] + @states [:draft, :enabled, :disabled] + + schema "curation_configurations" do + field :slug, :string + field :name, :string + field :ownership_kind, Ecto.Enum, values: [:system], default: :system + field :role, Ecto.Enum, values: @roles + field :state, Ecto.Enum, values: @states, default: :draft + field :current_version_id, :id + field :created_by_actor_id, :id + + timestamps(type: :utc_datetime_usec) + end + + def roles, do: @roles + def states, do: @states +end diff --git a/lib/devils_dictionary/curation/configuration_activation.ex b/lib/devils_dictionary/curation/configuration_activation.ex new file mode 100644 index 00000000..28a7c7df --- /dev/null +++ b/lib/devils_dictionary/curation/configuration_activation.ex @@ -0,0 +1,22 @@ +defmodule DevilsDictionary.Curation.ConfigurationActivation do + @moduledoc """ + The receipt for a configuration's pointer or state moving (C3, C8). It + records who activated or disabled which version, from which, and why. + + Append-only. The database checks at commit that the configuration matches + its latest receipt, and that each receipt continues from the one before it. + """ + use Ecto.Schema + + schema "curation_configuration_activations" do + field :configuration_id, :id + field :action, Ecto.Enum, values: [:activate, :disable] + field :configuration_version_id, :id + field :previous_version_id, :id + field :actor_id, :id + field :reason, :string + field :idempotency_key, :string + + timestamps(type: :utc_datetime_usec, inserted_at: :committed_at, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/configuration_member.ex b/lib/devils_dictionary/curation/configuration_member.ex new file mode 100644 index 00000000..fca68707 --- /dev/null +++ b/lib/devils_dictionary/curation/configuration_member.ex @@ -0,0 +1,17 @@ +defmodule DevilsDictionary.Curation.ConfigurationMember do + @moduledoc """ + One seat on a configuration version's roster: an exact profile version, a + slot, and weight 1 (C5). + """ + use Ecto.Schema + + schema "curation_configuration_members" do + field :configuration_version_id, :id + field :profile_id, :id + field :profile_version_id, :id + field :slot, :integer + field :voting_weight, :integer, default: 1 + + timestamps(type: :utc_datetime_usec, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/configuration_version.ex b/lib/devils_dictionary/curation/configuration_version.ex new file mode 100644 index 00000000..f27d8cc4 --- /dev/null +++ b/lib/devils_dictionary/curation/configuration_version.ex @@ -0,0 +1,25 @@ +defmodule DevilsDictionary.Curation.ConfigurationVersion do + @moduledoc """ + One immutable version of a configuration (#201). + + In this slice every version is **manual-only**. It carries a lead policy + and a highlight limit, and no model configuration, because none has been + validated (#195). `roster_hash` is the digest of its members, which the + database checks at commit, so a roster cannot be extended later (C4). + """ + use Ecto.Schema + + schema "curation_configuration_versions" do + field :configuration_id, :id + field :version, :integer + field :manifest_hash, :string + field :manifest, :map + field :lead_policy, Ecto.Enum, values: [:bierce_first_v1] + field :max_highlights, :integer, default: 3 + field :roster_hash, :string + field :created_by_actor_id, :id + field :change_reason, :string + + timestamps(type: :utc_datetime_usec, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/configurations.ex b/lib/devils_dictionary/curation/configurations.ex new file mode 100644 index 00000000..5774d7b4 --- /dev/null +++ b/lib/devils_dictionary/curation/configurations.ex @@ -0,0 +1,390 @@ +defmodule DevilsDictionary.Curation.Configurations do + @moduledoc """ + System-owned curation configurations (#201): identities, immutable + versions, activation, and resolution of the global default. + + Each rule below is enforced here and checked again by the database: + + * **Only `system` ownership exists**, and at most one global default is + enabled at a time (C1). + * **A pointer moves only with a receipt.** `activate/4` and `disable/3` + are a reviewer's audited act, each writing an activation receipt in the + pointer's transaction. The database checks at commit that the + configuration matches its latest receipt (C3). Neither publishes + anything (C8). + * **Resolution is server-side and default-only** (C7). + `resolve_default/1` takes no configuration argument. It answers the + enabled global default and its version, or an explicit `{:unavailable, + reason}` (`:missing`, `:no_active_version`, `:disabled`, `:unready`, + `:panel_not_available`). It never answers another configuration in the + default's place. + * **Seeding activates nothing.** `seed!/0` creates `global-default` as a + draft, with version 1 and no current version. That version is + manual-only: the Bierce-first lead policy, three highlights, and an + empty roster. Until a reviewer activates it, the default resolves as + `{:unavailable, :no_active_version}`. + """ + + import Ecto.Query + + alias DevilsDictionary.Curation.{ + Authority, + Configuration, + ConfigurationActivation, + ConfigurationMember, + ConfigurationVersion, + Digest, + Profile, + Profiles, + Transaction + } + + alias DevilsDictionary.Repo + alias DevilsDictionary.Sources.Actor + + @default_slug "global-default" + @seed_actor "Curation seed" + @mode "manual" + + def default_slug, do: @default_slug + + # ── seeding ─────────────────────────────────────────────────────────────── + + @doc """ + Seeds the `global-default` identity with its first version, and the five + proposed profiles. Idempotent, and never changes an existing row's state: + a configuration someone has activated stays activated. + """ + def seed! do + {:ok, result} = + Repo.transaction(fn -> + actor = seed_actor!() + profiles = Profiles.seed_proposed!() + + configuration = + Repo.get_by(Configuration, slug: @default_slug) || + Repo.insert!(%Configuration{ + slug: @default_slug, + name: "Global default", + role: :global_default, + state: :draft, + created_by_actor_id: actor.id + }) + + version = + Repo.get_by(ConfigurationVersion, configuration_id: configuration.id, version: 1) || + insert_version!(configuration, actor.id, %{ + reason: "Initial manual-only default: Bierce-first lead, up to three highlights." + }) + + %{configuration: configuration, version: version, profiles: profiles} + end) + + result + end + + defp seed_actor! do + Repo.get_by(Actor, actor_kind: :import, label: @seed_actor) || + Repo.insert!( + Actor.changeset(%Actor{}, %{ + actor_kind: :import, + label: @seed_actor, + metadata: %{"operation" => "curation_seed", "accountability" => "creates drafts only"} + }) + ) + end + + # ── identities and versions ─────────────────────────────────────────────── + + @doc """ + Creates an internal test configuration, as a draft. The global default is + created only by `seed!/0`. + """ + def create_test_configuration(scope, slug, name) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer) do + %Configuration{ + slug: slug, + name: name, + role: :internal_test, + state: :draft, + created_by_actor_id: actor.id + } + |> Ecto.Changeset.change() + |> Ecto.Changeset.unique_constraint(:slug) + |> Repo.insert() + end + end) + end + + @doc """ + Adds an immutable, manual-only version to a configuration, as a reviewer. + + `attrs`: `:reason`, and optionally `:max_highlights` (0–3, default 3) and + `:members`, a list of `%{profile_id, profile_version_id}`. Each member must + be an admitted profile's admitted version. Slots follow list order. The + version is not activated. + """ + def create_version(scope, configuration_id, attrs) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + {:ok, configuration} <- lock(configuration_id), + {:ok, _reason} <- Transaction.required(Map.to_list(attrs), :reason), + :ok <- admitted_members(Map.get(attrs, :members, [])) do + {members, manifest} = manifest(attrs) + + case Repo.get_by(ConfigurationVersion, + configuration_id: configuration.id, + manifest_hash: Digest.term(manifest) + ) do + nil -> {:ok, insert_version!(configuration, actor.id, attrs, members, manifest)} + %ConfigurationVersion{id: id} -> {:error, {:duplicate_version, id}} + end + end + end) + end + + defp admitted_members(members) do + admitted = + Enum.all?(members, fn %{profile_id: pid, profile_version_id: vid} -> + Repo.exists?( + from p in Profile, + where: p.id == ^pid and p.state == :admitted and p.current_version_id == ^vid + ) + end) + + Transaction.check(admitted, :profile_not_admitted) + end + + # What a version is: its roster seats and the manifest they are hashed into. + defp manifest(attrs) do + members = + attrs + |> Map.get(:members, []) + |> Enum.with_index(1) + |> Enum.map(fn {m, slot} -> + %{ + profile_id: m.profile_id, + profile_version_id: m.profile_version_id, + slot: slot, + voting_weight: 1 + } + end) + + manifest = %{ + "mode" => @mode, + "lead_policy" => "bierce_first_v1", + "max_highlights" => Map.get(attrs, :max_highlights, 3), + "roster" => Enum.map(members, &Map.new(&1, fn {k, v} -> {to_string(k), v} end)) + } + + {members, manifest} + end + + defp insert_version!(configuration, actor_id, attrs) do + {members, manifest} = manifest(attrs) + insert_version!(configuration, actor_id, attrs, members, manifest) + end + + defp insert_version!(configuration, actor_id, attrs, members, manifest) do + max = manifest["max_highlights"] + + version = + Repo.insert!(%ConfigurationVersion{ + configuration_id: configuration.id, + version: next_version(configuration.id), + manifest_hash: Digest.term(manifest), + manifest: manifest, + lead_policy: :bierce_first_v1, + max_highlights: max, + roster_hash: Digest.roster_hash(members), + created_by_actor_id: actor_id, + change_reason: attrs.reason + }) + + for m <- members do + Repo.insert!(struct(ConfigurationMember, Map.put(m, :configuration_version_id, version.id))) + end + + version + end + + defp next_version(configuration_id) do + (Repo.one( + from v in ConfigurationVersion, + where: v.configuration_id == ^configuration_id, + select: max(v.version) + ) || 0) + 1 + end + + # ── resolution ──────────────────────────────────────────────────────────── + + @doc """ + The global default and its current version, as `{:ok, %{configuration, + version}}`, or `{:unavailable, reason}`. + + `purpose: :manual` (the default) is what this slice supports. + `purpose: :panel` is always unavailable: no model configuration exists + (#195), and none is invented to make it answer. + """ + def resolve_default(opts \\ []) do + case Keyword.get(opts, :purpose, :manual) do + :panel -> {:unavailable, :panel_not_available} + :manual -> resolve_manual() + end + end + + defp resolve_manual do + defaults = Repo.all(from c in Configuration, where: c.role == :global_default) + + case Enum.find(defaults, &(&1.state == :enabled)) do + %Configuration{} = configuration -> + version = Repo.get!(ConfigurationVersion, configuration.current_version_id) + + if ready?(version), + do: {:ok, %{configuration: configuration, version: version}}, + else: {:unavailable, :unready} + + nil -> + cond do + defaults == [] -> {:unavailable, :missing} + Enum.any?(defaults, &(&1.state == :disabled)) -> {:unavailable, :disabled} + true -> {:unavailable, :no_active_version} + end + end + end + + @doc "Whether this build can compose under a configuration version: manual mode, a known lead policy." + def ready?(%ConfigurationVersion{manifest: manifest, lead_policy: :bierce_first_v1}), + do: manifest["mode"] == @mode + + def ready?(_version), do: false + + @doc """ + The configuration and its current version, if it is enabled and ready, as + `{:ok, configuration, version}`. Otherwise `{:error, + :configuration_unavailable}`. + """ + def current(configuration_id) do + with %Configuration{state: :enabled} = c <- Repo.get(Configuration, configuration_id), + %ConfigurationVersion{} = v <- Repo.get(ConfigurationVersion, c.current_version_id), + true <- ready?(v) do + {:ok, c, v} + else + _ -> {:error, :configuration_unavailable} + end + end + + # ── activation ──────────────────────────────────────────────────────────── + + @doc """ + Makes `version_id` a configuration's current version and enables it, as a + reviewer. + + `opts`: + + * `:reason`; + * `:idempotency_key`; + * `:expected`: the current version id the caller saw, or `nil`. + Required, so a stale screen cannot move a pointer it did not see. + + A replay of the same key answers the same receipt. Publishes nothing. + """ + def activate(scope, configuration_id, version_id, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, key} <- Transaction.required(opts, :idempotency_key), + {:ok, c} <- lock(configuration_id), + :fresh <- + Transaction.replay(ConfigurationActivation, key, fn r -> + r.configuration_id == c.id and r.action == :activate and + r.configuration_version_id == version_id + end), + {:ok, version} <- + Transaction.need( + Repo.get_by(ConfigurationVersion, id: version_id, configuration_id: c.id), + :version_not_found + ), + :ok <- Transaction.check(ready?(version), :unready), + :ok <- expected(c, Keyword.fetch!(opts, :expected)), + :ok <- + Transaction.check( + not (c.state == :enabled and c.current_version_id == version.id), + :already_active + ), + :ok <- sole_default(c) do + receipt(c, :activate, version.id, actor, reason, key, :enabled) + else + {:replay, row} -> {:ok, row} + error -> error + end + end) + end + + @doc """ + Disables an enabled configuration, as a reviewer, with a receipt. `opts` as + for `activate/4`. The pointer stays where it was, and resolution then + answers `{:unavailable, :disabled}`. + """ + def disable(scope, configuration_id, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, key} <- Transaction.required(opts, :idempotency_key), + {:ok, c} <- lock(configuration_id), + :fresh <- + Transaction.replay(ConfigurationActivation, key, fn r -> + r.configuration_id == c.id and r.action == :disable + end), + :ok <- Transaction.check(c.state == :enabled, :not_enabled), + :ok <- expected(c, Keyword.fetch!(opts, :expected)) do + receipt(c, :disable, c.current_version_id, actor, reason, key, :disabled) + else + {:replay, row} -> {:ok, row} + error -> error + end + end) + end + + defp receipt(c, action, version_id, actor, reason, key, state) do + activation = + Repo.insert!(%ConfigurationActivation{ + configuration_id: c.id, + action: action, + configuration_version_id: version_id, + previous_version_id: c.current_version_id, + actor_id: actor.id, + reason: reason, + idempotency_key: key + }) + + c |> Ecto.Changeset.change(state: state, current_version_id: version_id) |> Repo.update!() + + {:ok, activation} + end + + defp expected(%Configuration{current_version_id: current}, current), do: :ok + + defp expected(%Configuration{current_version_id: current}, _seen), + do: {:error, {:stale, current}} + + defp sole_default(%Configuration{role: :global_default, id: id}) do + Transaction.check( + not Repo.exists?( + from c in Configuration, + where: c.role == :global_default and c.state == :enabled and c.id != ^id + ), + :another_default_enabled + ) + end + + defp sole_default(_configuration), do: :ok + + defp lock(id) do + Transaction.need( + Repo.one(from c in Configuration, where: c.id == ^id, lock: "FOR UPDATE"), + :not_found + ) + end +end diff --git a/lib/devils_dictionary/curation/digest.ex b/lib/devils_dictionary/curation/digest.ex new file mode 100644 index 00000000..c2c5f304 --- /dev/null +++ b/lib/devils_dictionary/curation/digest.ex @@ -0,0 +1,81 @@ +defmodule DevilsDictionary.Curation.Digest do + @moduledoc """ + The digests curation records are identified and checked by. + + Three of them are recomputed by the database at COMMIT, so the functions + here and the SQL functions in the curation migration must agree byte for + byte. They do not need a test of their own: a disagreement fails the insert. + + * `scope_signature/1`: `editorial_scope_signature(composition)`, the + sorted `role:object_id` members joined by `,`; + * `roster_hash/1`: `curation_roster_hash(version)`, the + `slot:profile:profile_version:weight` seats in slot order; + * `arrangement_hash/1`: `editorial_arrangement_hash(version)`, every item + field length-prefixed (`~` for NULL), in `(role, position)` order. + + `term/1` digests a canonical JSON encoding, with keys sorted at every + depth, of a manifest or an eligibility record. The database never + recomputes it. + """ + + alias DevilsDictionary.Curation.CompositionItem + + @doc "SHA-256, lower-case hex." + def sha256(binary) when is_binary(binary), + do: :sha256 |> :crypto.hash(binary) |> Base.encode16(case: :lower) + + @doc "The signature of a scope, from its `{role, object_id}` members." + def scope_signature(members) do + members + |> Enum.map(fn {role, id} when is_integer(id) -> {to_string(role), id} end) + |> Enum.sort() + |> Enum.map_join(",", fn {role, id} -> "#{role}:#{id}" end) + |> sha256() + end + + @doc "The digest of a roster: maps with `slot`, `profile_id`, `profile_version_id` and `voting_weight`." + def roster_hash(members) do + members + |> Enum.sort_by(& &1.slot) + |> Enum.map_join(",", fn m -> + "#{m.slot}:#{m.profile_id}:#{m.profile_version_id}:#{m.voting_weight}" + end) + |> sha256() + end + + @arrangement_fields ~w(role position item_kind item_object_id content_revision_id + sense_revision_id source_record_revision_id catalog_manifest + catalog_checksum catalog_identity locator words_sha256 + meaning_sense_revision_id meaning_lexeme_id assertion_revision_id + selection_origin note note_author_kind note_author_label)a + + @doc "The digest of a version's items, as `%CompositionItem{}` structs." + def arrangement_hash(items) do + items + |> Enum.sort_by(&{to_string(&1.role), &1.position}) + |> Enum.map_join(fn %CompositionItem{} = item -> + Enum.map_join(@arrangement_fields, &field(Map.fetch!(item, &1))) + end) + |> sha256() + end + + defp field(nil), do: "~" + defp field(value) when is_atom(value), do: field(Atom.to_string(value)) + defp field(value) when is_integer(value), do: field(Integer.to_string(value)) + defp field(value) when is_binary(value), do: "#{byte_size(value)}:#{value}" + + @doc "The digest of a JSON-shaped term, canonically encoded." + def term(value), do: value |> canonical() |> Jason.encode!() |> sha256() + + defp canonical(%{__struct__: _} = struct), do: struct + + defp canonical(%{} = map) do + map + |> Enum.map(fn {key, value} -> {to_string(key), canonical(value)} end) + |> Enum.sort_by(&elem(&1, 0)) + |> Jason.OrderedObject.new() + end + + defp canonical(list) when is_list(list), do: Enum.map(list, &canonical/1) + defp canonical(value), do: value +end diff --git a/lib/devils_dictionary/curation/eligibility.ex b/lib/devils_dictionary/curation/eligibility.ex new file mode 100644 index 00000000..d6282175 --- /dev/null +++ b/lib/devils_dictionary/curation/eligibility.ex @@ -0,0 +1,386 @@ +defmodule DevilsDictionary.Curation.Eligibility do + @moduledoc """ + Whether each item of an arrangement may be shown now, from the registry as + it stands. It answers `:ok` or the reason it may not, and never a + replacement (R6). + + An item is eligible when: + + * **every reference it was made with still exists.** A deletion nulls a + reference (R7), and the item records which references it had + (`required_references`), so a deleted revision, object, meaning or claim + withholds it (`:revision_deleted`, `:object_deleted`, + `:meaning_deleted`, `:claim_deleted`). A deletion never makes an item + eligible again; + * **its exact revision is the current, active one of its own object.** A + newer revision is `:revision_superseded`; a pin is a pin; + * **display is allowed** at every layer: the revision's rights + (`Claims.Visibility`), its source record's `display_allowed`, and its + source being active; + * **for a quotation**, the example at its locator still hashes to the + words that were chosen (`:words_changed`); + * **for a work, its evidence is of that work.** + - A catalog pin must name a committed manifest with the pinned checksum + and row (`:catalog_changed`, `:catalog_row_missing`). If the item + also names a registry object, that object must carry the row's + identity as a verified external identifier. + - A source-record revision must be one whose record materialized the + item's object. + - Anything else is `:work_identity_mismatch`. A catalog pin with no + object is a legitimate catalog-only work; + * **its intended meaning is on the scope**: a member lexeme, or a + current sense of one; + * **its claim, if it was made with one**, is current, active and + publicly visible (`Claims.visible/2`). This slice writes no claim; it + reads the state #190's review workflow leaves. + + `evaluate/4` also applies the lead rule and returns the **eligibility + fingerprint**. That is the digest of: + + * the configuration version; + * the scope; + * the applicable Bierce entries; + * the lead rule and every item's result. + + A review accepts a fingerprint, and a publication publishes the one that + was accepted. Anything that changes an item's eligibility, the lead rule's + inputs or the configuration version changes it (R4). + """ + + import Ecto.Query + + alias DevilsDictionary.Artworks.Corpus.Manifest + alias DevilsDictionary.Claims + alias DevilsDictionary.Claims.{AssertionRevision, Visibility} + alias DevilsDictionary.Corpus.SourceRecordRevision + alias DevilsDictionary.Curation.{CompositionItem, Digest, LeadRule} + + alias DevilsDictionary.Registry.{ + ContentItem, + ContentRevision, + ExternalIdentifier, + Object, + Sense, + SenseRevision + } + + alias DevilsDictionary.Repo + alias DevilsDictionary.Sources.{MaterializedOutput, Source, SourceRecord} + + # The references a deletion can null, and what each one's loss means. + # `meaning_lexeme_id` is RESTRICT, so it is never nulled. + @references %{ + "item_object_id" => {:item_object_id, :object_deleted}, + "content_revision_id" => {:content_revision_id, :revision_deleted}, + "sense_revision_id" => {:sense_revision_id, :revision_deleted}, + "source_record_revision_id" => {:source_record_revision_id, :revision_deleted}, + "meaning_sense_revision_id" => {:meaning_sense_revision_id, :meaning_deleted}, + "assertion_revision_id" => {:assertion_revision_id, :claim_deleted} + } + + @doc """ + Evaluates an arrangement against a scope's member lexeme ids, under a + configuration version. Returns `%{results: [{role, position, :ok | + reason}], lead_rule: {:ok, rule} | {:error, reason}, applicable: [id], + fingerprint: hex, ok?: boolean}`. + """ + def evaluate(items, member_ids, scope_signature, configuration_version_id) do + applicable = LeadRule.applicable(member_ids) + lead = Enum.find(items, &(&1.role == :lead)) + lead_rule = LeadRule.check(member_ids, lead && lead.item_object_id, applicable) + + results = + items + |> Enum.sort_by(&{to_string(&1.role), &1.position}) + |> Enum.map(&{&1.role, &1.position, status(check(&1, member_ids))}) + + %{ + results: results, + lead_rule: lead_rule, + applicable: applicable, + ok?: match?({:ok, _}, lead_rule) and Enum.all?(results, &(elem(&1, 2) == :ok)), + fingerprint: + Digest.term(%{ + "configuration_version" => configuration_version_id, + "scope" => scope_signature, + "priority_leads" => applicable, + "lead_rule" => lead_rule |> elem(1) |> to_string(), + "items" => + Enum.map(results, fn {role, pos, st} -> [to_string(role), pos, to_string(st)] end) + }) + } + end + + defp status(:ok), do: :ok + defp status({:error, reason}), do: reason + + @doc "`:ok`, or `{:error, reason}` for one item on a scope of member lexeme ids." + def check(%CompositionItem{} = item, member_ids) do + with :ok <- still_present(item), + :ok <- reference(item), + :ok <- meaning(item, member_ids) do + assertion(item.assertion_revision_id) + end + end + + @doc """ + Whether content object `content_id` may lead now. Its current revision + must be active and displayable, its object and source active, and its + source record allowing display. A priority source's entry that fails this + neither leads nor blocks another lead (K10). + """ + def leadable?(content_id) do + case Repo.one(from r in ContentRevision, where: r.content_id == ^content_id and r.is_current) do + nil -> false + revision -> content_revision(revision, content_id) == :ok + end + end + + # A reference an item was made with and a deletion has since nulled. An + # item not yet stored has exactly the references it names. + defp still_present(%CompositionItem{required_references: nil}), do: :ok + + defp still_present(%CompositionItem{required_references: required} = item) do + Enum.find_value(required, :ok, fn column -> + case Map.fetch(@references, column) do + {:ok, {field, reason}} -> if is_nil(Map.fetch!(item, field)), do: {:error, reason} + :error -> {:error, :revision_deleted} + end + end) + end + + # ── the reference itself ────────────────────────────────────────────────── + + defp reference(%CompositionItem{item_kind: :content} = item) do + with {:ok, object_id} <- need(item.item_object_id, :object_missing), + {:ok, id} <- need(item.content_revision_id, :revision_deleted), + {:ok, revision} <- + need( + Repo.one( + from r in ContentRevision, where: r.id == ^id and r.content_id == ^object_id + ), + :revision_deleted + ) do + content_revision(revision, object_id) + end + end + + defp reference(%CompositionItem{item_kind: :sense_quotation} = item) do + with {:ok, object_id} <- need(item.item_object_id, :object_missing), + {:ok, id} <- need(item.sense_revision_id, :revision_deleted), + {:ok, revision} <- + need( + Repo.one(from r in SenseRevision, where: r.id == ^id and r.sense_id == ^object_id), + :revision_deleted + ), + :ok <- current(revision), + :ok <- object_active(object_id), + :ok <- sense_source_active(object_id), + :ok <- record_displayable(revision.source_record_revision_id) do + words(revision, item.locator, item.words_sha256) + end + end + + defp reference(%CompositionItem{item_kind: :work, catalog_manifest: name} = item) + when is_binary(name) do + with {:ok, manifest} <- catalog(name, item.catalog_checksum, item.catalog_identity) do + case item.item_object_id do + nil -> + :ok + + object_id -> + with :ok <- object_active(object_id), + do: catalog_identity(object_id, manifest, item.catalog_identity) + end + end + end + + defp reference(%CompositionItem{item_kind: :work} = item) do + with {:ok, object_id} <- need(item.item_object_id, :object_missing), + {:ok, id} <- need(item.source_record_revision_id, :revision_deleted), + :ok <- record_displayable(id), + :ok <- object_active(object_id) do + record_describes(id, object_id) + end + end + + defp content_revision(revision, object_id) do + with :ok <- current(revision), + :ok <- ensure(Visibility.content_displayable?(revision), :display_restricted), + :ok <- object_active(object_id), + :ok <- content_source_active(object_id) do + record_displayable(revision.source_record_revision_id) + end + end + + defp current(%{is_current: true, lifecycle_state: :active}), do: :ok + defp current(%{is_current: true}), do: {:error, :revision_withdrawn} + defp current(_revision), do: {:error, :revision_superseded} + + defp object_active(id) do + ensure( + Repo.exists?(from o in Object, where: o.id == ^id and o.lifecycle_state == :active), + :object_retired + ) + end + + defp content_source_active(content_id) do + query = + from ci in ContentItem, + left_join: s in Source, + on: s.id == ci.source_id, + where: ci.object_id == ^content_id and (is_nil(ci.source_id) or s.active) + + ensure(Repo.exists?(query), :source_inactive) + end + + defp sense_source_active(sense_id) do + query = + from s in Sense, + join: src in Source, + on: src.id == s.source_id, + where: s.object_id == ^sense_id and src.active and s.identity_state == :active + + ensure(Repo.exists?(query), :source_inactive) + end + + # A revision with no source record is our own; one with a record is shown + # only while the record allows display and its source is active. + defp record_displayable(nil), do: :ok + + defp record_displayable(revision_id) do + case Repo.one( + from rev in SourceRecordRevision, + join: rec in SourceRecord, + on: rec.id == rev.source_record_id, + join: s in Source, + on: s.id == rec.source_id, + where: rev.id == ^revision_id, + select: {rec.display_allowed, s.active} + ) do + nil -> {:error, :revision_deleted} + {true, true} -> :ok + {false, _active} -> {:error, :display_restricted} + {_allowed, false} -> {:error, :source_inactive} + end + end + + # A source record is evidence of a work only if it materialized that work: + # a displayable record about something else is not. + defp record_describes(revision_id, object_id) do + query = + from rev in SourceRecordRevision, + join: out in MaterializedOutput, + on: out.source_record_id == rev.source_record_id, + where: + rev.id == ^revision_id and out.output_object_id == ^object_id and + is_nil(out.retired_at) + + ensure(Repo.exists?(query), :work_identity_mismatch) + end + + defp words(%SenseRevision{examples: examples}, "quotation:" <> index, sha256) + when is_list(examples) do + with {n, ""} <- Integer.parse(index), + %{"type" => "quotation", "text" => text} when is_binary(text) <- Enum.at(examples, n), + true <- Digest.sha256(text) == sha256 do + :ok + else + _ -> {:error, :words_changed} + end + end + + defp words(_revision, _locator, _sha256), do: {:error, :words_changed} + + # A committed catalog manifest is the exact revision of a catalog work. It + # is read from this build's priv/ and verified, and any file or validation + # failure withholds rather than raises. + @manifest_name ~r/\A[a-z0-9][a-z0-9-]*\z/ + + defp catalog(name, checksum, identity) do + with :ok <- ensure(Regex.match?(@manifest_name, name), :catalog_changed), + {:ok, manifest} <- committed_manifest(name), + :ok <- ensure(manifest["checksum"] == checksum, :catalog_changed), + field = Manifest.identity_field(manifest["kind"]), + :ok <- + ensure( + Enum.any?(manifest["rows"], &(to_string(&1[field]) == identity)), + :catalog_row_missing + ), + :ok <- + ensure( + Repo.exists?(from s in Source, where: s.slug == ^manifest["source"] and s.active), + :source_inactive + ) do + {:ok, manifest} + end + end + + # A registry object shown with a catalog pin must be the pinned work: it + # carries the row's identity in the kind's namespace, verified. + defp catalog_identity(object_id, manifest, identity) do + namespace = Manifest.identity_namespace(manifest["kind"]) + + query = + from x in ExternalIdentifier, + where: + x.object_id == ^object_id and x.namespace == ^namespace and + x.external_id == ^identity and x.status == :verified + + ensure(Repo.exists?(query), :work_identity_mismatch) + end + + defp committed_manifest(name) do + path = Application.app_dir(:devils_dictionary, "priv/artworks/manifests/#{name}.json") + + try do + {:ok, Manifest.load!(path)} + rescue + _error in [File.Error, Jason.DecodeError, ArgumentError, KeyError] -> + {:error, :catalog_changed} + end + end + + # ── the meaning and the claim ───────────────────────────────────────────── + + defp meaning(%CompositionItem{meaning_lexeme_id: id}, member_ids) when is_integer(id), + do: ensure(id in member_ids, :meaning_off_scope) + + defp meaning(%CompositionItem{meaning_sense_revision_id: id}, member_ids) + when is_integer(id) do + query = + from r in SenseRevision, + join: s in Sense, + on: s.object_id == r.sense_id, + where: r.id == ^id, + select: {r, s.lexeme_id} + + case Repo.one(query) do + nil -> {:error, :meaning_deleted} + {revision, lexeme_id} -> with :ok <- current(revision), do: on_scope(lexeme_id, member_ids) + end + end + + defp meaning(_item, _member_ids), do: {:error, :meaning_deleted} + + defp on_scope(lexeme_id, member_ids), do: ensure(lexeme_id in member_ids, :meaning_off_scope) + + # An item made without a claim needs none (a deleted one was caught by + # `still_present/1`). + defp assertion(nil), do: :ok + + defp assertion(id) do + from(r in AssertionRevision, + where: r.id == ^id and r.is_current and r.lifecycle_state == :active + ) + |> Claims.visible(:public) + |> Repo.exists?() + |> ensure(:claim_not_visible) + end + + defp need(nil, reason), do: {:error, reason} + defp need(value, _reason), do: {:ok, value} + + defp ensure(true, _reason), do: :ok + defp ensure(_condition, reason), do: {:error, reason} +end diff --git a/lib/devils_dictionary/curation/lead_rule.ex b/lib/devils_dictionary/curation/lead_rule.ex new file mode 100644 index 00000000..98727249 --- /dev/null +++ b/lib/devils_dictionary/curation/lead_rule.ex @@ -0,0 +1,109 @@ +defmodule DevilsDictionary.Curation.LeadRule do + @moduledoc """ + Bierce first (K10), read from the registry. + + The editorial rule of #156 and #193: **where a word has a *Devil's + Dictionary* entry, that entry leads.** A composition cannot outvote it. + + A definition is *on* a scope's page when a current, active `defines` claim + links it to a member lexeme or to a sense of one, and that claim passes the + canonical public visibility policy (`Claims.visible/2`). A rejected or + withdrawn claim, or one whose endpoint is retired or withdrawn, puts nothing + on the page. The content's own current revision must be active. This is + the same relationship the word page reads (`Lexicon.WordPage`), with the + same review policy the rest of the public site applies. + + An entry is *applicable* when it is on the page, is from the active `bierce` + source, and may itself lead now (`Eligibility.leadable?/1`). A source's + priority never overrides its eligibility: a Bierce entry whose rights, + record or source forbid display neither leads nor blocks another lead. + + * Where an applicable entry exists, only an applicable entry may lead + (`:priority_source`). A version naming another lead is refused + (`:priority_source_available`), and one naming none is refused + (`:priority_source_missing`). + * Where none exists, any definition on the page may lead + (`:manual_fallback`), or no lead (`:none`). A definition not on the + page, including one whose claim was rejected, is `:lead_not_on_scope`. + There is no automatic fallback order; that is still an open product + decision. + + The check runs when a version is made, reviewed, published and read. A + published lead whose claim is later rejected, or a Bierce entry that + appears after a fallback was published, withholds the lead. Nothing is + substituted for it. + """ + + import Ecto.Query + + alias DevilsDictionary.Claims + alias DevilsDictionary.Claims.AssertionRevision + alias DevilsDictionary.Curation.Eligibility + alias DevilsDictionary.Registry.{ContentItem, ContentRevision, Sense} + alias DevilsDictionary.Repo + alias DevilsDictionary.Sources.Source + + @priority_source "bierce" + + @doc "The slug of the source whose applicable entry leads." + def priority_source, do: @priority_source + + @doc "The applicable Bierce entries for these member lexemes, as sorted content object ids." + def applicable(member_ids) do + member_ids + |> defining() + |> join(:inner, [content: ci], s in Source, + on: s.id == ci.source_id and s.slug == @priority_source and s.active + ) + |> select([content: ci], ci.object_id) + |> distinct(true) + |> Repo.all() + |> Enum.filter(&Eligibility.leadable?/1) + |> Enum.sort() + end + + @doc "Whether content object `content_id` is a definition on this scope's page." + def defines?(content_id, member_ids) do + member_ids + |> defining() + |> where([content: ci], ci.object_id == ^content_id) + |> Repo.exists?() + end + + @doc """ + The rule a lead (a content object id, or `nil`) is admitted under: + `{:ok, :priority_source | :manual_fallback | :none}`, or `{:error, reason}`. + """ + def check(member_ids, lead_id, applicable \\ nil) do + applicable = applicable || applicable(member_ids) + + cond do + is_nil(lead_id) and applicable == [] -> {:ok, :none} + is_nil(lead_id) -> {:error, :priority_source_missing} + lead_id in applicable -> {:ok, :priority_source} + applicable != [] -> {:error, :priority_source_available} + defines?(lead_id, member_ids) -> {:ok, :manual_fallback} + true -> {:error, :lead_not_on_scope} + end + end + + # The publicly visible `defines` claims from a content item to a member + # lexeme or a member's sense. The claim revision is the query's root + # binding, which is what `Claims.visible/2` filters. + defp defining(member_ids) do + from(link in AssertionRevision, + join: pr in assoc(link, :predicate), + on: pr.key == "defines", + join: ci in ContentItem, + as: :content, + on: ci.object_id == link.subject_object_id, + join: cr in ContentRevision, + on: cr.content_id == ci.object_id and cr.is_current and cr.lifecycle_state == :active, + left_join: sense in Sense, + on: sense.object_id == link.object_object_id, + where: link.is_current and link.lifecycle_state == :active, + where: link.object_object_id in ^member_ids or sense.lexeme_id in ^member_ids + ) + |> Claims.visible(:public) + end +end diff --git a/lib/devils_dictionary/curation/profile.ex b/lib/devils_dictionary/curation/profile.ex new file mode 100644 index 00000000..66a7197e --- /dev/null +++ b/lib/devils_dictionary/curation/profile.ex @@ -0,0 +1,36 @@ +defmodule DevilsDictionary.Curation.Profile do + @moduledoc """ + A stable curator identity (#196 `curator_profiles`): *inspired by* a dead + writer, never impersonating one. + + A profile is `proposed` until a human reviewer admits one sourced version of + its dossier (`Profiles.admit/4`). The database refuses an `admitted` row + unless it has: + + * a bot principal; + * a human admitter and a reason; + * a version citing its sources and deceased-status evidence (C6). + + Seeding creates the five proposed identities and nothing else: no dossier, + no evidence, no quotation. + """ + use Ecto.Schema + + @states [:proposed, :admitted, :retired] + + schema "curator_profiles" do + field :slug, :string + field :label, :string + field :subject_label, :string + field :state, Ecto.Enum, values: @states, default: :proposed + field :bot_actor_id, :id + field :admitted_by_actor_id, :id + field :admitted_at, :utc_datetime_usec + field :admission_reason, :string + field :current_version_id, :id + + timestamps(type: :utc_datetime_usec) + end + + def states, do: @states +end diff --git a/lib/devils_dictionary/curation/profile_version.ex b/lib/devils_dictionary/curation/profile_version.ex new file mode 100644 index 00000000..5d3e12ea --- /dev/null +++ b/lib/devils_dictionary/curation/profile_version.ex @@ -0,0 +1,25 @@ +defmodule DevilsDictionary.Curation.ProfileVersion do + @moduledoc """ + One immutable dossier of a curator profile (#196 `curator_profile_versions`). + + `source_refs` and `deceased_evidence_refs` are references, not text. They + record what a reviewer checked; no biography is written here. + """ + use Ecto.Schema + + alias DevilsDictionary.Types.JsonValue + + schema "curator_profile_versions" do + field :profile_id, :id + field :version, :integer + field :manifest_hash, :string + field :dossier, :map, default: %{} + field :source_refs, JsonValue, default: [] + field :deceased_evidence_refs, JsonValue, default: [] + field :template_version, :string + field :created_by_actor_id, :id + field :change_reason, :string + + timestamps(type: :utc_datetime_usec, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/profiles.ex b/lib/devils_dictionary/curation/profiles.ex new file mode 100644 index 00000000..e9ca2db1 --- /dev/null +++ b/lib/devils_dictionary/curation/profiles.ex @@ -0,0 +1,180 @@ +defmodule DevilsDictionary.Curation.Profiles do + @moduledoc """ + Curator profile identities, their dossier versions, and admission (#196). + + The five inspired-by identities are seeded as `proposed`, with no version, + bot actor or subject entity: names, and nothing asserted about the people + they are named for. A profile is admitted only by a reviewer, from a + version that cites its sources and deceased-status evidence (C6). Admission + is what a later panel slice will require of every roster seat. No seat + exists yet, because no dossier has been reviewed. + """ + + import Ecto.Query + + alias DevilsDictionary.Curation.{Authority, Digest, Profile, ProfileVersion, Transaction} + alias DevilsDictionary.Repo + alias DevilsDictionary.Sources.Actor + + @proposed [ + %{slug: "bierce", subject_label: "Ambrose Bierce"}, + %{slug: "voltaire", subject_label: "Voltaire"}, + %{slug: "vonnegut", subject_label: "Kurt Vonnegut"}, + %{slug: "hitchens", subject_label: "Christopher Hitchens"}, + %{slug: "le-guin", subject_label: "Ursula K. Le Guin"} + ] + + @doc "The slugs of the five proposed identities, in roster order." + def proposed_slugs, do: Enum.map(@proposed, & &1.slug) + + @doc """ + Inserts the five proposed identities that are missing and returns all five. + Idempotent: an existing profile, whatever its state, is left as it is. + """ + def seed_proposed! do + now = DateTime.utc_now() + + rows = + for p <- @proposed do + %{ + slug: p.slug, + label: "Inspired by #{p.subject_label}", + subject_label: p.subject_label, + state: :proposed, + inserted_at: now, + updated_at: now + } + end + + Repo.insert_all(Profile, rows, on_conflict: :nothing, conflict_target: :slug) + + slugs = proposed_slugs() + by_slug = Repo.all(from p in Profile, where: p.slug in ^slugs) |> Map.new(&{&1.slug, &1}) + Enum.map(slugs, &Map.fetch!(by_slug, &1)) + end + + @doc """ + Adds a dossier version to a profile, as a human author. A proposed + profile's pointer follows its newest version. An admitted profile keeps + its admitted version until it is admitted again. + + `attrs`: `:dossier` (map), `:source_refs` and `:deceased_evidence_refs` + (lists of references), `:template_version`, `:reason`. + """ + def add_version(scope, profile_id, attrs) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :author), + {:ok, profile} <- lock(profile_id), + {:ok, reason} <- Transaction.required(Map.to_list(attrs), :reason) do + body = %{ + "dossier" => Map.get(attrs, :dossier, %{}), + "source_refs" => Map.get(attrs, :source_refs, []), + "deceased_evidence_refs" => Map.get(attrs, :deceased_evidence_refs, []), + "template_version" => Map.fetch!(attrs, :template_version) + } + + hash = Digest.term(body) + + case Repo.get_by(ProfileVersion, profile_id: profile.id, manifest_hash: hash) do + %ProfileVersion{id: id} -> + {:error, {:duplicate_version, id}} + + nil -> + version = + Repo.insert!(%ProfileVersion{ + profile_id: profile.id, + version: next_version(profile.id), + manifest_hash: hash, + dossier: body["dossier"], + source_refs: body["source_refs"], + deceased_evidence_refs: body["deceased_evidence_refs"], + template_version: body["template_version"], + created_by_actor_id: actor.id, + change_reason: reason + }) + + if profile.state == :proposed do + profile + |> Ecto.Changeset.change(current_version_id: version.id) + |> Repo.update!() + end + + {:ok, version} + end + end + end) + end + + @doc """ + Admits one version of a profile, as a reviewer, acting through `bot_actor_id` + (a `bot` actor no other profile uses). The version must cite at least one + source and at least one piece of deceased-status evidence. Nothing is + judged here about whether the evidence is true: a reviewer did that, and + this records who. + """ + def admit(scope, profile_id, version_id, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, profile} <- lock(profile_id), + {:ok, version} <- + Transaction.need( + Repo.get_by(ProfileVersion, id: version_id, profile_id: profile.id), + :version_not_found + ), + :ok <- Transaction.check(cited?(version.source_refs), :sources_missing), + :ok <- + Transaction.check( + cited?(version.deceased_evidence_refs), + :deceased_evidence_missing + ), + {:ok, bot} <- bot(Keyword.get(opts, :bot_actor_id), profile.id) do + profile + |> Ecto.Changeset.change( + state: :admitted, + current_version_id: version.id, + bot_actor_id: bot.id, + admitted_by_actor_id: actor.id, + admitted_at: DateTime.utc_now(), + admission_reason: reason + ) + |> Repo.update() + end + end) + end + + defp cited?(refs), do: is_list(refs) and refs != [] + + defp bot(nil, _profile_id), do: {:error, :bot_actor_missing} + + defp bot(id, profile_id) do + with {:ok, actor} <- + Transaction.need( + Repo.get_by(Actor, id: id, actor_kind: :bot), + :bot_actor_missing + ), + :ok <- + Transaction.check( + not Repo.exists?( + from p in Profile, where: p.bot_actor_id == ^id and p.id != ^profile_id + ), + :bot_actor_in_use + ) do + {:ok, actor} + end + end + + defp lock(id) do + Transaction.need( + Repo.one(from p in Profile, where: p.id == ^id, lock: "FOR UPDATE"), + :not_found + ) + end + + defp next_version(profile_id) do + (Repo.one( + from v in ProfileVersion, where: v.profile_id == ^profile_id, select: max(v.version) + ) || + 0) + 1 + end +end diff --git a/lib/devils_dictionary/curation/publications.ex b/lib/devils_dictionary/curation/publications.ex new file mode 100644 index 00000000..46ffe571 --- /dev/null +++ b/lib/devils_dictionary/curation/publications.ex @@ -0,0 +1,206 @@ +defmodule DevilsDictionary.Curation.Publications do + @moduledoc """ + Transactional publication of composition versions (R3–R5). + + A publication has two parts, kept separate: + + * **an authority**, which says who or what decided this version should + be shown; + * **a commit**, which checks the version still stands and moves the + pointer with a receipt. + + The commit is the same whatever the authority. It locks the composition + row and checks: + + 1. the **expected pointer** the caller saw (`{:stale_pointer, current}`); + 2. that the composition is active and the version is not already + published; + 3. that the version still stands: same scope, same configuration version, + and a clean evaluation to its own eligibility fingerprint; + 4. that the authority is valid. + + It then writes a receipt naming its authority (`authority_kind`) and moves + the pointer, in one transaction. The database repeats what matters: + + * the receipt must satisfy its authority's rule; + * the pointer must equal the latest receipt at commit; + * each receipt must continue the one before it. + + So two publications racing from one pointer cannot both commit, even if a + caller skips this module. + + **Authorities.** + + * **`:operator`** (`publish/3`) is available in this slice. A reviewer + publishes a version whose latest review is an acceptance of its + fingerprint. It is an *optional* operator path, for manual compositions, + overrides and corrections. It is not the gate every selection must pass. + * **`:panel_decision`** is the intended routine path, but it is **not + available**. The configured panel reaching consensus on a version that + passes the evidence and policy checks would publish it with no + per-selection human sign-off. It needs genuine, finalized decision + records, which do not exist until runs land (#197). The database refuses + any authority but `operator`, and nothing here fabricates a run, vote or + decision to stand in for one. What the integration must add is set out + in `docs/curation/persistence-slice-1.md`, "Publication authority". + + `withdraw/3` moves the pointer to none, with a receipt of its own, as an + operator. The reader withholds an ineligible item on its own, without any + withdrawal. + """ + + import Ecto.Query + + alias DevilsDictionary.Curation.{ + Authority, + Composition, + CompositionPublication, + CompositionVersion, + Standing, + Transaction + } + + alias DevilsDictionary.Repo + + @doc """ + Publishes a version on operator authority, as a reviewer. `opts`: + `:reason`, `:idempotency_key`, and `:expected_pointer` (the currently + published version id the caller saw, or `nil`). All three are required. + """ + def publish(scope, version_id, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, key} <- Transaction.required(opts, :idempotency_key), + expected = Keyword.fetch!(opts, :expected_pointer), + {:ok, version} <- + Transaction.need(Repo.get(CompositionVersion, version_id), :not_found), + {:ok, composition} <- Standing.lock_composition(version.composition_id), + :fresh <- + Transaction.replay(CompositionPublication, key, fn r -> + r.composition_id == composition.id and r.action == :publish and + r.published_version_id == version.id + end) do + commit(composition, version, expected, operator(actor), reason, key) + else + {:replay, receipt} -> {:ok, receipt} + error -> error + end + end) + end + + # The operator authority: the reviewer acting, and the version's latest + # review, which must be an acceptance. The acceptance may be another + # reviewer's; the receipt names both. + defp operator(actor) do + %{ + kind: :operator, + actor: actor, + authorize: fn version -> + case Standing.latest_review(version.id) do + %{decision: :accepted} = review -> {:ok, %{authorizing_review_id: review.id}} + _ -> {:error, :not_approved} + end + end + } + end + + defp commit(composition, version, expected, authority, reason, key) do + with :ok <- Transaction.check(composition.state == :active, :composition_retired), + :ok <- pointer(composition, expected), + :ok <- + Transaction.check( + composition.current_published_version_id != version.id, + :already_published + ), + :ok <- Standing.scope(version, composition), + :ok <- Standing.configuration(version), + {:ok, authorization} <- authority.authorize.(version), + :ok <- Standing.eligible(Standing.evaluate(version, composition), version) do + receipt = + Repo.insert!( + struct( + CompositionPublication, + Map.merge(authorization, %{ + composition_id: composition.id, + action: :publish, + authority_kind: authority.kind, + previous_version_id: composition.current_published_version_id, + published_version_id: version.id, + actor_id: authority.actor.id, + reason: reason, + eligibility_fingerprint: version.eligibility_fingerprint, + idempotency_key: key + }) + ) + ) + + move!(composition, version.id) + {:ok, receipt} + end + end + + @doc """ + Withdraws a composition's publication, as a reviewer. Its pointer becomes + `nil`. `opts` as for `publish/3`; `:expected_pointer` is the published + version being withdrawn. + """ + def withdraw(scope, composition_id, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, key} <- Transaction.required(opts, :idempotency_key), + expected = Keyword.fetch!(opts, :expected_pointer), + {:ok, composition} <- Standing.lock_composition(composition_id), + :fresh <- + Transaction.replay(CompositionPublication, key, fn r -> + r.composition_id == composition.id and r.action == :withdraw and + r.previous_version_id == expected + end), + :ok <- pointer(composition, expected), + :ok <- + Transaction.check(not is_nil(composition.current_published_version_id), :unpublished) do + receipt = + Repo.insert!(%CompositionPublication{ + composition_id: composition.id, + action: :withdraw, + authority_kind: :operator, + previous_version_id: composition.current_published_version_id, + actor_id: actor.id, + reason: reason, + idempotency_key: key + }) + + move!(composition, nil) + {:ok, receipt} + else + {:replay, receipt} -> {:ok, receipt} + error -> error + end + end) + end + + @doc "The latest publication receipt of a composition, or `nil`." + def latest(composition_id) do + Repo.one( + from p in CompositionPublication, + where: p.composition_id == ^composition_id, + order_by: [desc: p.id], + limit: 1 + ) + end + + defp pointer(%Composition{current_published_version_id: current}, current), do: :ok + + defp pointer(%Composition{current_published_version_id: current}, _seen), + do: {:error, {:stale_pointer, current}} + + defp move!(composition, version_id) do + composition + |> Ecto.Changeset.change( + current_published_version_id: version_id, + lock_version: composition.lock_version + 1 + ) + |> Repo.update!() + end +end diff --git a/lib/devils_dictionary/curation/published.ex b/lib/devils_dictionary/curation/published.ex new file mode 100644 index 00000000..3b6b3048 --- /dev/null +++ b/lib/devils_dictionary/curation/published.ex @@ -0,0 +1,115 @@ +defmodule DevilsDictionary.Curation.Published do + @moduledoc """ + The narrow read of a composition's published arrangement (R6). + + It answers only what was published, revalidated now. **It withholds and + never substitutes.** A problem with the version as a whole withholds all + of it (`{:withheld, reasons}`): + + * the composition is retired; + * its configuration moved on or was disabled; + * its scope changed; + * its latest review is no longer an acceptance. + + A problem with one item withholds that item alone and says why. A + withheld lead is not replaced by another definition, and a withheld + highlight is not replaced by the next candidate. A Bierce entry that + appears after a fallback lead was published withholds that lead + (`:priority_source_available`); it does not promote itself. + + Nothing in the web layer calls this yet. Production selection stays + disabled: no page reads compositions, and binding one to a page is #194's + (#205). Mapping the answer onto the #156 opening view model waits for PR + #202, which is not merged. + """ + + alias DevilsDictionary.Curation.{ + Composition, + CompositionVersion, + Compositions, + Publications, + Standing + } + + alias DevilsDictionary.Repo + + @doc """ + The published arrangement of a composition, as one of: + + * `{:ok, %{composition, version, receipt, lead, highlights, withheld}}`, + where `lead` is an item or `nil`, `highlights` are the eligible + highlights in order, and `withheld` lists `%{role, position, reason}`; + * `{:withheld, reasons}`; + * `:unpublished`; + * `{:error, :not_found}`. + """ + def current(composition_id) do + case Repo.get(Composition, composition_id) do + nil -> + {:error, :not_found} + + %Composition{current_published_version_id: nil} -> + :unpublished + + %Composition{} = composition -> + version = Repo.get!(CompositionVersion, composition.current_published_version_id) + + case version_reasons(version, composition) do + [] -> {:ok, arrangement(version, composition)} + reasons -> {:withheld, reasons} + end + end + end + + defp version_reasons(version, composition) do + [ + composition.state != :active && :composition_retired, + reason(Standing.configuration(version)), + reason(Standing.scope(version, composition)), + not match?(%{decision: :accepted}, Standing.latest_review(version.id)) && + :approval_withdrawn + ] + |> Enum.filter(& &1) + end + + defp reason(:ok), do: false + defp reason({:error, reason}), do: reason + + defp arrangement(version, composition) do + items = Compositions.items(version.id) + evaluation = Standing.evaluate(version, composition) + statuses = Map.new(evaluation.results, fn {role, pos, status} -> {{role, pos}, status} end) + + status_of = fn item -> + case {item.role, evaluation.lead_rule} do + {:lead, {:error, reason}} -> reason + _ -> Map.fetch!(statuses, {item.role, item.position}) + end + end + + {shown, held} = Enum.split_with(items, &(status_of.(&1) == :ok)) + + withheld = + Enum.map(held, &%{role: &1.role, position: &1.position, reason: status_of.(&1)}) ++ + missing_lead(items, evaluation.lead_rule) + + %{ + composition: composition, + version: version, + receipt: Publications.latest(composition.id), + lead: Enum.find(shown, &(&1.role == :lead)), + highlights: Enum.filter(shown, &(&1.role == :highlight)), + withheld: withheld + } + end + + # A version published with no lead, on a scope where a Bierce entry has + # since appeared, reports the gap rather than filling it. + defp missing_lead(items, {:error, reason}) do + if Enum.any?(items, &(&1.role == :lead)), + do: [], + else: [%{role: :lead, position: 1, reason: reason}] + end + + defp missing_lead(_items, _rule), do: [] +end diff --git a/lib/devils_dictionary/curation/reviews.ex b/lib/devils_dictionary/curation/reviews.ex new file mode 100644 index 00000000..59c43379 --- /dev/null +++ b/lib/devils_dictionary/curation/reviews.ex @@ -0,0 +1,89 @@ +defmodule DevilsDictionary.Curation.Reviews do + @moduledoc """ + Operator presentation review of composition versions (R1, R2). + + This is the **optional operator path**: a person accepting a manual + composition, an override or a correction, which then authorizes an + operator publication (`Publications.publish/3`). It is not the gate every + selection must pass. Routine selections are meant to publish on the + authority of a genuine panel decision, which is not available yet + (`docs/curation/persistence-slice-1.md`, "Publication authority"). + + Only an account with the reviewer role decides. The role is rechecked + under a row lock, and the database checks it again. There is no bot + operator: a panel's authority is its own decision record, never a review + written in a person's name. + + Decisions are append-only, and the latest one is the version's state. An + idempotency key replays its own decision and refuses a different one. + + **Acceptance is not publication.** Accepting a version: + + * writes no pointer, no publication receipt, no `assertion_reviews` row + and no page row; + * says the arrangement may be shown, not that any claim in it is true; + * is refused unless the version still stands (`Standing`). Its + configuration, its scope and its eligibility must all be as it was + made. What was accepted is recorded as the version's eligibility + fingerprint. + """ + + alias DevilsDictionary.Curation.{ + Authority, + CompositionReview, + CompositionVersion, + Standing, + Transaction + } + + alias DevilsDictionary.Repo + + @doc """ + Records `decision` (`:accepted`, `:rejected`, `:withdrawn` or + `:needs_review`) on a version, as a reviewer. `opts`: `:reason`, + `:idempotency_key`. + """ + def decide(scope, version_id, decision, opts) do + Transaction.run(fn -> + with {:ok, actor} <- Authority.actor(scope, :reviewer), + :ok <- Transaction.check(decision in CompositionReview.decisions(), :unknown_decision), + {:ok, reason} <- Transaction.required(opts, :reason), + {:ok, key} <- Transaction.required(opts, :idempotency_key), + {:ok, version} <- + Transaction.need(Repo.get(CompositionVersion, version_id), :not_found), + {:ok, composition} <- Standing.lock_composition(version.composition_id), + :fresh <- + Transaction.replay(CompositionReview, key, fn r -> + r.composition_version_id == version.id and r.decision == decision + end) do + evaluation = Standing.evaluate(version, composition) + + with :ok <- acceptable(decision, version, composition, evaluation) do + {:ok, + Repo.insert!(%CompositionReview{ + composition_id: composition.id, + composition_version_id: version.id, + reviewer_actor_id: actor.id, + decision: decision, + reason: reason, + reviewed_eligibility_fingerprint: evaluation.fingerprint, + idempotency_key: key + })} + end + else + {:replay, review} -> {:ok, review} + error -> error + end + end) + end + + defp acceptable(:accepted, version, composition, evaluation) do + with :ok <- Transaction.check(composition.state == :active, :composition_retired), + :ok <- Standing.scope(version, composition), + :ok <- Standing.configuration(version) do + Standing.eligible(evaluation, version) + end + end + + defp acceptable(_decision, _version, _composition, _evaluation), do: :ok +end diff --git a/lib/devils_dictionary/curation/scope_change.ex b/lib/devils_dictionary/curation/scope_change.ex new file mode 100644 index 00000000..3985be34 --- /dev/null +++ b/lib/devils_dictionary/curation/scope_change.ex @@ -0,0 +1,21 @@ +defmodule DevilsDictionary.Curation.ScopeChange do + @moduledoc """ + The receipt for a composition's scope being set or changed (K2, K3). It + records the member ids, the signature before and after, who, and why. + Append-only. + """ + use Ecto.Schema + + alias DevilsDictionary.Types.JsonValue + + schema "editorial_composition_scope_changes" do + field :composition_id, :id + field :previous_signature, :string + field :scope_signature, :string + field :members, JsonValue + field :actor_id, :id + field :reason, :string + + timestamps(type: :utc_datetime_usec, inserted_at: :changed_at, updated_at: false) + end +end diff --git a/lib/devils_dictionary/curation/standing.ex b/lib/devils_dictionary/curation/standing.ex new file mode 100644 index 00000000..efd0b0e6 --- /dev/null +++ b/lib/devils_dictionary/curation/standing.ex @@ -0,0 +1,93 @@ +defmodule DevilsDictionary.Curation.Standing do + @moduledoc """ + Whether a composition version still stands: the checks shared by review, + publication and reading. + + A version was made against one configuration version, one scope and one + eligibility fingerprint. It stands while all three are still true: + + * `configuration/1`: its configuration is enabled and still on the + version it was made under (`:configuration_changed`); + * `scope/2`: its composition's scope is the one it was made for + (`:scope_changed`); + * `eligible/2`: its items and its lead rule evaluate `:ok`, to the same + fingerprint it was made with (`:stale_eligibility`). + + Plus, for publication and reading, `latest_review/1`: the latest human + decision on it. + """ + + import Ecto.Query + + alias DevilsDictionary.Curation.{ + Composition, + CompositionReview, + CompositionVersion, + Compositions, + Configurations, + Eligibility + } + + alias DevilsDictionary.Repo + + @doc """ + The version's items evaluated against its composition's current members, + under the configuration version it was made with. + """ + def evaluate(%CompositionVersion{} = version, %Composition{} = composition) do + version.id + |> Compositions.items() + |> Eligibility.evaluate( + Compositions.member_ids(composition.id), + version.scope_signature, + version.configuration_version_id + ) + end + + @doc "`:ok` while the configuration is enabled, ready and on this version's configuration version." + def configuration(%CompositionVersion{} = version) do + case Configurations.current(version.curation_configuration_id) do + {:ok, _configuration, %{id: id}} when id == version.configuration_version_id -> :ok + _ -> {:error, :configuration_changed} + end + end + + @doc "`:ok` while the composition's scope is the version's." + def scope(%CompositionVersion{scope_signature: s}, %Composition{scope_signature: s}), do: :ok + def scope(_version, _composition), do: {:error, :scope_changed} + + @doc "`:ok` when an evaluation is clean and matches the version's fingerprint." + def eligible(evaluation, %CompositionVersion{} = version) do + cond do + match?({:error, _}, evaluation.lead_rule) -> + evaluation.lead_rule + + not evaluation.ok? -> + {:error, {:ineligible, Enum.reject(evaluation.results, &(elem(&1, 2) == :ok))}} + + evaluation.fingerprint != version.eligibility_fingerprint -> + {:error, :stale_eligibility} + + true -> + :ok + end + end + + @doc "The latest review of a version, or `nil`." + def latest_review(version_id) do + Repo.one( + from r in CompositionReview, + where: r.composition_version_id == ^version_id, + order_by: [desc: r.id], + limit: 1 + ) + end + + @doc "The composition row, locked for the rest of the transaction." + def lock_composition(id) do + case Repo.one(from c in Composition, where: c.id == ^id, lock: "FOR UPDATE") do + nil -> {:error, :not_found} + composition -> {:ok, composition} + end + end +end diff --git a/lib/devils_dictionary/curation/transaction.ex b/lib/devils_dictionary/curation/transaction.ex new file mode 100644 index 00000000..9e273da9 --- /dev/null +++ b/lib/devils_dictionary/curation/transaction.ex @@ -0,0 +1,55 @@ +defmodule DevilsDictionary.Curation.Transaction do + @moduledoc """ + The small vocabulary every curation write shares: one transaction per + action, a failure that rolls it back, and idempotency keys that replay an + earlier receipt rather than write a second one. + """ + + alias DevilsDictionary.Repo + + @doc """ + Runs `fun` in a transaction. `{:ok, value}` commits; `{:error, reason}` rolls + back and is returned as is. + """ + def run(fun) do + Repo.transaction(fn -> + case fun.() do + {:ok, value} -> value + {:error, reason} -> Repo.rollback(reason) + end + end) + end + + @doc "`{:ok, value}`, or `{:error, reason}` for `nil` and `false`." + def need(nil, reason), do: {:error, reason} + def need(false, reason), do: {:error, reason} + def need(value, _reason), do: {:ok, value} + + @doc "`:ok` when `condition` is `true`, else `{:error, reason}`." + def check(true, _reason), do: :ok + def check(_condition, reason), do: {:error, reason} + + @doc """ + What an idempotency key has already done. `:fresh` if nothing, `{:replay, + row}` if the same action, `{:error, :idempotency_conflict}` if a different + one. Call after taking the row lock that serializes the action, so a + concurrent first use is seen. + """ + def replay(schema, key, same?) do + case Repo.get_by(schema, idempotency_key: key) do + nil -> :fresh + row -> if same?.(row), do: {:replay, row}, else: {:error, :idempotency_conflict} + end + end + + @doc "The non-blank string at `key` in `opts`, or `{:error, {:required, key}}`." + def required(opts, key) do + case Keyword.get(opts, key) do + value when is_binary(value) -> + if String.trim(value) == "", do: {:error, {:required, key}}, else: {:ok, value} + + _ -> + {:error, {:required, key}} + end + end +end diff --git a/lib/mix/tasks/dd.curation.seed.ex b/lib/mix/tasks/dd.curation.seed.ex new file mode 100644 index 00000000..3a67064d --- /dev/null +++ b/lib/mix/tasks/dd.curation.seed.ex @@ -0,0 +1,60 @@ +defmodule Mix.Tasks.Dd.Curation.Seed do + @shortdoc "Seeds the draft global-default curation configuration and five proposed profiles (#201)" + + @moduledoc """ + Seeds curation's system-owned identities (#196, #201): + + mix dd.curation.seed + + What it writes, if missing: + + * the `global-default` configuration, as a **draft**; + * its manual-only version 1: Bierce-first lead, up to three highlights, + empty roster; + * the five proposed profile identities: Bierce, Voltaire, Vonnegut, + Hitchens and Le Guin. + + It **activates nothing and approves nothing**. The default resolves as + unavailable (`:no_active_version`) until a reviewer calls + `Curation.Configurations.activate/4`. No profile gets a dossier, evidence + or a bot actor, and no composition, version, review or publication is + written. A re-run changes nothing, and an activated configuration stays + activated. + + Curation state is restored from a backup, not regenerated, like routing + state: registry object ids exist only in the database. + """ + + use Mix.Task + + alias DevilsDictionary.Curation.Configurations + + @impl Mix.Task + def run(_args) do + start_repo_only() + + %{configuration: c, version: v, profiles: profiles} = Configurations.seed!() + + Mix.shell().info( + "#{c.slug}: #{c.state}, version #{v.version} (#{v.lead_policy}, " <> + "#{v.max_highlights} highlights, roster #{String.slice(v.roster_hash, 0, 12)}…)" + ) + + Mix.shell().info("profiles: " <> Enum.map_join(profiles, ", ", &"#{&1.slug} (#{&1.state})")) + + Mix.shell().info("resolution: #{inspect(Configurations.resolve_default())}") + end + + # The repo and nothing else: `app.start` would start Oban, and a second + # Oban node beside a running server executes that server's queued runs with + # this checkout's code. + defp start_repo_only do + Mix.Task.run("app.config") + {:ok, _} = Application.ensure_all_started([:postgrex, :ecto_sql]) + + case DevilsDictionary.Repo.start_link() do + {:ok, _pid} -> :ok + {:error, {:already_started, _pid}} -> :ok + end + end +end diff --git a/priv/repo/migrations/20260926233642_create_curation_foundation.exs b/priv/repo/migrations/20260926233642_create_curation_foundation.exs new file mode 100644 index 00000000..20ce412d --- /dev/null +++ b/priv/repo/migrations/20260926233642_create_curation_foundation.exs @@ -0,0 +1,1138 @@ +defmodule DevilsDictionary.Repo.Migrations.CreateCurationFoundation do + @moduledoc """ + Curation persistence, slice 1 (#196, #201): configuration and profile + identities, and the manual composition → review → publication path. + + Read `docs/curation/persistence-slice-1.md` first. Every constraint here is a + row in its invariant map (C1–C8, K1–K10, R1–R7), and that document lists the + reconciliations with the #196 draft. In short: + + * no model table and no model column (a configuration version is + manual-only until a validated model exists); + * profiles are identities without dossiers; + * nothing is activated, approved or published by this migration; + * nothing here writes a claim, a run or a page. + + Three kinds of enforcement: + + * **composite foreign keys** for ownership — a pointer, parent, review or + revision must belong to the same configuration, composition or object; + * **immutability triggers** — versions, rosters, items, decisions and + receipts are never updated or deleted. The one exception: an item's + reference to a source row may be nulled by `ON DELETE SET NULL`, so a + mandatory source deletion is never blocked; + * **deferred constraint triggers**, checked at COMMIT against the rows as + committed then: + - a pointer moves only with a receipt; + - each receipt continues the one before it, so two racing writers + cannot both win; + - a roster, an arrangement or a scope cannot be extended after the + transaction that created it. + """ + use Ecto.Migration + + def change do + require_postgresql_15() + profiles() + configurations() + revision_ownership_indexes() + compositions() + triggers() + end + + # Column-list referential actions (`ON DELETE SET NULL (column)`, R7) arrived + # in PostgreSQL 15. On an older server the migration would fail halfway + # through with a syntax error; it says why instead. + defp require_postgresql_15 do + execute """ + DO $$ + BEGIN + IF current_setting('server_version_num')::int < 150000 THEN + RAISE EXCEPTION 'the curation foundation needs PostgreSQL 15 or newer (column-list ON DELETE SET NULL); this server is %', + current_setting('server_version'); + END IF; + END + $$; + """, + "SELECT 1" + end + + # ── profiles (#196 curator_profiles, curator_profile_versions) ────────────── + + defp profiles do + create table(:curator_profiles) do + add :slug, :text, null: false + add :label, :text, null: false + add :subject_label, :text, null: false + add :state, :text, null: false, default: "proposed" + add :bot_actor_id, references(:actors, on_delete: :restrict) + add :admitted_by_actor_id, references(:actors, on_delete: :restrict) + add :admitted_at, :utc_datetime_usec + add :admission_reason, :text + add :current_version_id, :bigint + + timestamps(type: :utc_datetime_usec) + end + + create unique_index(:curator_profiles, [:slug]) + + create unique_index(:curator_profiles, [:bot_actor_id], + where: "bot_actor_id IS NOT NULL", + name: :curator_profiles_bot_actor_index + ) + + create constraint(:curator_profiles, :curator_profiles_state, + check: "state IN ('proposed', 'admitted', 'retired')" + ) + + # C6: an admitted profile is a sourced, human-admitted one. The sources and + # the admitter's kind live in other rows, so a trigger checks them. + create constraint(:curator_profiles, :curator_profiles_admission, + check: """ + state <> 'admitted' OR ( + current_version_id IS NOT NULL AND bot_actor_id IS NOT NULL AND + admitted_by_actor_id IS NOT NULL AND admitted_at IS NOT NULL AND + admission_reason IS NOT NULL AND btrim(admission_reason) <> '' + ) + """ + ) + + create table(:curator_profile_versions) do + add :profile_id, references(:curator_profiles, on_delete: :restrict), null: false + add :version, :integer, null: false + add :manifest_hash, :text, null: false + add :dossier, :map, null: false, default: %{} + add :source_refs, :jsonb, null: false, default: fragment("'[]'::jsonb") + add :deceased_evidence_refs, :jsonb, null: false, default: fragment("'[]'::jsonb") + add :template_version, :text, null: false + add :created_by_actor_id, references(:actors, on_delete: :restrict), null: false + add :change_reason, :text, null: false + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:curator_profile_versions, [:profile_id, :version]) + create unique_index(:curator_profile_versions, [:profile_id, :manifest_hash]) + create unique_index(:curator_profile_versions, [:id, :profile_id]) + + create constraint(:curator_profile_versions, :curator_profile_versions_shape, + check: "version >= 1 AND btrim(change_reason) <> ''" + ) + + execute """ + ALTER TABLE curator_profiles + ADD CONSTRAINT curator_profiles_current_version_fkey + FOREIGN KEY (current_version_id, id) + REFERENCES curator_profile_versions (id, profile_id) + """, + "ALTER TABLE curator_profiles DROP CONSTRAINT IF EXISTS curator_profiles_current_version_fkey" + end + + # ── configurations (#201) ───────────────────────────────────────────────── + + defp configurations do + create table(:curation_configurations) do + add :slug, :text, null: false + add :name, :text, null: false + add :ownership_kind, :text, null: false, default: "system" + add :role, :text, null: false + add :state, :text, null: false, default: "draft" + add :current_version_id, :bigint + add :created_by_actor_id, references(:actors, on_delete: :restrict), null: false + + timestamps(type: :utc_datetime_usec) + end + + create unique_index(:curation_configurations, [:slug]) + + # C1: at most one *enabled* global default. + create unique_index(:curation_configurations, [:role], + where: "role = 'global_default' AND state = 'enabled'", + name: :curation_configurations_one_enabled_global_default + ) + + create constraint(:curation_configurations, :curation_configurations_shape, + check: """ + ownership_kind = 'system' AND + role IN ('global_default', 'internal_test') AND + state IN ('draft', 'enabled', 'disabled') AND + (state = 'draft') = (current_version_id IS NULL) + """ + ) + + create table(:curation_configuration_versions) do + add :configuration_id, references(:curation_configurations, on_delete: :restrict), + null: false + + add :version, :integer, null: false + add :manifest_hash, :text, null: false + add :manifest, :map, null: false + add :lead_policy, :text, null: false + add :max_highlights, :integer, null: false, default: 3 + add :roster_hash, :text, null: false + add :created_by_actor_id, references(:actors, on_delete: :restrict), null: false + add :change_reason, :text, null: false + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:curation_configuration_versions, [:configuration_id, :version]) + create unique_index(:curation_configuration_versions, [:configuration_id, :manifest_hash]) + create unique_index(:curation_configuration_versions, [:id, :configuration_id]) + + create constraint(:curation_configuration_versions, :curation_configuration_versions_shape, + check: """ + version >= 1 AND max_highlights BETWEEN 0 AND 3 AND + lead_policy IN ('bierce_first_v1') AND btrim(change_reason) <> '' + """ + ) + + # C2: the current version belongs to this configuration. + execute """ + ALTER TABLE curation_configurations + ADD CONSTRAINT curation_configurations_current_version_fkey + FOREIGN KEY (current_version_id, id) + REFERENCES curation_configuration_versions (id, configuration_id) + """, + "ALTER TABLE curation_configurations DROP CONSTRAINT IF EXISTS curation_configurations_current_version_fkey" + + create table(:curation_configuration_members) do + add :configuration_version_id, + references(:curation_configuration_versions, on_delete: :restrict), + null: false + + add :profile_id, references(:curator_profiles, on_delete: :restrict), null: false + add :profile_version_id, :bigint, null: false + add :slot, :integer, null: false + add :voting_weight, :integer, null: false, default: 1 + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + # C5: one profile and one slot per version; weight 1; the exact profile version. + create unique_index(:curation_configuration_members, [:configuration_version_id, :profile_id]) + create unique_index(:curation_configuration_members, [:configuration_version_id, :slot]) + + create constraint(:curation_configuration_members, :curation_configuration_members_shape, + check: "slot >= 1 AND voting_weight = 1" + ) + + execute """ + ALTER TABLE curation_configuration_members + ADD CONSTRAINT curation_configuration_members_profile_version_fkey + FOREIGN KEY (profile_version_id, profile_id) + REFERENCES curator_profile_versions (id, profile_id) + """, + "ALTER TABLE curation_configuration_members DROP CONSTRAINT IF EXISTS curation_configuration_members_profile_version_fkey" + + # C3: the only way a configuration's pointer or state moves. + create table(:curation_configuration_activations) do + add :configuration_id, references(:curation_configurations, on_delete: :restrict), + null: false + + add :action, :text, null: false + add :configuration_version_id, :bigint, null: false + add :previous_version_id, :bigint + add :actor_id, references(:actors, on_delete: :restrict), null: false + add :reason, :text, null: false + add :idempotency_key, :text, null: false + add :committed_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:curation_configuration_activations, [:idempotency_key]) + create index(:curation_configuration_activations, [:configuration_id, :id]) + + create constraint( + :curation_configuration_activations, + :curation_configuration_activations_shape, + check: "action IN ('activate', 'disable') AND btrim(reason) <> ''" + ) + + for column <- ~w(configuration_version_id previous_version_id) do + execute """ + ALTER TABLE curation_configuration_activations + ADD CONSTRAINT curation_configuration_activations_#{column}_fkey + FOREIGN KEY (#{column}, configuration_id) + REFERENCES curation_configuration_versions (id, configuration_id) + """, + "ALTER TABLE curation_configuration_activations DROP CONSTRAINT IF EXISTS curation_configuration_activations_#{column}_fkey" + end + end + + # ── compositions (#196) ─────────────────────────────────────────────────── + + defp compositions do + create table(:editorial_compositions) do + add :curation_configuration_id, + references(:curation_configurations, on_delete: :restrict), + null: false + + add :scope_kind, :text, null: false + add :language_tag, :text, null: false + add :scope_signature, :text, null: false + add :state, :text, null: false, default: "active" + add :current_published_version_id, :bigint + add :lock_version, :integer, null: false, default: 1 + add :created_by_actor_id, references(:actors, on_delete: :restrict), null: false + + timestamps(type: :utc_datetime_usec) + end + + # K1: the identity is (scope kind, signature, language, configuration). The + # configuration *version* is deliberately not part of it. + create unique_index( + :editorial_compositions, + [:scope_kind, :scope_signature, :language_tag, :curation_configuration_id], + where: "state = 'active'", + name: :editorial_compositions_active_scope_index + ) + + create unique_index(:editorial_compositions, [:id, :curation_configuration_id]) + + create constraint(:editorial_compositions, :editorial_compositions_shape, + check: """ + scope_kind IN ('lexical_page', 'lexeme') AND state IN ('active', 'retired') AND + btrim(language_tag) <> '' + """ + ) + + # A lexeme in a curated scope is not deleted from under it: the scope is + # changed or retired first (K3). Lexemes hold no source text, so R7 is not + # at stake here. + create table(:editorial_composition_memberships) do + add :composition_id, references(:editorial_compositions, on_delete: :restrict), null: false + add :object_id, references(:objects, on_delete: :restrict), null: false + add :role, :text, null: false + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:editorial_composition_memberships, [:composition_id, :object_id, :role]) + create index(:editorial_composition_memberships, [:object_id]) + + create constraint(:editorial_composition_memberships, :editorial_composition_memberships_role, + check: "role IN ('lexeme')" + ) + + create table(:editorial_composition_scope_changes) do + add :composition_id, references(:editorial_compositions, on_delete: :restrict), null: false + add :previous_signature, :text + add :scope_signature, :text, null: false + add :members, :jsonb, null: false + add :actor_id, references(:actors, on_delete: :restrict), null: false + add :reason, :text, null: false + add :changed_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create index(:editorial_composition_scope_changes, [:composition_id, :id]) + + create constraint( + :editorial_composition_scope_changes, + :editorial_composition_scope_changes_reason, + check: "btrim(reason) <> ''" + ) + + create table(:editorial_composition_versions) do + add :composition_id, :bigint, null: false + add :curation_configuration_id, :bigint, null: false + add :configuration_version_id, :bigint, null: false + add :version, :integer, null: false + add :parent_version_id, :bigint + add :origin, :text, null: false, default: "manual" + add :change_reason, :text, null: false + add :created_by_actor_id, references(:actors, on_delete: :restrict), null: false + add :scope_signature, :text, null: false + add :scope_members, :jsonb, null: false + add :resolution, :map, null: false + add :lead_policy, :text, null: false + add :arrangement_hash, :text, null: false + add :eligibility_fingerprint, :text, null: false + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:editorial_composition_versions, [:composition_id, :version]) + + create unique_index(:editorial_composition_versions, [ + :composition_id, + :arrangement_hash, + :eligibility_fingerprint + ]) + + create unique_index(:editorial_composition_versions, [:id, :composition_id]) + + # K9: manual only in this slice. A first version has no parent; every later + # one does. + create constraint(:editorial_composition_versions, :editorial_composition_versions_shape, + check: """ + version >= 1 AND origin = 'manual' AND btrim(change_reason) <> '' AND + lead_policy IN ('bierce_first_v1') AND + (version = 1) = (parent_version_id IS NULL) + """ + ) + + # K4: the same configuration and the same composition throughout. + execute """ + ALTER TABLE editorial_composition_versions + ADD CONSTRAINT editorial_composition_versions_composition_fkey + FOREIGN KEY (composition_id, curation_configuration_id) + REFERENCES editorial_compositions (id, curation_configuration_id), + ADD CONSTRAINT editorial_composition_versions_configuration_version_fkey + FOREIGN KEY (configuration_version_id, curation_configuration_id) + REFERENCES curation_configuration_versions (id, configuration_id), + ADD CONSTRAINT editorial_composition_versions_parent_fkey + FOREIGN KEY (parent_version_id, composition_id) + REFERENCES editorial_composition_versions (id, composition_id) + """, + """ + ALTER TABLE editorial_composition_versions + DROP CONSTRAINT IF EXISTS editorial_composition_versions_composition_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_versions_configuration_version_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_versions_parent_fkey + """ + + execute """ + ALTER TABLE editorial_compositions + ADD CONSTRAINT editorial_compositions_published_version_fkey + FOREIGN KEY (current_published_version_id, id) + REFERENCES editorial_composition_versions (id, composition_id) + """, + "ALTER TABLE editorial_compositions DROP CONSTRAINT IF EXISTS editorial_compositions_published_version_fkey" + + # Items store ids, hashes and locators, never source text (R7). Every + # reference to a source row yields to that row's deletion by nulling + # itself; the reader then withholds the item. + create table(:editorial_composition_items) do + add :composition_version_id, + references(:editorial_composition_versions, on_delete: :restrict), + null: false + + add :role, :text, null: false + add :position, :integer, null: false + add :item_kind, :text, null: false + add :item_object_id, :bigint + add :content_revision_id, :bigint + add :sense_revision_id, :bigint + add :source_record_revision_id, :bigint + add :catalog_manifest, :text + add :catalog_checksum, :text + add :catalog_identity, :text + add :locator, :text + add :words_sha256, :text + add :meaning_sense_revision_id, :bigint + add :meaning_lexeme_id, references(:lexemes, column: :object_id, on_delete: :restrict) + add :assertion_revision_id, :bigint + add :selection_origin, :text, null: false, default: "manual" + add :note, :text + add :note_author_kind, :text + add :note_author_label, :text + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + # K6: one lead at position 1, highlights at 1–3, one row per slot. + create unique_index(:editorial_composition_items, [:composition_version_id, :role, :position], + name: :editorial_composition_items_slot_index + ) + + create constraint(:editorial_composition_items, :editorial_composition_items_shape, + check: """ + role IN ('lead', 'highlight') AND + item_kind IN ('content', 'sense_quotation', 'work') AND + selection_origin = 'manual' AND + (role <> 'lead' OR (position = 1 AND item_kind = 'content')) AND + (role <> 'highlight' OR position BETWEEN 1 AND 3) AND + (item_kind = 'content' OR content_revision_id IS NULL) AND + (item_kind = 'sense_quotation' OR (sense_revision_id IS NULL AND locator IS NULL + AND words_sha256 IS NULL)) AND + (item_kind = 'work' OR (catalog_manifest IS NULL AND catalog_checksum IS NULL + AND catalog_identity IS NULL AND source_record_revision_id IS NULL)) AND + (item_kind <> 'sense_quotation' OR locator ~ '^quotation:[0-9]+$') AND + ((meaning_sense_revision_id IS NOT NULL)::int + (meaning_lexeme_id IS NOT NULL)::int) <= 1 AND + ((note IS NULL AND note_author_kind IS NULL AND note_author_label IS NULL) OR + (btrim(note) <> '' AND note_author_kind IN ('human', 'model') AND + btrim(note_author_label) <> '')) + """ + ) + + # K7 and R7: an exact revision belongs to its item's object, and every + # source reference nulls only its own column when its target is deleted. + execute """ + ALTER TABLE editorial_composition_items + ADD CONSTRAINT editorial_composition_items_object_fkey + FOREIGN KEY (item_object_id) REFERENCES objects (id) ON DELETE SET NULL, + ADD CONSTRAINT editorial_composition_items_content_revision_fkey + FOREIGN KEY (content_revision_id, item_object_id) + REFERENCES content_revisions (id, content_id) + ON DELETE SET NULL (content_revision_id), + ADD CONSTRAINT editorial_composition_items_sense_revision_fkey + FOREIGN KEY (sense_revision_id, item_object_id) + REFERENCES sense_revisions (id, sense_id) + ON DELETE SET NULL (sense_revision_id), + ADD CONSTRAINT editorial_composition_items_source_record_revision_fkey + FOREIGN KEY (source_record_revision_id) + REFERENCES source_record_revisions (id) ON DELETE SET NULL, + ADD CONSTRAINT editorial_composition_items_meaning_sense_revision_fkey + FOREIGN KEY (meaning_sense_revision_id) + REFERENCES sense_revisions (id) ON DELETE SET NULL, + ADD CONSTRAINT editorial_composition_items_assertion_revision_fkey + FOREIGN KEY (assertion_revision_id) + REFERENCES assertion_revisions (id) ON DELETE SET NULL + """, + """ + ALTER TABLE editorial_composition_items + DROP CONSTRAINT IF EXISTS editorial_composition_items_object_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_items_content_revision_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_items_sense_revision_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_items_source_record_revision_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_items_meaning_sense_revision_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_items_assertion_revision_fkey + """ + + create table(:editorial_composition_reviews) do + add :composition_id, :bigint, null: false + add :composition_version_id, :bigint, null: false + add :reviewer_actor_id, references(:actors, on_delete: :restrict), null: false + add :decision, :text, null: false + add :reason, :text, null: false + add :reviewed_eligibility_fingerprint, :text, null: false + add :idempotency_key, :text, null: false + add :inserted_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:editorial_composition_reviews, [:idempotency_key]) + create unique_index(:editorial_composition_reviews, [:id, :composition_version_id]) + create index(:editorial_composition_reviews, [:composition_version_id, :id]) + + create constraint(:editorial_composition_reviews, :editorial_composition_reviews_shape, + check: """ + decision IN ('accepted', 'rejected', 'withdrawn', 'needs_review') AND + btrim(reason) <> '' + """ + ) + + execute """ + ALTER TABLE editorial_composition_reviews + ADD CONSTRAINT editorial_composition_reviews_version_fkey + FOREIGN KEY (composition_version_id, composition_id) + REFERENCES editorial_composition_versions (id, composition_id) + """, + "ALTER TABLE editorial_composition_reviews DROP CONSTRAINT IF EXISTS editorial_composition_reviews_version_fkey" + + # R3, R4: the authoritative receipt, written in the pointer's transaction. + create table(:editorial_composition_publications) do + add :composition_id, references(:editorial_compositions, on_delete: :restrict), null: false + add :action, :text, null: false + add :previous_version_id, :bigint + add :published_version_id, :bigint + add :authorizing_review_id, :bigint + add :actor_id, references(:actors, on_delete: :restrict), null: false + add :reason, :text, null: false + add :eligibility_fingerprint, :text + add :idempotency_key, :text, null: false + add :committed_at, :utc_datetime_usec, null: false, default: fragment("now()") + end + + create unique_index(:editorial_composition_publications, [:idempotency_key]) + create index(:editorial_composition_publications, [:composition_id, :id]) + + create constraint( + :editorial_composition_publications, + :editorial_composition_publications_shape, + check: """ + btrim(reason) <> '' AND ( + (action = 'publish' AND published_version_id IS NOT NULL AND + authorizing_review_id IS NOT NULL AND eligibility_fingerprint IS NOT NULL) OR + (action = 'withdraw' AND previous_version_id IS NOT NULL AND + published_version_id IS NULL AND authorizing_review_id IS NULL AND + eligibility_fingerprint IS NULL) + ) + """ + ) + + execute """ + ALTER TABLE editorial_composition_publications + ADD CONSTRAINT editorial_composition_publications_previous_fkey + FOREIGN KEY (previous_version_id, composition_id) + REFERENCES editorial_composition_versions (id, composition_id), + ADD CONSTRAINT editorial_composition_publications_published_fkey + FOREIGN KEY (published_version_id, composition_id) + REFERENCES editorial_composition_versions (id, composition_id), + ADD CONSTRAINT editorial_composition_publications_review_fkey + FOREIGN KEY (authorizing_review_id, published_version_id) + REFERENCES editorial_composition_reviews (id, composition_version_id) + """, + """ + ALTER TABLE editorial_composition_publications + DROP CONSTRAINT IF EXISTS editorial_composition_publications_previous_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_publications_published_fkey, + DROP CONSTRAINT IF EXISTS editorial_composition_publications_review_fkey + """ + end + + # The targets of K7's composite foreign keys. `id` is already unique; these + # let an item name (revision, object) and have the database check the pair. + defp revision_ownership_indexes do + create unique_index(:content_revisions, [:id, :content_id], + name: :content_revisions_id_content_index + ) + + create unique_index(:sense_revisions, [:id, :sense_id], name: :sense_revisions_id_sense_index) + end + + # ── triggers ────────────────────────────────────────────────────────────── + + @immutable ~w(curator_profile_versions curation_configuration_versions + curation_configuration_members curation_configuration_activations + editorial_composition_scope_changes editorial_composition_versions + editorial_composition_reviews editorial_composition_publications) + + defp triggers do + immutability() + human_actor_checks() + insert_checks() + commit_checks() + end + + # C4, K5, R1, R3: history is appended, never edited. + defp immutability do + execute """ + CREATE FUNCTION curation_row_is_immutable() RETURNS trigger AS $$ + BEGIN + RAISE EXCEPTION '% rows are immutable (% on id %)', TG_TABLE_NAME, TG_OP, OLD.id + USING ERRCODE = 'integrity_constraint_violation'; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS curation_row_is_immutable() CASCADE" + + for table <- @immutable do + execute """ + CREATE TRIGGER #{table}_immutable BEFORE UPDATE OR DELETE ON #{table} + FOR EACH ROW EXECUTE FUNCTION curation_row_is_immutable(); + """, + "DROP TRIGGER IF EXISTS #{table}_immutable ON #{table}" + end + + # R7: an item never changes, except that a source reference may become + # NULL when its target is deleted. Nothing else about the row may move, and + # no reference may be set or repointed. + execute """ + CREATE FUNCTION editorial_composition_items_guard() RETURNS trigger AS $$ + DECLARE + refs text[] := ARRAY['item_object_id', 'content_revision_id', 'sense_revision_id', + 'source_record_revision_id', 'meaning_sense_revision_id', 'assertion_revision_id']; + ref text; + BEGIN + IF TG_OP = 'DELETE' THEN + RAISE EXCEPTION 'editorial_composition_items rows are immutable (DELETE on id %)', OLD.id + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + IF (to_jsonb(NEW) - refs) IS DISTINCT FROM (to_jsonb(OLD) - refs) THEN + RAISE EXCEPTION 'editorial_composition_items rows are immutable (UPDATE on id %)', OLD.id + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + FOREACH ref IN ARRAY refs LOOP + IF to_jsonb(NEW) -> ref <> 'null'::jsonb AND + (to_jsonb(NEW) -> ref) IS DISTINCT FROM (to_jsonb(OLD) -> ref) THEN + RAISE EXCEPTION 'editorial_composition_items.% may only be nulled by a deletion (id %)', ref, OLD.id + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + END LOOP; + + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_composition_items_guard() CASCADE" + + execute """ + CREATE TRIGGER editorial_composition_items_guard BEFORE UPDATE OR DELETE + ON editorial_composition_items + FOR EACH ROW EXECUTE FUNCTION editorial_composition_items_guard(); + """, + "DROP TRIGGER IF EXISTS editorial_composition_items_guard ON editorial_composition_items" + end + + # C6, C8, K9, R1, R3: who may do what. A bot never authors, reviews, + # activates or publishes, and only an account with the reviewer role + # reviews, activates or publishes. Checked on insert, from the rows as they + # stand then. + defp human_actor_checks do + execute """ + CREATE FUNCTION curation_actor_is(candidate bigint, want_reviewer boolean) RETURNS boolean AS $$ + SELECT EXISTS ( + SELECT 1 FROM actors a JOIN users u ON u.id = a.user_id + WHERE a.id = candidate AND a.actor_kind = 'user' + AND (NOT want_reviewer OR u.reviewer) + ); + $$ LANGUAGE sql STABLE; + """, + "DROP FUNCTION IF EXISTS curation_actor_is(bigint, boolean) CASCADE" + + execute """ + CREATE FUNCTION curation_human_actor_guard() RETURNS trigger AS $$ + DECLARE + candidate bigint := (to_jsonb(NEW) ->> TG_ARGV[0])::bigint; + want_reviewer boolean := TG_ARGV[1]::boolean; + BEGIN + IF NOT curation_actor_is(candidate, want_reviewer) THEN + RAISE EXCEPTION '%.% must be a human account%', TG_TABLE_NAME, TG_ARGV[0], + CASE WHEN want_reviewer THEN ' with the reviewer role' ELSE '' END + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS curation_human_actor_guard() CASCADE" + + for {table, column, reviewer} <- [ + {"curator_profile_versions", "created_by_actor_id", false}, + {"editorial_composition_versions", "created_by_actor_id", false}, + {"editorial_composition_scope_changes", "actor_id", false}, + {"editorial_composition_reviews", "reviewer_actor_id", true}, + {"editorial_composition_publications", "actor_id", true}, + {"curation_configuration_activations", "actor_id", true} + ] do + execute """ + CREATE TRIGGER #{table}_human_actor BEFORE INSERT ON #{table} + FOR EACH ROW EXECUTE FUNCTION curation_human_actor_guard('#{column}', '#{reviewer}'); + """, + "DROP TRIGGER IF EXISTS #{table}_human_actor ON #{table}" + end + + # C6: admission names a human reviewer, a bot principal, and a version + # that cites its sources and the subject's death. Admission covers one + # version: moving an admitted profile to another version is a new + # admission. + execute """ + CREATE FUNCTION curator_profiles_admission_guard() RETURNS trigger AS $$ + BEGIN + IF NEW.state = 'admitted' THEN + IF TG_OP = 'UPDATE' AND OLD.state = 'admitted' + AND NEW.current_version_id IS DISTINCT FROM OLD.current_version_id + AND NEW.admitted_at IS NOT DISTINCT FROM OLD.admitted_at THEN + RAISE EXCEPTION 'a new version of an admitted profile needs a new admission' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + IF NOT curation_actor_is(NEW.admitted_by_actor_id, true) THEN + RAISE EXCEPTION 'a profile is admitted by a human reviewer' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + IF NOT EXISTS (SELECT 1 FROM actors WHERE id = NEW.bot_actor_id AND actor_kind = 'bot') THEN + RAISE EXCEPTION 'an admitted profile acts through a bot actor' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + IF NOT EXISTS ( + SELECT 1 FROM curator_profile_versions v + WHERE v.id = NEW.current_version_id + AND jsonb_typeof(v.source_refs) = 'array' AND jsonb_array_length(v.source_refs) > 0 + AND jsonb_typeof(v.deceased_evidence_refs) = 'array' + AND jsonb_array_length(v.deceased_evidence_refs) > 0 + ) THEN + RAISE EXCEPTION 'an admitted profile''s version cites its sources and deceased-status evidence' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS curator_profiles_admission_guard() CASCADE" + + execute """ + CREATE TRIGGER curator_profiles_admission_guard BEFORE INSERT OR UPDATE ON curator_profiles + FOR EACH ROW EXECUTE FUNCTION curator_profiles_admission_guard(); + """, + "DROP TRIGGER IF EXISTS curator_profiles_admission_guard ON curator_profiles" + end + + defp insert_checks do + # K6, K7, K8: what an item must carry when it is written. The check + # constraint admits the NULLs a later deletion leaves behind, so presence + # is checked here, at insert only. + execute """ + CREATE FUNCTION editorial_composition_items_complete() RETURNS trigger AS $$ + BEGIN + -- A work pinned from a committed catalog may have no registry + -- object yet; everything else names the object it shows. + IF NEW.item_object_id IS NULL AND + NOT (NEW.item_kind = 'work' AND NEW.catalog_manifest IS NOT NULL) THEN + RAISE EXCEPTION 'an item names its registry object' + USING ERRCODE = 'check_violation'; + END IF; + + IF (NEW.meaning_sense_revision_id IS NULL) = (NEW.meaning_lexeme_id IS NULL) THEN + RAISE EXCEPTION 'an item has exactly one intended meaning' + USING ERRCODE = 'check_violation'; + END IF; + + IF (NEW.item_kind = 'content' AND NEW.content_revision_id IS NULL) + OR (NEW.item_kind = 'sense_quotation' AND + (NEW.sense_revision_id IS NULL OR NEW.words_sha256 IS NULL)) + OR (NEW.item_kind = 'work' AND + (NEW.source_record_revision_id IS NULL) = + (NEW.catalog_manifest IS NULL OR NEW.catalog_checksum IS NULL + OR NEW.catalog_identity IS NULL)) + THEN + RAISE EXCEPTION 'a % item names exactly one exact revision', NEW.item_kind + USING ERRCODE = 'check_violation'; + END IF; + + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_composition_items_complete() CASCADE" + + execute """ + CREATE TRIGGER editorial_composition_items_complete BEFORE INSERT + ON editorial_composition_items + FOR EACH ROW EXECUTE FUNCTION editorial_composition_items_complete(); + """, + "DROP TRIGGER IF EXISTS editorial_composition_items_complete ON editorial_composition_items" + + # K2, K4: a version is made against the composition's scope as it stands, + # and only while the composition is active. + execute """ + CREATE FUNCTION editorial_composition_versions_scope_guard() RETURNS trigger AS $$ + BEGIN + IF NOT EXISTS ( + SELECT 1 FROM editorial_compositions c + WHERE c.id = NEW.composition_id AND c.state = 'active' + AND c.scope_signature = NEW.scope_signature + ) THEN + RAISE EXCEPTION 'a version is made against its active composition''s current scope' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_composition_versions_scope_guard() CASCADE" + + execute """ + CREATE TRIGGER editorial_composition_versions_scope_guard BEFORE INSERT + ON editorial_composition_versions + FOR EACH ROW EXECUTE FUNCTION editorial_composition_versions_scope_guard(); + """, + "DROP TRIGGER IF EXISTS editorial_composition_versions_scope_guard ON editorial_composition_versions" + + # R1, R4: an acceptance is of the version's own eligibility. + execute """ + CREATE FUNCTION editorial_composition_reviews_guard() RETURNS trigger AS $$ + BEGIN + IF NEW.decision = 'accepted' AND NOT EXISTS ( + SELECT 1 FROM editorial_composition_versions v + WHERE v.id = NEW.composition_version_id + AND v.eligibility_fingerprint = NEW.reviewed_eligibility_fingerprint + ) THEN + RAISE EXCEPTION 'an acceptance reviews the version''s own eligibility fingerprint' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_composition_reviews_guard() CASCADE" + + execute """ + CREATE TRIGGER editorial_composition_reviews_guard BEFORE INSERT + ON editorial_composition_reviews + FOR EACH ROW EXECUTE FUNCTION editorial_composition_reviews_guard(); + """, + "DROP TRIGGER IF EXISTS editorial_composition_reviews_guard ON editorial_composition_reviews" + + # R3: a publication is authorized by the *latest* decision on that very + # version, which is an acceptance of the fingerprint being published. + execute """ + CREATE FUNCTION editorial_composition_publications_review_guard() RETURNS trigger AS $$ + DECLARE + latest record; + BEGIN + IF NEW.action = 'publish' THEN + SELECT r.id, r.decision, r.reviewed_eligibility_fingerprint INTO latest + FROM editorial_composition_reviews r + WHERE r.composition_version_id = NEW.published_version_id + ORDER BY r.id DESC LIMIT 1; + + IF latest.id IS DISTINCT FROM NEW.authorizing_review_id + OR latest.decision IS DISTINCT FROM 'accepted' + OR latest.reviewed_eligibility_fingerprint IS DISTINCT FROM NEW.eligibility_fingerprint THEN + RAISE EXCEPTION 'a publication is authorized by the latest review of its version, an acceptance of this fingerprint' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_composition_publications_review_guard() CASCADE" + + execute """ + CREATE TRIGGER editorial_composition_publications_review_guard BEFORE INSERT + ON editorial_composition_publications + FOR EACH ROW EXECUTE FUNCTION editorial_composition_publications_review_guard(); + """, + "DROP TRIGGER IF EXISTS editorial_composition_publications_review_guard ON editorial_composition_publications" + end + + # Checked at COMMIT, from the rows as they then stand. Each function takes, + # as its trigger argument, the column of the triggering row that names the + # parent it checks: a PL/pgSQL expression cannot name a column the row's + # table lacks, even in a branch it never takes. + defp commit_checks do + # C3: a configuration's pointer and state are its latest activation's, and + # each activation continues from the one before it. + execute """ + CREATE FUNCTION curation_configuration_receipt_check() RETURNS trigger AS $$ + DECLARE + parent bigint := (to_jsonb(NEW) ->> TG_ARGV[0])::bigint; + c record; + latest record; + prior bigint; + BEGIN + SELECT state, current_version_id INTO c FROM curation_configurations WHERE id = parent; + + SELECT action, configuration_version_id, previous_version_id INTO latest + FROM curation_configuration_activations + WHERE configuration_id = parent ORDER BY id DESC LIMIT 1; + + SELECT configuration_version_id INTO prior + FROM curation_configuration_activations + WHERE configuration_id = parent ORDER BY id DESC OFFSET 1 LIMIT 1; + + IF latest.action IS NULL THEN + IF c.current_version_id IS NOT NULL OR c.state <> 'draft' THEN + RAISE EXCEPTION 'configuration % left draft without an activation receipt', parent + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + ELSIF c.current_version_id IS DISTINCT FROM latest.configuration_version_id + OR c.state IS DISTINCT FROM + (CASE latest.action WHEN 'activate' THEN 'enabled' ELSE 'disabled' END) + OR latest.previous_version_id IS DISTINCT FROM prior THEN + RAISE EXCEPTION 'configuration % does not match its latest activation receipt', parent + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS curation_configuration_receipt_check() CASCADE" + + for {table, column, events} <- [ + {"curation_configurations", "id", "INSERT OR UPDATE"}, + {"curation_configuration_activations", "configuration_id", "INSERT"} + ] do + execute """ + CREATE CONSTRAINT TRIGGER #{table}_receipt AFTER #{events} ON #{table} + DEFERRABLE INITIALLY DEFERRED + FOR EACH ROW EXECUTE FUNCTION curation_configuration_receipt_check('#{column}'); + """, + "DROP TRIGGER IF EXISTS #{table}_receipt ON #{table}" + end + + # R3, R5: a composition's pointer is its latest publication receipt's, and + # each receipt continues from the one before it. Two writers that both read + # the same pointer cannot both commit: whichever commits second finds that + # its receipt no longer continues the chain. + execute """ + CREATE FUNCTION editorial_composition_receipt_check() RETURNS trigger AS $$ + DECLARE + parent bigint := (to_jsonb(NEW) ->> TG_ARGV[0])::bigint; + pointer bigint; + latest record; + prior bigint; + BEGIN + SELECT current_published_version_id INTO pointer FROM editorial_compositions WHERE id = parent; + + SELECT id, published_version_id, previous_version_id INTO latest + FROM editorial_composition_publications + WHERE composition_id = parent ORDER BY id DESC LIMIT 1; + + SELECT published_version_id INTO prior + FROM editorial_composition_publications + WHERE composition_id = parent ORDER BY id DESC OFFSET 1 LIMIT 1; + + IF pointer IS DISTINCT FROM latest.published_version_id + OR (latest.id IS NOT NULL AND latest.previous_version_id IS DISTINCT FROM prior) THEN + RAISE EXCEPTION 'composition % does not match its latest publication receipt', parent + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_composition_receipt_check() CASCADE" + + for {table, column, events} <- [ + {"editorial_compositions", "id", "INSERT OR UPDATE"}, + {"editorial_composition_publications", "composition_id", "INSERT"} + ] do + execute """ + CREATE CONSTRAINT TRIGGER #{table}_receipt AFTER #{events} ON #{table} + DEFERRABLE INITIALLY DEFERRED + FOR EACH ROW EXECUTE FUNCTION editorial_composition_receipt_check('#{column}'); + """, + "DROP TRIGGER IF EXISTS #{table}_receipt ON #{table}" + end + + # K2, K3: a composition's signature is its members', it is the latest + # scope-change receipt's, each receipt continues the one before it, and a + # single-lexeme scope has exactly one member. + execute """ + CREATE FUNCTION editorial_scope_signature(composition bigint) RETURNS text AS $$ + SELECT encode(sha256(convert_to(COALESCE(string_agg( + m.role || ':' || m.object_id, ',' ORDER BY m.role, m.object_id), ''), 'UTF8')), 'hex') + FROM editorial_composition_memberships m + WHERE m.composition_id = composition; + $$ LANGUAGE sql STABLE; + """, + "DROP FUNCTION IF EXISTS editorial_scope_signature(bigint) CASCADE" + + execute """ + CREATE FUNCTION editorial_scope_check() RETURNS trigger AS $$ + DECLARE + parent bigint := (COALESCE(to_jsonb(NEW), to_jsonb(OLD)) ->> TG_ARGV[0])::bigint; + c record; + latest record; + prior text; + members integer; + BEGIN + SELECT scope_kind, scope_signature INTO c FROM editorial_compositions WHERE id = parent; + IF NOT FOUND THEN RETURN NULL; END IF; + + SELECT count(*) INTO members FROM editorial_composition_memberships WHERE composition_id = parent; + + SELECT scope_signature, previous_signature INTO latest + FROM editorial_composition_scope_changes + WHERE composition_id = parent ORDER BY id DESC LIMIT 1; + + SELECT scope_signature INTO prior + FROM editorial_composition_scope_changes + WHERE composition_id = parent ORDER BY id DESC OFFSET 1 LIMIT 1; + + IF c.scope_signature IS DISTINCT FROM editorial_scope_signature(parent) + OR c.scope_signature IS DISTINCT FROM latest.scope_signature + OR latest.previous_signature IS DISTINCT FROM prior + OR members = 0 + OR (c.scope_kind = 'lexeme' AND members <> 1) THEN + RAISE EXCEPTION 'composition % scope does not match its members and latest scope-change receipt', parent + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_scope_check() CASCADE" + + for {table, column, events} <- [ + {"editorial_compositions", "id", "INSERT OR UPDATE"}, + {"editorial_composition_memberships", "composition_id", "INSERT OR DELETE"}, + {"editorial_composition_scope_changes", "composition_id", "INSERT"} + ] do + execute """ + CREATE CONSTRAINT TRIGGER #{table}_scope AFTER #{events} ON #{table} + DEFERRABLE INITIALLY DEFERRED + FOR EACH ROW EXECUTE FUNCTION editorial_scope_check('#{column}'); + """, + "DROP TRIGGER IF EXISTS #{table}_scope ON #{table}" + end + + # C4: a roster is exactly the members its version was created with. + execute """ + CREATE FUNCTION curation_roster_hash(version_id bigint) RETURNS text AS $$ + SELECT encode(sha256(convert_to(COALESCE(string_agg( + m.slot || ':' || m.profile_id || ':' || m.profile_version_id || ':' || m.voting_weight, + ',' ORDER BY m.slot), ''), 'UTF8')), 'hex') + FROM curation_configuration_members m + WHERE m.configuration_version_id = version_id; + $$ LANGUAGE sql STABLE; + """, + "DROP FUNCTION IF EXISTS curation_roster_hash(bigint) CASCADE" + + execute """ + CREATE FUNCTION curation_roster_check() RETURNS trigger AS $$ + DECLARE + version_id bigint := (to_jsonb(NEW) ->> TG_ARGV[0])::bigint; + BEGIN + IF (SELECT roster_hash FROM curation_configuration_versions WHERE id = version_id) + IS DISTINCT FROM curation_roster_hash(version_id) THEN + RAISE EXCEPTION 'configuration version % roster differs from the roster it was created with', version_id + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS curation_roster_check() CASCADE" + + for {table, column} <- [ + {"curation_configuration_versions", "id"}, + {"curation_configuration_members", "configuration_version_id"} + ] do + execute """ + CREATE CONSTRAINT TRIGGER #{table}_roster AFTER INSERT ON #{table} + DEFERRABLE INITIALLY DEFERRED + FOR EACH ROW EXECUTE FUNCTION curation_roster_check('#{column}'); + """, + "DROP TRIGGER IF EXISTS #{table}_roster ON #{table}" + end + + # K5: an arrangement is exactly the items its version was created with. + # Each field is length-prefixed (NULL is `~`), so no note or locator can + # make two different arrangements hash alike. + # `DevilsDictionary.Curation.Digest` computes the same digest. + execute """ + CREATE FUNCTION curation_field(value text) RETURNS text AS $$ + SELECT CASE WHEN value IS NULL THEN '~' ELSE octet_length(value) || ':' || value END; + $$ LANGUAGE sql IMMUTABLE; + """, + "DROP FUNCTION IF EXISTS curation_field(text) CASCADE" + + execute """ + CREATE FUNCTION editorial_arrangement_hash(version_id bigint) RETURNS text AS $$ + SELECT encode(sha256(convert_to(COALESCE(string_agg( + curation_field(i.role) || curation_field(i.position::text) || + curation_field(i.item_kind) || curation_field(i.item_object_id::text) || + curation_field(i.content_revision_id::text) || curation_field(i.sense_revision_id::text) || + curation_field(i.source_record_revision_id::text) || curation_field(i.catalog_manifest) || + curation_field(i.catalog_checksum) || curation_field(i.catalog_identity) || + curation_field(i.locator) || curation_field(i.words_sha256) || + curation_field(i.meaning_sense_revision_id::text) || curation_field(i.meaning_lexeme_id::text) || + curation_field(i.assertion_revision_id::text) || curation_field(i.selection_origin) || + curation_field(i.note) || curation_field(i.note_author_kind) || + curation_field(i.note_author_label), + '' ORDER BY i.role, i.position), ''), 'UTF8')), 'hex') + FROM editorial_composition_items i + WHERE i.composition_version_id = version_id; + $$ LANGUAGE sql STABLE; + """, + "DROP FUNCTION IF EXISTS editorial_arrangement_hash(bigint) CASCADE" + + execute """ + CREATE FUNCTION editorial_arrangement_check() RETURNS trigger AS $$ + DECLARE + version_id bigint := (to_jsonb(NEW) ->> TG_ARGV[0])::bigint; + BEGIN + IF (SELECT arrangement_hash FROM editorial_composition_versions WHERE id = version_id) + IS DISTINCT FROM editorial_arrangement_hash(version_id) THEN + RAISE EXCEPTION 'composition version % arrangement differs from the one it was created with', version_id + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + """, + "DROP FUNCTION IF EXISTS editorial_arrangement_check() CASCADE" + + for {table, column} <- [ + {"editorial_composition_versions", "id"}, + {"editorial_composition_items", "composition_version_id"} + ] do + execute """ + CREATE CONSTRAINT TRIGGER #{table}_arrangement AFTER INSERT ON #{table} + DEFERRABLE INITIALLY DEFERRED + FOR EACH ROW EXECUTE FUNCTION editorial_arrangement_check('#{column}'); + """, + "DROP TRIGGER IF EXISTS #{table}_arrangement ON #{table}" + end + end +end diff --git a/priv/repo/migrations/20260927094256_repair_curation_integrity.exs b/priv/repo/migrations/20260927094256_repair_curation_integrity.exs new file mode 100644 index 00000000..f63432a4 --- /dev/null +++ b/priv/repo/migrations/20260927094256_repair_curation_integrity.exs @@ -0,0 +1,309 @@ +defmodule DevilsDictionary.Repo.Migrations.RepairCurationIntegrity do + @moduledoc """ + Repairs to the curation foundation from the independent audit of PR #206 + (#196, #201). It is a separate migration, not an edit of + `20260926233642_create_curation_foundation`, so that every database that + already applied that one, including test partitions, is repaired by + `ecto.migrate` rather than silently left behind. + + 1. **A deleted reference is not a missing one.** Each item records, at + insert, which of its references it was made with + (`required_references`). A reference that deletion later nulls is then + distinguishable from one the item never had, so deleting a claim or an + object withholds the item rather than making it eligible again. + 2. **Version deduplication includes the frozen configuration version.** + The same arrangement can be made again under a new configuration + version. It then needs its own review. + 3. **A note's author is an actor, not a caller-supplied label.** + `note_author_actor_id` names who wrote it. A human note is its + version's author's, and the stored label is that actor's. + 4. **Every publication receipt names its authority.** `authority_kind` is + `operator` (an accepted operator review) and nothing else in this + slice. The panel-decision authority described in + `docs/curation/persistence-slice-1.md` needs decision records that do + not exist yet (#197). Until a later migration adds them, the database + refuses it. + """ + use Ecto.Migration + + # The references an item was made with. `meaning_lexeme_id` is absent: it is + # RESTRICT, so a deletion never nulls it. + @references_function """ + CREATE FUNCTION curation_item_references(i editorial_composition_items) RETURNS text[] AS $$ + SELECT array_remove(ARRAY[ + CASE WHEN i.item_object_id IS NOT NULL THEN 'item_object_id' END, + CASE WHEN i.content_revision_id IS NOT NULL THEN 'content_revision_id' END, + CASE WHEN i.sense_revision_id IS NOT NULL THEN 'sense_revision_id' END, + CASE WHEN i.source_record_revision_id IS NOT NULL THEN 'source_record_revision_id' END, + CASE WHEN i.meaning_sense_revision_id IS NOT NULL THEN 'meaning_sense_revision_id' END, + CASE WHEN i.assertion_revision_id IS NOT NULL THEN 'assertion_revision_id' END + ], NULL); + $$ LANGUAGE sql IMMUTABLE; + """ + + @complete_checks """ + -- A work pinned from a committed catalog may have no registry + -- object yet; everything else names the object it shows. + IF NEW.item_object_id IS NULL AND + NOT (NEW.item_kind = 'work' AND NEW.catalog_manifest IS NOT NULL) THEN + RAISE EXCEPTION 'an item names its registry object' + USING ERRCODE = 'check_violation'; + END IF; + + IF (NEW.meaning_sense_revision_id IS NULL) = (NEW.meaning_lexeme_id IS NULL) THEN + RAISE EXCEPTION 'an item has exactly one intended meaning' + USING ERRCODE = 'check_violation'; + END IF; + + IF (NEW.item_kind = 'content' AND NEW.content_revision_id IS NULL) + OR (NEW.item_kind = 'sense_quotation' AND + (NEW.sense_revision_id IS NULL OR NEW.words_sha256 IS NULL)) + OR (NEW.item_kind = 'work' AND + (NEW.source_record_revision_id IS NULL) = + (NEW.catalog_manifest IS NULL OR NEW.catalog_checksum IS NULL + OR NEW.catalog_identity IS NULL)) + THEN + RAISE EXCEPTION 'a % item names exactly one exact revision', NEW.item_kind + USING ERRCODE = 'check_violation'; + END IF; + """ + + @complete_function_v1 """ + CREATE OR REPLACE FUNCTION editorial_composition_items_complete() RETURNS trigger AS $$ + BEGIN + #{@complete_checks} + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """ + + # K7, K8, R6, R7 and note authorship. The database, not the caller, records + # which references the item was made with, and checks who wrote its note: a + # human note is its version's author's, under that actor's own label; a + # model note (none is written in this slice) is a bot's. + @complete_function_v2 """ + CREATE OR REPLACE FUNCTION editorial_composition_items_complete() RETURNS trigger AS $$ + DECLARE + author record; + BEGIN + #{@complete_checks} + NEW.required_references := curation_item_references(NEW); + + IF NEW.note IS NULL THEN + IF NEW.note_author_actor_id IS NOT NULL THEN + RAISE EXCEPTION 'an item without a note has no note author' + USING ERRCODE = 'check_violation'; + END IF; + ELSE + SELECT a.id, a.actor_kind, COALESCE(a.label, 'Account #' || a.id) AS label INTO author + FROM actors a WHERE a.id = NEW.note_author_actor_id; + + IF author.id IS NULL THEN + RAISE EXCEPTION 'a note names the actor who wrote it' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + IF NEW.note_author_kind = 'human' AND (author.actor_kind <> 'user' OR author.id IS DISTINCT FROM + (SELECT created_by_actor_id FROM editorial_composition_versions WHERE id = NEW.composition_version_id)) THEN + RAISE EXCEPTION 'a human note is written by its version''s author' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + IF NEW.note_author_kind = 'model' AND author.actor_kind <> 'bot' THEN + RAISE EXCEPTION 'a model note is written by a bot actor' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + IF NEW.note_author_label IS DISTINCT FROM author.label THEN + RAISE EXCEPTION 'a note is attributed under its author''s own label' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + END IF; + + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """ + + # R1, R3, R5: who may publish under which authority. An operator publication + # is a reviewer's, authorized by the latest review of that very version, an + # acceptance of the fingerprint published; an operator withdrawal is a + # reviewer's. Any other authority is refused: a panel decision needs the + # decision records a later migration adds (#197). + @authority_function """ + CREATE FUNCTION editorial_composition_publications_authority() RETURNS trigger AS $$ + DECLARE + latest record; + BEGIN + IF NEW.authority_kind IS DISTINCT FROM 'operator' THEN + RAISE EXCEPTION 'publication authority % is not available', NEW.authority_kind + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + IF NOT curation_actor_is(NEW.actor_id, true) THEN + RAISE EXCEPTION 'an operator publication is made by a human account with the reviewer role' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + + IF NEW.action = 'publish' THEN + SELECT r.id, r.decision, r.reviewed_eligibility_fingerprint INTO latest + FROM editorial_composition_reviews r + WHERE r.composition_version_id = NEW.published_version_id + ORDER BY r.id DESC LIMIT 1; + + IF latest.id IS DISTINCT FROM NEW.authorizing_review_id + OR latest.decision IS DISTINCT FROM 'accepted' + OR latest.reviewed_eligibility_fingerprint IS DISTINCT FROM NEW.eligibility_fingerprint THEN + RAISE EXCEPTION 'a publication is authorized by the latest review of its version, an acceptance of this fingerprint' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + END IF; + + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """ + + @review_guard_function_v1 """ + CREATE FUNCTION editorial_composition_publications_review_guard() RETURNS trigger AS $$ + DECLARE + latest record; + BEGIN + IF NEW.action = 'publish' THEN + SELECT r.id, r.decision, r.reviewed_eligibility_fingerprint INTO latest + FROM editorial_composition_reviews r + WHERE r.composition_version_id = NEW.published_version_id + ORDER BY r.id DESC LIMIT 1; + + IF latest.id IS DISTINCT FROM NEW.authorizing_review_id + OR latest.decision IS DISTINCT FROM 'accepted' + OR latest.reviewed_eligibility_fingerprint IS DISTINCT FROM NEW.eligibility_fingerprint THEN + RAISE EXCEPTION 'a publication is authorized by the latest review of its version, an acceptance of this fingerprint' + USING ERRCODE = 'integrity_constraint_violation'; + END IF; + END IF; + RETURN NEW; + END; + $$ LANGUAGE plpgsql; + """ + + def up do + alter table(:editorial_composition_items) do + add :required_references, {:array, :text} + add :note_author_actor_id, references(:actors, on_delete: :restrict) + end + + execute @references_function + execute @complete_function_v2 + + # Existing rows: backfill what they were made with. The item guard forbids + # every non-reference change, so it steps aside for this one statement. + # A reference an earlier deletion already nulled cannot be recovered here. + execute "ALTER TABLE editorial_composition_items DISABLE TRIGGER editorial_composition_items_guard" + + execute """ + UPDATE editorial_composition_items i + SET required_references = curation_item_references(i), + note_author_actor_id = CASE WHEN i.note IS NULL THEN NULL ELSE v.created_by_actor_id END + FROM editorial_composition_versions v + WHERE v.id = i.composition_version_id + """ + + execute "ALTER TABLE editorial_composition_items ENABLE TRIGGER editorial_composition_items_guard" + + execute "ALTER TABLE editorial_composition_items ALTER COLUMN required_references SET NOT NULL" + + create constraint(:editorial_composition_items, :editorial_composition_items_note_author, + check: "(note IS NULL) = (note_author_actor_id IS NULL)" + ) + + drop index(:editorial_composition_versions, [ + :composition_id, + :arrangement_hash, + :eligibility_fingerprint + ]) + + create unique_index( + :editorial_composition_versions, + [ + :composition_id, + :configuration_version_id, + :arrangement_hash, + :eligibility_fingerprint + ], + name: :editorial_composition_versions_dedup_index + ) + + alter table(:editorial_composition_publications) do + add :authority_kind, :text, null: false, default: "operator" + end + + execute "ALTER TABLE editorial_composition_publications ALTER COLUMN authority_kind DROP DEFAULT" + + create constraint( + :editorial_composition_publications, + :editorial_composition_publications_authority, + check: "authority_kind IN ('operator')" + ) + + execute "DROP TRIGGER editorial_composition_publications_human_actor ON editorial_composition_publications" + execute @authority_function + + execute "DROP TRIGGER editorial_composition_publications_review_guard ON editorial_composition_publications" + + execute "DROP FUNCTION editorial_composition_publications_review_guard()" + + execute """ + CREATE TRIGGER editorial_composition_publications_authority BEFORE INSERT + ON editorial_composition_publications + FOR EACH ROW EXECUTE FUNCTION editorial_composition_publications_authority(); + """ + end + + def down do + execute "DROP TRIGGER editorial_composition_publications_authority ON editorial_composition_publications" + execute "DROP FUNCTION editorial_composition_publications_authority()" + execute @review_guard_function_v1 + + execute """ + CREATE TRIGGER editorial_composition_publications_review_guard BEFORE INSERT + ON editorial_composition_publications + FOR EACH ROW EXECUTE FUNCTION editorial_composition_publications_review_guard(); + """ + + execute """ + CREATE TRIGGER editorial_composition_publications_human_actor BEFORE INSERT + ON editorial_composition_publications + FOR EACH ROW EXECUTE FUNCTION curation_human_actor_guard('actor_id', 'true'); + """ + + drop constraint( + :editorial_composition_publications, + :editorial_composition_publications_authority + ) + + alter table(:editorial_composition_publications) do + remove :authority_kind + end + + drop index(:editorial_composition_versions, [], + name: :editorial_composition_versions_dedup_index + ) + + create unique_index(:editorial_composition_versions, [ + :composition_id, + :arrangement_hash, + :eligibility_fingerprint + ]) + + drop constraint(:editorial_composition_items, :editorial_composition_items_note_author) + execute @complete_function_v1 + + alter table(:editorial_composition_items) do + remove :note_author_actor_id + remove :required_references + end + + execute "DROP FUNCTION curation_item_references(editorial_composition_items)" + end +end diff --git a/test/devils_dictionary/curation/audit_findings_test.exs b/test/devils_dictionary/curation/audit_findings_test.exs new file mode 100644 index 00000000..2543d830 --- /dev/null +++ b/test/devils_dictionary/curation/audit_findings_test.exs @@ -0,0 +1,610 @@ +defmodule DevilsDictionary.Curation.AuditFindingsTest do + @moduledoc """ + Regressions for the independent audit of PR #206 (at `b8211ed`), one + `describe` per finding. Each one runs the whole sequence the audit + reproduced: creation, review, publication and reading, then the change + that exposed the defect, then reading again. + + 1. A deleted claim revision must not make its dependent item eligible + again. + 2. A rejected or withdrawn `defines` claim must not authorize a lead. + 3. A new configuration version must allow an unchanged arrangement to be + reissued, under a fresh review. + 4. Work evidence must be evidence of the selected work. + 5. A note is attributed to its authenticated author, never to a + caller-supplied name. + 6. Every publication names its authority. Only the operator authority + exists, and missing, forged, stale and cross-configuration + authorizations are refused. + """ + use DevilsDictionary.DataCase, async: true + + import DevilsDictionary.CurationFixtures + + alias DevilsDictionary.Claims + alias DevilsDictionary.Claims.AssertionRevision + + alias DevilsDictionary.Curation.{ + CompositionItem, + CompositionPublication, + CompositionVersion, + Compositions, + Configurations, + LeadRule, + Published, + Publications, + Reviews + } + + alias DevilsDictionary.WordFixtures + + setup do + world = world!() + {config, config_version} = enabled_test_configuration!(world.reviewer, "audit") + + {:ok, composition} = + Compositions.provision(world.contributor, config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [world.love.object_id], + language_tag: "en", + reason: "curate love" + }) + + Map.merge(world, %{config: config, config_version: config_version, composition: composition}) + end + + defp create(ctx, attrs, composition \\ nil) do + Compositions.create_version( + ctx.contributor, + (composition || ctx.composition).id, + Map.merge( + %{lead: content_spec(ctx.bierce, ctx.love), reason: "audit", expected_parent: nil}, + attrs + ) + ) + end + + defp version!(ctx, attrs, composition \\ nil) do + {:ok, v} = create(ctx, attrs, composition) + v + end + + defp accept!(ctx, version) do + {:ok, review} = + Reviews.decide(ctx.reviewer, version.id, :accepted, reason: "audit", idempotency_key: key()) + + review + end + + defp publish(ctx, version, expected \\ nil) do + Publications.publish(ctx.reviewer, version.id, + reason: "audit", + idempotency_key: key(), + expected_pointer: expected + ) + end + + defp publish!(ctx, version, expected \\ nil) do + accept!(ctx, version) + {:ok, receipt} = publish(ctx, version, expected) + settle!() + receipt + end + + defp current!(ctx, composition \\ nil) do + {:ok, published} = Published.current((composition || ctx.composition).id) + published + end + + defp raw_receipt(ctx, attrs) do + struct( + CompositionPublication, + Map.merge( + %{ + composition_id: ctx.composition.id, + action: :publish, + authority_kind: :operator, + published_version_id: ctx.v.id, + authorizing_review_id: ctx.review.id, + actor_id: actor!(ctx.reviewer).id, + reason: "raw", + eligibility_fingerprint: ctx.v.eligibility_fingerprint, + idempotency_key: key("raw") + }, + attrs + ) + ) + end + + # Finding 2: v1 (the Bierce lead, the definition as a highlight) is + # published, and v2 (the same lead, a quotation added) is accepted and + # waiting. Then the defining claim changes. + defp published_and_waiting!(ctx) do + v1 = version!(ctx, %{highlights: [content_spec(ctx.definition, ctx.love)]}) + publish!(ctx, v1) + + v2 = + version!(ctx, %{ + highlights: [content_spec(ctx.definition, ctx.love), quotation_spec(ctx.sense, 1)], + expected_parent: v1.id + }) + + accept!(ctx, v2) + {v1, v2} + end + + defp assert_lead_lost!(ctx, v1, v2) do + assert LeadRule.applicable([ctx.love.object_id]) == [] + + # Reading: the published lead is withheld, and nothing takes its place. + published = current!(ctx) + assert published.version.id == v1.id + assert published.lead == nil + assert published.withheld == [%{role: :lead, position: 1, reason: :lead_not_on_scope}] + assert [%{item_object_id: id}] = published.highlights + assert id == ctx.definition.object_id + + # Publication: an acceptance made before the change authorizes nothing now. + assert {:error, :lead_not_on_scope} = publish(ctx, v2, v1.id) + + # Creation: the entry cannot lead a new version either. + assert {:error, :lead_not_on_scope} = create(ctx, %{reason: "again", expected_parent: v2.id}) + + # Another definition on the page may lead instead, as a new version with + # its own review, never by substitution. + fallback = + version!(ctx, %{lead: content_spec(ctx.definition, ctx.love), expected_parent: v2.id}) + + assert fallback.resolution["lead_rule"] == "manual_fallback" + assert {:error, :not_approved} = publish(ctx, fallback, v1.id) + assert current!(ctx).version.id == v1.id + settle!() + end + + describe "finding 1: deleted claim evidence" do + test "a deleted claim keeps its item withheld; an item made without one is unaffected", ctx do + claim = defines_claim!(ctx.definition, ctx.love) + + highlight = + Map.put(content_spec(ctx.definition, ctx.love), :assertion_revision_id, claim.id) + + v = version!(ctx, %{highlights: [highlight]}) + publish!(ctx, v) + + assert [%{assertion_revision_id: pinned}] = current!(ctx).highlights + assert pinned == claim.id + + # The claim is revised, so the pinned revision is history and the item + # it supported is withheld. + {:ok, _} = Claims.revise(claim.assertion_id, %{rationale: "a replacement revision"}) + settle!() + + assert %{ + highlights: [], + withheld: [%{role: :highlight, position: 1, reason: :claim_not_visible}] + } = + current!(ctx) + + # A mandatory deletion of that historical revision is not blocked... + assert {1, _} = Repo.delete_all(from r in AssertionRevision, where: r.id == ^claim.id) + settle!() + + [lead, item] = Enum.sort_by(Compositions.items(v.id), &(&1.role == :highlight)) + assert item.assertion_revision_id == nil + assert "assertion_revision_id" in item.required_references + refute "assertion_revision_id" in lead.required_references + + # ...and it does not make the item eligible again. + published = current!(ctx) + assert published.highlights == [] + assert published.withheld == [%{role: :highlight, position: 1, reason: :claim_deleted}] + assert published.lead.item_object_id == ctx.bierce.object_id + end + + test "a deleted object withholds its catalog work instead of leaving it catalog-only", ctx do + work = catalog_work!() + + v = + version!(ctx, %{highlights: [Map.put(catalog_spec(ctx.love), :object_id, work.object_id)]}) + + publish!(ctx, v) + assert [%{item_object_id: id}] = current!(ctx).highlights + assert id == work.object_id + + assert {1, _} = Repo.delete_all(from o in "objects", where: o.id == ^work.object_id) + settle!() + + assert %{ + highlights: [], + withheld: [%{role: :highlight, position: 1, reason: :object_deleted}] + } = + current!(ctx) + end + end + + describe "finding 2: the defining claim's review" do + test "a rejected defining claim loses its lead at read, publication and creation", ctx do + {v1, v2} = published_and_waiting!(ctx) + claim = defines_claim!(ctx.bierce, ctx.love) + + {:ok, _} = Claims.review(claim.id, :rejected, %{reason: "not this word"}) + + refute Repo.exists?( + Claims.visible(from(r in AssertionRevision, where: r.id == ^claim.id), :public) + ) + + assert_lead_lost!(ctx, v1, v2) + end + + test "a withdrawn defining claim loses its lead at read, publication and creation", ctx do + {v1, v2} = published_and_waiting!(ctx) + claim = defines_claim!(ctx.bierce, ctx.love) + + {:ok, _} = Claims.withdraw(claim.assertion_id, reason: "the source withdrew it") + + assert_lead_lost!(ctx, v1, v2) + end + + test "a fallback definition needs a visible claim too", ctx do + {:ok, oats} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.oats.object_id], + language_tag: "en", + reason: "oats" + }) + + v = version!(ctx, %{lead: content_spec(ctx.oats_definition, ctx.oats)}, oats) + publish!(ctx, v) + + {:ok, _} = + Claims.review(defines_claim!(ctx.oats_definition, ctx.oats).id, :rejected, %{ + reason: "not oats" + }) + + assert %{lead: nil, withheld: [%{role: :lead, reason: :lead_not_on_scope}]} = + current!(ctx, oats) + + assert {:error, :lead_not_on_scope} = + create( + ctx, + %{lead: content_spec(ctx.oats_definition, ctx.oats), expected_parent: v.id}, + oats + ) + end + + test "a Bierce entry that may not be displayed neither leads nor blocks another lead", ctx do + disallow_record!(ctx.bierce) + + assert LeadRule.applicable([ctx.love.object_id]) == [] + + assert {:error, {:ineligible, [{:lead, 1, :display_restricted}]}} = create(ctx, %{}) + + v = version!(ctx, %{lead: content_spec(ctx.definition, ctx.love)}) + assert v.resolution["lead_rule"] == "manual_fallback" + end + end + + describe "finding 3: a configuration upgrade" do + test "an unchanged arrangement is reissued under the new version, and needs its own review", + ctx do + v1 = version!(ctx, %{}) + publish!(ctx, v1) + + {:ok, cv2} = + Configurations.create_version(ctx.reviewer, ctx.config.id, %{ + reason: "a lower highlight limit", + max_highlights: 2 + }) + + {:ok, _} = + Configurations.activate(ctx.reviewer, ctx.config.id, cv2.id, + reason: "activate", + idempotency_key: key(), + expected: ctx.config_version.id + ) + + assert Published.current(ctx.composition.id) == {:withheld, [:configuration_changed]} + + assert {:ok, v2} = create(ctx, %{reason: "reissued under v2", expected_parent: v1.id}) + assert v2.configuration_version_id == cv2.id + assert v2.composition_id == v1.composition_id and v2.version == 2 + assert v2.arrangement_hash == v1.arrangement_hash + refute v2.eligibility_fingerprint == v1.eligibility_fingerprint + + # Approval never transfers: the old acceptance authorizes nothing here. + assert {:error, :not_approved} = publish(ctx, v2, v1.id) + + # The same arrangement under the same version is still one version, in + # the service and in the database. + assert {:error, {:duplicate_version, id}} = + create(ctx, %{reason: "again", expected_parent: v2.id}) + + assert id == v2.id + + assert {:refused, :unique, "editorial_composition_versions_dedup_index"} = + refused(fn -> + fields = + v2 + |> Map.from_struct() + |> Map.drop([:__meta__, :id, :inserted_at]) + |> Map.merge(%{version: 3, parent_version_id: v2.id}) + + Repo.insert!(struct(CompositionVersion, fields)) + end) + + publish!(ctx, v2, v1.id) + assert current!(ctx).version.id == v2.id + assert Repo.reload!(v1).configuration_version_id == ctx.config_version.id + end + end + + describe "finding 4: work evidence identifies the work" do + test "a catalog pin must be the selected object's, at creation and at read", ctx do + unrelated = WordFixtures.concept!(nil, "Unrelated work") + + assert {:error, {:ineligible, [{:highlight, 1, :work_identity_mismatch}]}} = + create(ctx, %{ + highlights: [Map.put(catalog_spec(ctx.love), :object_id, unrelated.object_id)] + }) + + work = catalog_work!() + + v = + version!(ctx, %{highlights: [Map.put(catalog_spec(ctx.love), :object_id, work.object_id)]}) + + publish!(ctx, v) + assert [%{item_object_id: id}] = current!(ctx).highlights + assert id == work.object_id + + # The registry stops treating the object as that row: the pin no longer + # describes it, and it is withheld. + Repo.update_all(from(x in "external_identifiers", where: x.object_id == ^work.object_id), + set: [status: "rejected"] + ) + + assert %{ + highlights: [], + withheld: [%{role: :highlight, position: 1, reason: :work_identity_mismatch}] + } = current!(ctx) + end + + test "a catalog pin without an object is still a catalog-only work", ctx do + v = version!(ctx, %{highlights: [catalog_spec(ctx.love)]}) + publish!(ctx, v) + assert [%{item_object_id: nil, catalog_identity: "Q8777422"}] = current!(ctx).highlights + end + + test "a source record must have materialized the selected object", ctx do + work = WordFixtures.concept!(nil, "Fixture work") + other = WordFixtures.concept!(nil, "Another work") + {_record, others_revision} = owned_record!(ctx, other) + + assert {:error, {:ineligible, [{:highlight, 1, :work_identity_mismatch}]}} = + create(ctx, %{highlights: [record_work_spec(work, others_revision, ctx.love)]}) + + {record, revision} = owned_record!(ctx, work) + v = version!(ctx, %{highlights: [record_work_spec(work, revision, ctx.love)]}) + publish!(ctx, v) + assert [%{item_object_id: id}] = current!(ctx).highlights + assert id == work.object_id + + Repo.update_all( + from(o in "source_materialized_outputs", where: o.source_record_id == ^record.id), + set: [retired_at: DateTime.utc_now()] + ) + + assert %{withheld: [%{role: :highlight, reason: :work_identity_mismatch}]} = current!(ctx) + end + end + + describe "finding 5: note attribution" do + test "a caller cannot supply a note's author or kind", ctx do + for extra <- [%{author_label: "Ambrose Bierce"}, %{author_kind: :model}] do + note = Map.merge(%{text: "fixture note"}, extra) + + assert {:error, {:note_attribution_not_an_input, :lead, 1}} = + create(ctx, %{lead: Map.put(content_spec(ctx.bierce, ctx.love), :note, note)}) + end + end + + test "a note is its authenticated author's through creation, publication and reading", ctx do + note = %{text: "fixture note"} + v = version!(ctx, %{lead: Map.put(content_spec(ctx.bierce, ctx.love), :note, note)}) + publish!(ctx, v) + + author = actor!(ctx.contributor) + lead = current!(ctx).lead + + assert {lead.note, lead.note_author_kind, lead.note_author_actor_id} == + {"fixture note", :human, author.id} + + assert lead.note_author_label == Compositions.author_label(author) + refute lead.note_author_label =~ "Bierce" + end + + test "the database refuses a note attributed to anyone but its version's author", ctx do + v = version!(ctx, %{}) + settle!() + author = actor!(ctx.contributor) + someone = actor!(account([:contributor])) + + item = fn attrs -> + struct( + CompositionItem, + Map.merge( + %{ + composition_version_id: v.id, + role: :highlight, + position: 1, + item_kind: :content, + item_object_id: ctx.definition.object_id, + content_revision_id: + DevilsDictionary.Registry.current_content_revision(ctx.definition.object_id).id, + meaning_lexeme_id: ctx.love.object_id, + note: "fixture note", + note_author_kind: :human + }, + attrs + ) + ) + end + + cases = [ + {%{note_author_actor_id: author.id, note_author_label: "Ambrose Bierce"}, "own label"}, + {%{ + note_author_actor_id: someone.id, + note_author_label: Compositions.author_label(someone) + }, "version's author"}, + {%{note_author_actor_id: nil, note_author_label: "Account"}, "names the actor"} + ] + + for {attrs, expected} <- cases do + assert {:refused, _code, message} = refused(fn -> Repo.insert!(item.(attrs)) end) + assert message =~ expected + end + end + end + + describe "finding 6: publication authority" do + setup ctx do + v = version!(ctx, %{}) + review = accept!(ctx, v) + settle!() + %{v: v, review: review} + end + + test "the service's receipts name the operator authority, for publication and withdrawal", + ctx do + {:ok, published} = publish(ctx, ctx.v) + + {:ok, withdrawn} = + Publications.withdraw(ctx.reviewer, ctx.composition.id, + reason: "take down", + idempotency_key: key(), + expected_pointer: ctx.v.id + ) + + assert {published.authority_kind, withdrawn.authority_kind} == {:operator, :operator} + assert published.authorizing_review_id == ctx.review.id + settle!() + end + + test "a receipt with no authority, or a panel authority, is refused", ctx do + assert {:refused, _code, _} = + refused(fn -> Repo.insert!(raw_receipt(ctx, %{authority_kind: nil})) end) + + # A panel decision needs decision records that do not exist yet. Nothing + # may stand in for one. + assert {:refused, _code, _} = + refused(fn -> + Repo.query!( + """ + INSERT INTO editorial_composition_publications + (composition_id, action, authority_kind, published_version_id, + authorizing_review_id, actor_id, reason, eligibility_fingerprint, + idempotency_key, committed_at) + VALUES ($1, 'publish', 'panel_decision', $2, $3, $4, 'forged', $5, $6, now()) + """, + [ + ctx.composition.id, + ctx.v.id, + ctx.review.id, + actor!(ctx.reviewer).id, + ctx.v.eligibility_fingerprint, + key("forged") + ] + ) + end) + + # The same receipt with the operator authority, moving the pointer with it, stands. + assert :accepted = + refused(fn -> + Repo.insert!(raw_receipt(ctx, %{})) + + Repo.query!( + "UPDATE editorial_compositions SET current_published_version_id = $1 WHERE id = $2", + [ctx.v.id, ctx.composition.id] + ) + end) + end + + test "a forged operator authority is refused: a bot, a non-reviewer, or a review that did not accept", + ctx do + bot = bot_actor!(ctx.sources["wikidata"]) + + for actor <- [bot, actor!(ctx.contributor)] do + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> Repo.insert!(raw_receipt(ctx, %{actor_id: actor.id})) end) + + assert message =~ "reviewer role" + end + + {:ok, rejection} = + Reviews.decide(ctx.reviewer, ctx.v.id, :rejected, reason: "no", idempotency_key: key()) + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.insert!(raw_receipt(ctx, %{authorizing_review_id: rejection.id})) + end) + end + + test "a stale authority is refused: a superseded acceptance, or another fingerprint", ctx do + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.insert!( + raw_receipt(ctx, %{eligibility_fingerprint: "an older fingerprint"}) + ) + end) + + {:ok, _} = + Reviews.decide(ctx.reviewer, ctx.v.id, :needs_review, + reason: "second look", + idempotency_key: key() + ) + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> Repo.insert!(raw_receipt(ctx, %{})) end) + + assert message =~ "latest review" + assert {:error, :not_approved} = publish(ctx, ctx.v) + end + + test "a cross-configuration authority is refused", ctx do + {config_b, _} = enabled_test_configuration!(ctx.reviewer, "audit-b") + + {:ok, b} = + Compositions.provision(ctx.contributor, config_b.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.love.object_id], + language_tag: "en", + reason: "the same scope, another configuration" + }) + + vb = version!(ctx, %{}, b) + review_b = accept!(ctx, vb) + + # Another configuration's version published into this composition, or + # its acceptance cited for this composition's version. + assert {:refused, :foreign_key, _} = + refused(fn -> + Repo.insert!( + raw_receipt(ctx, %{ + published_version_id: vb.id, + authorizing_review_id: review_b.id, + eligibility_fingerprint: vb.eligibility_fingerprint + }) + ) + end) + + # Here the authority check refuses it before the ownership key would. + assert {:refused, _code, _} = + refused(fn -> + Repo.insert!(raw_receipt(ctx, %{authorizing_review_id: review_b.id})) + end) + + assert %CompositionVersion{curation_configuration_id: config_id} = vb + assert config_id == config_b.id + end + end +end diff --git a/test/devils_dictionary/curation/compositions_test.exs b/test/devils_dictionary/curation/compositions_test.exs new file mode 100644 index 00000000..55679712 --- /dev/null +++ b/test/devils_dictionary/curation/compositions_test.exs @@ -0,0 +1,586 @@ +defmodule DevilsDictionary.Curation.CompositionsTest do + @moduledoc """ + K1–K10 of `docs/curation/persistence-slice-1.md`: composition identity, + scope, and manual versions with exact references. + """ + use DevilsDictionary.DataCase, async: true + + import DevilsDictionary.CurationFixtures + + alias DevilsDictionary.Curation.{ + Composition, + CompositionItem, + CompositionVersion, + Compositions, + Configurations, + Digest, + ScopeChange + } + + alias DevilsDictionary.Registry + + setup do + world = world!() + {config, config_version} = enabled_test_configuration!(world.reviewer, "test-a") + Map.merge(world, %{config: config, config_version: config_version}) + end + + defp provision!(ctx, config \\ nil, ids \\ nil) do + {:ok, c} = + Compositions.provision(ctx.contributor, (config || ctx.config).id, %{ + scope_kind: :lexeme, + lexeme_ids: ids || [ctx.love.object_id], + language_tag: "en", + reason: "curate love" + }) + + c + end + + defp version!(ctx, composition, attrs) do + {:ok, v} = + Compositions.create_version( + ctx.contributor, + composition.id, + Map.merge(%{reason: "manual arrangement", expected_parent: nil}, attrs) + ) + + v + end + + describe "identity (K1–K3)" do + test "the identity is (kind, signature, language, configuration) and provisioning is idempotent", + ctx do + c = provision!(ctx) + assert c == provision!(ctx) + + assert %Composition{ + scope_kind: :lexeme, + language_tag: "en", + curation_configuration_id: config_id, + scope_signature: signature + } = c + + assert config_id == ctx.config.id + assert signature == Digest.scope_signature([{:lexeme, ctx.love.object_id}]) + assert Compositions.member_ids(c.id) == [ctx.love.object_id] + + assert [ + %ScopeChange{ + previous_signature: nil, + scope_signature: ^signature, + members: members + } + ] = + Repo.all(from s in ScopeChange, where: s.composition_id == ^c.id) + + assert members == [["lexeme", ctx.love.object_id]] + settle!() + end + + test "two configurations over one scope keep independent histories, pointers and clocks", + ctx do + {config_b, _} = enabled_test_configuration!(ctx.reviewer, "test-b") + a = provision!(ctx) + b = provision!(ctx, config_b) + + refute a.id == b.id + assert a.scope_signature == b.scope_signature + + lead = content_spec(ctx.bierce, ctx.love) + va1 = version!(ctx, a, %{lead: lead}) + + va2 = + version!(ctx, a, %{ + lead: lead, + highlights: [quotation_spec(ctx.sense, 1)], + expected_parent: va1.id + }) + + vb1 = version!(ctx, b, %{lead: lead}) + + # Each composition numbers its own versions and chains its own parents. + assert {va1.version, va2.version, vb1.version} == {1, 2, 1} + assert va2.parent_version_id == va1.id and is_nil(vb1.parent_version_id) + assert vb1.configuration_version_id != va1.configuration_version_id + + # A version of one can never be the parent of, or published by, the other. + assert {:refused, :foreign_key_violation, _} = + refused(fn -> + Repo.query!( + "UPDATE editorial_compositions SET current_published_version_id = $1 WHERE id = $2", + [ + va1.id, + b.id + ] + ) + end) + + assert {:error, {:stale_parent, id}} = + Compositions.create_version(ctx.contributor, b.id, %{ + lead: lead, + reason: "x", + expected_parent: va1.id + }) + + assert id == vb1.id + settle!() + end + + test "provisioning takes registry ids, refuses overlap and a mixed language", ctx do + page = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexical_page, + lexeme_ids: [ctx.love.object_id, ctx.amor.object_id], + language_tag: "en", + reason: "love and amor share a page" + }) + + assert {:ok, %Composition{id: page_id}} = page + + assert {:error, {:overlapping_scope, [^page_id]}} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexical_page, + lexeme_ids: [ctx.love.object_id], + language_tag: "en", + reason: "a split nobody reconciled" + }) + + # A lexeme scope is a different kind, and has exactly one member. + assert {:ok, _} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.love.object_id], + language_tag: "en", + reason: "the lexeme itself" + }) + + assert {:error, :lexeme_scope_has_one_member} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.love.object_id, ctx.oats.object_id], + language_tag: "en", + reason: "x" + }) + + assert {:error, :language_mismatch} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.oats.object_id], + language_tag: "fr", + reason: "x" + }) + + assert {:error, {:not_lexemes, [id]}} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.bierce.object_id], + language_tag: "en", + reason: "a content item is not a scope" + }) + + assert id == ctx.bierce.object_id + + assert {:error, :unauthorized} = + Compositions.provision(account(), ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.oats.object_id], + language_tag: "en", + reason: "x" + }) + + settle!() + end + + test "a draft or disabled configuration has no compositions made under it", ctx do + {:ok, draft} = Configurations.create_test_configuration(ctx.reviewer, "test-draft", "draft") + + assert {:error, :configuration_unavailable} = + Compositions.provision(ctx.contributor, draft.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.love.object_id], + language_tag: "en", + reason: "x" + }) + end + + test "a scope change is explicit and audited; a signature that does not match fails at commit", + ctx do + {:ok, page} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexical_page, + lexeme_ids: [ctx.love.object_id], + language_tag: "en", + reason: "love's page" + }) + + assert {:ok, changed} = + Compositions.change_scope( + ctx.contributor, + page.id, + [ctx.love.object_id, ctx.amor.object_id], + reason: "amor merges into love's page" + ) + + assert changed.scope_signature == + Compositions.signature([ctx.love.object_id, ctx.amor.object_id]) + + assert [first, second] = + Repo.all( + from s in ScopeChange, where: s.composition_id == ^page.id, order_by: s.id + ) + + assert second.previous_signature == first.scope_signature + settle!() + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.query!( + "UPDATE editorial_compositions SET scope_signature = 'love' WHERE id = $1", + [ + page.id + ] + ) + end) + + assert message =~ "scope does not match" + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.query!( + "INSERT INTO editorial_composition_memberships (composition_id, object_id, role, inserted_at) VALUES ($1, $2, 'lexeme', now())", + [page.id, ctx.oats.object_id] + ) + end) + end + end + + describe "manual versions (K4–K9)" do + test "a version is manual, human, reasoned, exact, and invents no history", ctx do + c = provision!(ctx) + + v = + version!(ctx, c, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [ + quotation_spec(ctx.sense, 1), + content_spec(ctx.definition, ctx.love) + |> Map.put(:note, %{text: "a plain gloss beside it"}), + catalog_spec(ctx.love) + ] + }) + + settle!() + + assert %CompositionVersion{ + version: 1, + origin: :manual, + parent_version_id: nil, + change_reason: "manual arrangement", + lead_policy: :bierce_first_v1 + } = v + + assert v.created_by_actor_id == actor!(ctx.contributor).id + assert v.resolution["lead_rule"] == "priority_source" + assert v.resolution["priority_leads"] == [ctx.bierce.object_id] + + items = Compositions.items(v.id) + + assert Enum.map(items, &{&1.role, &1.position, &1.item_kind}) == + [ + {:highlight, 1, :sense_quotation}, + {:highlight, 2, :content}, + {:highlight, 3, :work}, + {:lead, 1, :content} + ] + + assert Enum.all?(items, &(&1.selection_origin == :manual)) + assert v.arrangement_hash == Digest.arrangement_hash(items) + + note = Enum.find(items, & &1.note) + + assert {note.note_author_kind, note.note_author_label} == + {:human, actor!(ctx.contributor).label} + + # Manual work has no run, participant, ballot or model anywhere. + for table <- ~w(curation_runs curation_participants curation_ballots local_model_configs) do + assert %{rows: [[false]]} = + Repo.query!("SELECT to_regclass($1) IS NOT NULL", [table]) + end + + # And no model-authored note can be written into it. + assert {:error, {:note_invalid, :highlight, 1}} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [ + Map.put(content_spec(ctx.definition, ctx.love), :note, "model says") + ], + reason: "x", + expected_parent: v.id + }) + end + + test "a version cannot be edited, removed or have items added", ctx do + v = version!(ctx, provision!(ctx), %{lead: content_spec(ctx.bierce, ctx.love)}) + settle!() + [lead] = Compositions.items(v.id) + + for sql <- [ + "UPDATE editorial_composition_versions SET change_reason = 'rewritten' WHERE id = $1", + "DELETE FROM editorial_composition_versions WHERE id = $1" + ] do + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> Repo.query!(sql, [v.id]) end) + end + + for sql <- [ + "UPDATE editorial_composition_items SET position = 2 WHERE id = $1", + "UPDATE editorial_composition_items SET content_revision_id = content_revision_id + 1 WHERE id = $1", + "DELETE FROM editorial_composition_items WHERE id = $1" + ] do + assert {:refused, _code, _} = refused(fn -> Repo.query!(sql, [lead.id]) end) + end + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.insert!(%CompositionItem{ + composition_version_id: v.id, + role: :highlight, + position: 1, + item_kind: :content, + item_object_id: ctx.definition.object_id, + content_revision_id: + Registry.current_content_revision(ctx.definition.object_id).id, + meaning_lexeme_id: ctx.love.object_id + }) + end) + + assert message =~ "arrangement differs" + end + + test "a fourth highlight or a second lead cannot be stored", ctx do + c = provision!(ctx) + v = version!(ctx, c, %{lead: content_spec(ctx.bierce, ctx.love)}) + q = quotation_spec(ctx.sense, 1) + + assert {:error, {:too_many_highlights, 3}} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [q, q, q, q], + reason: "x", + expected_parent: v.id + }) + + item = fn role, position -> + %CompositionItem{ + composition_version_id: v.id, + role: role, + position: position, + item_kind: :content, + item_object_id: ctx.definition.object_id, + content_revision_id: Registry.current_content_revision(ctx.definition.object_id).id, + meaning_lexeme_id: ctx.love.object_id + } + end + + assert {:refused, :check, _} = refused(fn -> Repo.insert!(item.(:highlight, 4)) end) + assert {:refused, :check, _} = refused(fn -> Repo.insert!(item.(:lead, 2)) end) + assert {:refused, :unique, _} = refused(fn -> Repo.insert!(item.(:lead, 1)) end) + end + + test "cross-configuration, cross-composition and cross-object links are refused", ctx do + {config_b, config_b_version} = enabled_test_configuration!(ctx.reviewer, "test-b") + a = provision!(ctx) + b = provision!(ctx, config_b) + va = version!(ctx, a, %{lead: content_spec(ctx.bierce, ctx.love)}) + actor = actor!(ctx.contributor) + + raw_version = fn attrs -> + Repo.insert!( + struct( + CompositionVersion, + Map.merge( + %{ + composition_id: a.id, + curation_configuration_id: a.curation_configuration_id, + configuration_version_id: va.configuration_version_id, + version: 10 + System.unique_integer([:positive]), + parent_version_id: va.id, + change_reason: "raw", + created_by_actor_id: actor.id, + scope_signature: a.scope_signature, + scope_members: [], + resolution: %{}, + lead_policy: :bierce_first_v1, + arrangement_hash: Digest.arrangement_hash([]), + eligibility_fingerprint: key("fp") + }, + attrs + ) + ) + ) + end + + assert :accepted = refused(fn -> raw_version.(%{}) end) + + # Another configuration's version, or a parent from another composition. + assert {:refused, :foreign_key, _} = + refused(fn -> raw_version.(%{configuration_version_id: config_b_version.id}) end) + + vb = version!(ctx, b, %{lead: content_spec(ctx.bierce, ctx.love)}) + + assert {:refused, :foreign_key, _} = + refused(fn -> raw_version.(%{parent_version_id: vb.id}) end) + + # A revision of another object than the item names. + assert {:refused, :foreign_key, _} = + refused(fn -> + v = raw_version.(%{}) + + Repo.insert!(%CompositionItem{ + composition_version_id: v.id, + role: :lead, + position: 1, + item_kind: :content, + item_object_id: ctx.definition.object_id, + content_revision_id: + Registry.current_content_revision(ctx.bierce.object_id).id, + meaning_lexeme_id: ctx.love.object_id + }) + end) + + # A bot is never an author. + bot = bot_actor!(ctx.sources["wikidata"]) + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> raw_version.(%{created_by_actor_id: bot.id}) end) + end + + test "exact references are checked against the registry when the version is made", ctx do + c = provision!(ctx) + lead = content_spec(ctx.bierce, ctx.love) + + attempt = fn highlight -> + Compositions.create_version(ctx.contributor, c.id, %{ + lead: lead, + highlights: [highlight], + reason: "x", + expected_parent: nil + }) + end + + # A meaning off the scope. + assert {:error, {:ineligible, [{:highlight, 1, :meaning_off_scope}]}} = + attempt.(content_spec(ctx.definition, ctx.oats)) + + # Words that are not what the source says. + assert {:error, {:ineligible, [{:highlight, 1, :words_changed}]}} = + attempt.(%{ + quotation_spec(ctx.sense, 1) + | words_sha256: Digest.sha256("other words") + }) + + # A locator at an example that is not a quotation. + assert {:error, {:ineligible, [{:highlight, 1, :words_changed}]}} = + attempt.(%{quotation_spec(ctx.sense, 1) | locator: "quotation:0"}) + + # A catalog pin to a checksum or a row this build does not have. + assert {:error, {:ineligible, [{:highlight, 1, :catalog_changed}]}} = + attempt.(catalog_spec(ctx.love, %{checksum: "not-the-committed-one"})) + + assert {:error, {:ineligible, [{:highlight, 1, :catalog_row_missing}]}} = + attempt.(catalog_spec(ctx.love, %{identity: "Q0"})) + + assert {:error, {:ineligible, [{:highlight, 1, :catalog_changed}]}} = + attempt.(catalog_spec(ctx.love, %{manifest: "../../etc/passwd"})) + + # A superseded revision is not a pin to the current one. + stale = content_spec(ctx.definition, ctx.love) + + {:ok, _} = + Registry.add_content_revision(ctx.definition.object_id, %{body: "fixture revised body"}) + + assert {:error, {:ineligible, [{:highlight, 1, :revision_superseded}]}} = attempt.(stale) + + # No meaning at all. + assert {:error, {:meaning_required, :highlight, 1}} = + attempt.(Map.delete(content_spec(ctx.definition, ctx.love), :meaning)) + end + + test "two authors cannot both write the next version", ctx do + c = provision!(ctx) + v1 = version!(ctx, c, %{lead: content_spec(ctx.bierce, ctx.love)}) + + v2 = + version!(ctx, c, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [quotation_spec(ctx.sense, 1)], + expected_parent: v1.id + }) + + assert {:error, {:stale_parent, id}} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [content_spec(ctx.definition, ctx.love)], + reason: "from a stale screen", + expected_parent: v1.id + }) + + assert id == v2.id + + # And the same arrangement twice is one version. + assert {:error, {:duplicate_version, id}} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [quotation_spec(ctx.sense, 1)], + reason: "again", + expected_parent: v2.id + }) + + assert id == v2.id + settle!() + end + end + + describe "Bierce first (K10)" do + test "an applicable Bierce entry leads, and a version without it is refused", ctx do + c = provision!(ctx) + + assert {:error, :priority_source_available} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.definition, ctx.love), + reason: "x", + expected_parent: nil + }) + + assert {:error, :priority_source_missing} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: nil, + highlights: [quotation_spec(ctx.sense, 1)], + reason: "x", + expected_parent: nil + }) + + assert %CompositionVersion{} = version!(ctx, c, %{lead: content_spec(ctx.bierce, ctx.love)}) + end + + test "where Bierce has no entry, a person may choose a page definition, or none", ctx do + c = provision!(ctx, nil, [ctx.oats.object_id]) + + v = version!(ctx, c, %{lead: content_spec(ctx.oats_definition, ctx.oats)}) + assert v.resolution["lead_rule"] == "manual_fallback" + + # A definition from another page is not this page's to lead. + assert {:error, :lead_not_on_scope} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.definition, ctx.oats), + reason: "x", + expected_parent: v.id + }) + + empty = version!(ctx, c, %{lead: nil, expected_parent: v.id, reason: "no lead"}) + assert empty.resolution["lead_rule"] == "none" + settle!() + end + end +end diff --git a/test/devils_dictionary/curation/configurations_test.exs b/test/devils_dictionary/curation/configurations_test.exs new file mode 100644 index 00000000..69a3abd8 --- /dev/null +++ b/test/devils_dictionary/curation/configurations_test.exs @@ -0,0 +1,582 @@ +defmodule DevilsDictionary.Curation.ConfigurationsTest do + @moduledoc """ + C1–C8 of `docs/curation/persistence-slice-1.md`: configuration identities, + immutable versions, rosters, profile admission, activation and default + resolution. + """ + use DevilsDictionary.DataCase, async: true + + import DevilsDictionary.CurationFixtures + + alias DevilsDictionary.Curation.{ + Composition, + CompositionPublication, + Configuration, + ConfigurationActivation, + ConfigurationMember, + ConfigurationVersion, + Digest, + Profile, + Profiles, + Configurations + } + + alias DevilsDictionary.Fixtures + + setup do + ctx = Fixtures.seed_catalog!() + Map.merge(ctx, %{reviewer: account([:reviewer]), contributor: account([:contributor])}) + end + + describe "seeding" do + test "creates a draft default with a manual-only version and five proposed identities" do + %{configuration: c, version: v, profiles: profiles} = Configurations.seed!() + settle!() + + assert %Configuration{ + slug: "global-default", + role: :global_default, + ownership_kind: :system, + state: :draft, + current_version_id: nil + } = c + + assert %ConfigurationVersion{version: 1, lead_policy: :bierce_first_v1, max_highlights: 3} = + v + + assert v.manifest["mode"] == "manual" + assert v.manifest["roster"] == [] + assert v.roster_hash == Digest.roster_hash([]) + refute Map.has_key?(v.manifest, "model") + + assert Enum.map(profiles, & &1.slug) == ~w(bierce voltaire vonnegut hitchens le-guin) + + for p <- profiles do + assert %Profile{state: :proposed, current_version_id: nil, bot_actor_id: nil} = p + end + + # A seed is not a human: nothing it makes is approved or active. + assert Configurations.resolve_default() == {:unavailable, :no_active_version} + assert Repo.aggregate(ConfigurationActivation, :count) == 0 + end + + test "is idempotent and never changes an existing row's state", %{reviewer: reviewer} do + {c, v} = enabled_default!(reviewer) + + %{configuration: again, version: v_again} = Configurations.seed!() + settle!() + + assert again.id == c.id and v_again.id == v.id + assert Repo.reload!(c).state == :enabled + assert Repo.aggregate(Configuration, :count) == 1 + assert Repo.aggregate(ConfigurationVersion, :count) == 1 + assert Repo.aggregate(Profile, :count) == 5 + end + end + + describe "resolution (C7)" do + test "answers the default or an explicit unavailable, never another configuration", + %{reviewer: reviewer} do + assert Configurations.resolve_default() == {:unavailable, :missing} + + # An enabled *test* configuration is never the default. + enabled_test_configuration!(reviewer, "test-a") + assert Configurations.resolve_default() == {:unavailable, :missing} + + %{configuration: c, version: v} = Configurations.seed!() + assert Configurations.resolve_default() == {:unavailable, :no_active_version} + + {:ok, _} = + Configurations.activate(reviewer, c.id, v.id, + reason: "go", + idempotency_key: key(), + expected: nil + ) + + assert {:ok, %{configuration: %{id: id}, version: %{id: vid}}} = + Configurations.resolve_default() + + assert {id, vid} == {c.id, v.id} + + {:ok, _} = + Configurations.disable(reviewer, c.id, + reason: "stop", + idempotency_key: key(), + expected: v.id + ) + + assert Configurations.resolve_default() == {:unavailable, :disabled} + settle!() + end + + test "a panel is never available, and a version this build cannot run is unready", + %{reviewer: reviewer} do + assert Configurations.resolve_default(purpose: :panel) == + {:unavailable, :panel_not_available} + + %{configuration: c} = Configurations.seed!() + actor = actor!(reviewer) + + # A version from some later build: its mode is not one this build runs. + panel = + Repo.insert!(%ConfigurationVersion{ + configuration_id: c.id, + version: 2, + manifest_hash: "later", + manifest: %{"mode" => "panel"}, + lead_policy: :bierce_first_v1, + roster_hash: Digest.roster_hash([]), + created_by_actor_id: actor.id, + change_reason: "a version this build cannot run" + }) + + assert {:error, :unready} = + Configurations.activate(reviewer, c.id, panel.id, + reason: "go", + idempotency_key: key(), + expected: nil + ) + + assert Configurations.resolve_default() == {:unavailable, :no_active_version} + + assert Configurations.resolve_default(purpose: :panel) == + {:unavailable, :panel_not_available} + end + end + + describe "ownership and uniqueness (C1, C2)" do + test "only system ownership exists and one global default is enabled", %{reviewer: reviewer} do + {_c, _v} = enabled_default!(reviewer) + actor = actor!(reviewer) + + assert {:refused, :check_violation, _} = + refused(fn -> + Repo.query!( + "INSERT INTO curation_configurations (slug, name, ownership_kind, role, state, created_by_actor_id, inserted_at, updated_at) VALUES ('mine', 'x', 'personal', 'internal_test', 'draft', $1, now(), now())", + [actor.id] + ) + end) + + # A second global default can exist as a draft, but never be enabled + # beside the first: refused by the service, and by the index. + second = + Repo.insert!(%Configuration{ + slug: "second-default", + name: "second", + role: :global_default, + created_by_actor_id: actor.id + }) + + {:ok, v2} = Configurations.create_version(reviewer, second.id, %{reason: "v1"}) + + assert {:error, :another_default_enabled} = + Configurations.activate(reviewer, second.id, v2.id, + reason: "go", + idempotency_key: key(), + expected: nil + ) + + assert {:refused, :unique_violation, _} = + refused(fn -> + Repo.query!( + "UPDATE curation_configurations SET state = 'enabled', current_version_id = $1 WHERE id = $2", + [v2.id, second.id] + ) + end) + end + + test "a pointer to another configuration's version is refused", %{reviewer: reviewer} do + {a, _va} = enabled_test_configuration!(reviewer, "test-a") + {_b, vb} = enabled_test_configuration!(reviewer, "test-b") + + assert {:refused, :foreign_key_violation, _} = + refused(fn -> + Repo.query!( + "UPDATE curation_configurations SET current_version_id = $1 WHERE id = $2", + [ + vb.id, + a.id + ] + ) + end) + + assert {:error, :version_not_found} = + Configurations.activate(reviewer, a.id, vb.id, + reason: "cross", + idempotency_key: key(), + expected: a.current_version_id + ) + end + end + + describe "receipts (C3, C8)" do + test "a pointer moved without a receipt fails at commit", %{reviewer: reviewer} do + {c, _v} = enabled_test_configuration!(reviewer, "test-a") + + {:ok, v2} = + Configurations.create_version(reviewer, c.id, %{reason: "v2", max_highlights: 2}) + + settle!() + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.query!( + "UPDATE curation_configurations SET current_version_id = $1 WHERE id = $2", + [ + v2.id, + c.id + ] + ) + end) + + assert message =~ "latest activation receipt" + + # And a receipt that does not continue the one before it is refused too. + actor = actor!(reviewer) + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.insert!(%ConfigurationActivation{ + configuration_id: c.id, + action: :activate, + configuration_version_id: v2.id, + previous_version_id: nil, + actor_id: actor.id, + reason: "forged", + idempotency_key: key() + }) + + Repo.query!( + "UPDATE curation_configurations SET current_version_id = $1 WHERE id = $2", + [v2.id, c.id] + ) + end) + end + + test "activation is a reviewer's audited, idempotent act and publishes nothing", + %{reviewer: reviewer, contributor: contributor} = ctx do + %{configuration: c, version: v} = Configurations.seed!() + k = key() + opts = [reason: "launch the manual default", idempotency_key: k, expected: nil] + + assert {:error, :unauthorized} = Configurations.activate(contributor, c.id, v.id, opts) + assert {:error, :unauthorized} = Configurations.activate(nil, c.id, v.id, opts) + + assert {:ok, %ConfigurationActivation{} = receipt} = + Configurations.activate(reviewer, c.id, v.id, opts) + + assert receipt.actor_id == actor!(reviewer).id + assert receipt.previous_version_id == nil and receipt.configuration_version_id == v.id + + # The same key replays the same receipt; a different use of it is refused. + assert {:ok, ^receipt} = Configurations.activate(reviewer, c.id, v.id, opts) + + assert {:error, :idempotency_conflict} = + Configurations.disable(reviewer, c.id, + reason: "x", + idempotency_key: k, + expected: v.id + ) + + # An identical version is the same version, not a second one. + assert {:error, {:duplicate_version, v_id}} = + Configurations.create_version(reviewer, c.id, %{reason: "again"}) + + assert v_id == v.id + + # A stale screen cannot move the pointer. + {:ok, v2} = + Configurations.create_version(reviewer, c.id, %{reason: "v2", max_highlights: 2}) + + assert {:error, {:stale, v_id}} = + Configurations.activate(reviewer, c.id, v2.id, + reason: "x", + idempotency_key: key(), + expected: nil + ) + + assert v_id == v.id + assert Repo.aggregate(ConfigurationActivation, :count) == 1 + assert Repo.aggregate(Composition, :count) == 0 + assert Repo.aggregate(CompositionPublication, :count) == 0 + + # The database refuses a bot or a non-reviewer receipt outright. + bot = bot_actor!(ctx.sources["wikidata"]) + + for actor <- [bot, actor!(contributor)] do + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.insert!(%ConfigurationActivation{ + configuration_id: c.id, + action: :activate, + configuration_version_id: v2.id, + previous_version_id: v.id, + actor_id: actor.id, + reason: "not a reviewer", + idempotency_key: key() + }) + end) + end + + settle!() + end + + test "a revoked reviewer is refused, rechecked under the account lock", %{reviewer: reviewer} do + %{configuration: c, version: v} = Configurations.seed!() + revoke!(reviewer) + + assert {:error, :unauthorized} = + Configurations.activate(reviewer, c.id, v.id, + reason: "go", + idempotency_key: key(), + expected: nil + ) + end + end + + describe "immutability and rosters (C4, C5)" do + setup %{reviewer: reviewer} do + profile = admitted_test_profile!(reviewer) + {c, _} = enabled_test_configuration!(reviewer, "test-roster") + + {:ok, v} = + Configurations.create_version(reviewer, c.id, %{ + reason: "one seat", + members: [%{profile_id: profile.id, profile_version_id: profile.current_version_id}] + }) + + settle!() + %{profile: profile, configuration: c, version: v} + end + + test "versions and rosters cannot be edited, removed or extended", ctx do + assert [%ConfigurationMember{slot: 1, voting_weight: 1}] = + Repo.all( + from m in ConfigurationMember, + where: m.configuration_version_id == ^ctx.version.id + ) + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.query!( + "UPDATE curation_configuration_versions SET max_highlights = 1 WHERE id = $1", + [ + ctx.version.id + ] + ) + end) + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.query!( + "DELETE FROM curation_configuration_members WHERE configuration_version_id = $1", + [ + ctx.version.id + ] + ) + end) + + # A seat added after the version's own transaction breaks its roster hash. + other = admitted_test_profile!(ctx.reviewer, "test-second") + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.insert!(%ConfigurationMember{ + configuration_version_id: ctx.version.id, + profile_id: other.id, + profile_version_id: other.current_version_id, + slot: 2 + }) + end) + + assert message =~ "roster differs" + end + + test "a seat's profile version is its profile's, one per slot and profile, weight 1", ctx do + other = admitted_test_profile!(ctx.reviewer, "test-third") + actor = actor!(ctx.reviewer) + + insert_version = fn members -> + v = + Repo.insert!(%ConfigurationVersion{ + configuration_id: ctx.configuration.id, + version: 99, + manifest_hash: key("manifest"), + manifest: %{"mode" => "manual"}, + lead_policy: :bierce_first_v1, + roster_hash: Digest.roster_hash(members), + created_by_actor_id: actor.id, + change_reason: "raw" + }) + + for m <- members, + do: + Repo.insert!( + struct(ConfigurationMember, Map.put(m, :configuration_version_id, v.id)) + ) + end + + seat = fn profile, version_id, slot, weight -> + %{ + profile_id: profile.id, + profile_version_id: version_id, + slot: slot, + voting_weight: weight + } + end + + assert {:refused, :foreign_key, _} = + refused(fn -> + insert_version.([seat.(ctx.profile, other.current_version_id, 1, 1)]) + end) + + assert {:refused, :unique, _} = + refused(fn -> + insert_version.([ + seat.(ctx.profile, ctx.profile.current_version_id, 1, 1), + seat.(other, other.current_version_id, 1, 1) + ]) + end) + + assert {:refused, :check, _} = + refused(fn -> + insert_version.([seat.(ctx.profile, ctx.profile.current_version_id, 1, 2)]) + end) + + assert :accepted = + refused(fn -> + insert_version.([ + seat.(ctx.profile, ctx.profile.current_version_id, 1, 1), + seat.(other, other.current_version_id, 2, 1) + ]) + end) + end + + test "a roster seat must be an admitted profile's admitted version", ctx do + [bierce | _] = Profiles.seed_proposed!() + + assert {:error, :profile_not_admitted} = + Configurations.create_version(ctx.reviewer, ctx.configuration.id, %{ + reason: "unadmitted seat", + members: [%{profile_id: bierce.id, profile_version_id: -1}] + }) + end + end + + describe "admission (C6)" do + test "needs a reviewer, a sourced version with deceased-status evidence, and a bot principal", + ctx do + {:ok, profile} = test_profile!() + + {:ok, unsourced} = + Profiles.add_version(ctx.contributor, profile.id, %{ + template_version: "test-v1", + reason: "draft dossier", + source_refs: [], + deceased_evidence_refs: [] + }) + + bot = bot_actor!(ctx.sources["wikidata"]) + opts = [reason: "reviewed", bot_actor_id: bot.id] + + assert {:error, :unauthorized} = + Profiles.admit(ctx.contributor, profile.id, unsourced.id, opts) + + assert {:error, :sources_missing} = + Profiles.admit(ctx.reviewer, profile.id, unsourced.id, opts) + + {:ok, sourced} = + Profiles.add_version(ctx.contributor, profile.id, %{ + template_version: "test-v1", + reason: "sourced dossier", + source_refs: [%{"ref" => "test:source-1"}], + deceased_evidence_refs: [%{"ref" => "test:evidence-1"}] + }) + + assert {:error, :bot_actor_missing} = + Profiles.admit(ctx.reviewer, profile.id, sourced.id, reason: "reviewed") + + assert {:ok, %Profile{state: :admitted} = admitted} = + Profiles.admit(ctx.reviewer, profile.id, sourced.id, opts) + + assert admitted.admitted_by_actor_id == actor!(ctx.reviewer).id + settle!() + + # The database refuses the same shortcuts on its own. + {:ok, raw} = test_profile!("test-raw") + + # The admission trigger answers first; the check constraint stands behind it. + assert {:refused, _code, _} = + refused(fn -> + Repo.query!("UPDATE curator_profiles SET state = 'admitted' WHERE id = $1", [ + raw.id + ]) + end) + + {:ok, raw_version} = + Profiles.add_version(ctx.contributor, raw.id, %{ + template_version: "test-v1", + reason: "sourced", + source_refs: [%{"ref" => "test:s"}], + deceased_evidence_refs: [%{"ref" => "test:e"}] + }) + + other_bot = bot_actor!(ctx.sources["wiktionary"]) + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.query!( + """ + UPDATE curator_profiles SET state = 'admitted', bot_actor_id = $2, + admitted_by_actor_id = $3, admitted_at = now(), admission_reason = 'self' + WHERE id = $1 + """, + [raw.id, other_bot.id, actor!(ctx.contributor).id] + ) + end) + + assert message =~ "human reviewer" + assert raw_version.profile_id == raw.id + end + end + + @bot_sources %{ + "test-admitted" => "wordnet", + "test-second" => "johnson", + "test-third" => "wikipedia" + } + + # A test profile with test references: no real person's dossier is written. + defp test_profile!(slug \\ "test-profile") do + {:ok, + Repo.insert!(%Profile{ + slug: slug, + label: "Test profile", + subject_label: "Nobody (test)", + state: :proposed + })} + end + + defp admitted_test_profile!(reviewer, slug \\ "test-admitted") do + {:ok, profile} = test_profile!(slug) + author = account([:contributor]) + + {:ok, version} = + Profiles.add_version(author, profile.id, %{ + template_version: "test-v1", + reason: "sourced", + source_refs: [%{"ref" => "test:#{slug}:source"}], + deceased_evidence_refs: [%{"ref" => "test:#{slug}:evidence"}] + }) + + # One bot principal per profile, each on its own source. + source_slug = Map.fetch!(@bot_sources, slug) + source = Repo.one!(from s in DevilsDictionary.Sources.Source, where: s.slug == ^source_slug) + bot = bot_actor!(source) + + {:ok, admitted} = + Profiles.admit(reviewer, profile.id, version.id, reason: "test", bot_actor_id: bot.id) + + admitted + end +end diff --git a/test/devils_dictionary/curation/publication_race_test.exs b/test/devils_dictionary/curation/publication_race_test.exs new file mode 100644 index 00000000..b635f058 --- /dev/null +++ b/test/devils_dictionary/curation/publication_race_test.exs @@ -0,0 +1,278 @@ +defmodule DevilsDictionary.Curation.PublicationRaceTest do + @moduledoc """ + R5, and the commit-time checks behind R3 and K1, on committed connections. + + A sandbox wraps a test in one transaction, which never commits. So a + deferred check never runs there, and two tasks share one connection. + These tests are `:unboxed`: each task checks out its own connection, rows + really commit, and the database's row locks and COMMIT-time checks are the + ones under test. Each task waits at a gate the test opens, and the order of + events is fixed by messages, never by sleeping. + """ + use DevilsDictionary.DataCase, async: false + + import DevilsDictionary.CurationFixtures + + alias DevilsDictionary.Curation.{ + Composition, + CompositionItem, + CompositionPublication, + Compositions, + Publications, + Reviews + } + + alias DevilsDictionary.Repo + + @moduletag :unboxed + + setup do + world = world!() + {config, _} = enabled_test_configuration!(world.reviewer, "race") + + {:ok, composition} = + Compositions.provision(world.contributor, config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [world.love.object_id], + language_tag: "en", + reason: "curate love" + }) + + {:ok, v1} = + Compositions.create_version(world.contributor, composition.id, %{ + lead: content_spec(world.bierce, world.love), + reason: "one", + expected_parent: nil + }) + + {:ok, v2} = + Compositions.create_version(world.contributor, composition.id, %{ + lead: content_spec(world.bierce, world.love), + highlights: [quotation_spec(world.sense, 1)], + reason: "two", + expected_parent: v1.id + }) + + reviews = + for v <- [v1, v2], into: %{} do + {:ok, r} = + Reviews.decide(world.reviewer, v.id, :accepted, reason: "ok", idempotency_key: key()) + + {v.id, r} + end + + Map.merge(world, %{config: config, composition: composition, v1: v1, v2: v2, reviews: reviews}) + end + + # A task on its own connection, which starts when it is sent `{:go, gate}`. + defp gated(gate, fun) do + parent = self() + + Task.async(fn -> + Ecto.Adapters.SQL.Sandbox.checkout(Repo, sandbox: false) + send(parent, {:ready, self()}) + receive do: ({:go, ^gate} -> fun.()) + end) + end + + test "two reviewers publishing from the same pointer: exactly one wins, nothing is overwritten", + ctx do + second = account([:reviewer]) + gate = make_ref() + + tasks = + for {reviewer, version} <- [{ctx.reviewer, ctx.v1}, {second, ctx.v2}] do + gated(gate, fn -> + Publications.publish(reviewer, version.id, + reason: "race", + idempotency_key: key("race"), + expected_pointer: nil + ) + end) + end + + for _ <- tasks, do: assert_receive({:ready, _pid}) + Enum.each(tasks, &send(&1.pid, {:go, gate})) + results = Enum.map(tasks, &Task.await(&1, 15_000)) + + assert [{:ok, receipt}] = Enum.filter(results, &match?({:ok, _}, &1)) + assert [{:error, {:stale_pointer, pointer}}] = Enum.filter(results, &match?({:error, _}, &1)) + + assert pointer == receipt.published_version_id + assert Repo.aggregate(CompositionPublication, :count) == 1 + assert Repo.get!(Composition, ctx.composition.id).current_published_version_id == pointer + end + + # Postgrex logs the connection it drops after the refused COMMIT. + @tag :capture_log + test "writers that skip the service's lock still cannot both commit from one pointer", ctx do + actor = actor!(ctx.reviewer) + parent = self() + + # Each writer inserts a receipt continuing from `nil`, then moves the + # pointer, the way a careless caller would, with no lock first. + writer = fn version, step -> + Task.async(fn -> + Ecto.Adapters.SQL.Sandbox.checkout(Repo, sandbox: false) + receive do: (:start -> :ok) + + try do + Repo.transaction(fn -> + Repo.insert!(%CompositionPublication{ + composition_id: ctx.composition.id, + action: :publish, + authority_kind: :operator, + published_version_id: version.id, + authorizing_review_id: ctx.reviews[version.id].id, + actor_id: actor.id, + reason: "raw", + eligibility_fingerprint: version.eligibility_fingerprint, + idempotency_key: key("raw") + }) + + send(parent, {step, :inserted}) + + Repo.query!( + "UPDATE editorial_compositions SET current_published_version_id = $1 WHERE id = $2", + [version.id, ctx.composition.id] + ) + + send(parent, {step, :moved}) + receive do: (:commit -> :ok) + end) + rescue + e in Postgrex.Error -> {:refused, e.postgres.code, e.postgres.message} + end + end) + end + + first = writer.(ctx.v1, :first) + second = writer.(ctx.v2, :second) + + send(first.pid, :start) + assert_receive {:first, :moved}, 5_000 + + # The second writer's UPDATE waits on the first's row lock; its receipt is + # already in. + send(second.pid, :start) + assert_receive {:second, :inserted}, 5_000 + + send(first.pid, :commit) + assert {:ok, _} = Task.await(first, 15_000) + + send(second.pid, :commit) + + assert {:refused, :integrity_constraint_violation, message} = Task.await(second, 15_000) + assert message =~ "does not match its latest publication receipt" + + assert Repo.get!(Composition, ctx.composition.id).current_published_version_id == ctx.v1.id + assert Repo.aggregate(CompositionPublication, :count) == 1 + end + + @tag :capture_log + test "every deferred check refuses at a real COMMIT", ctx do + item = hd(Compositions.items(ctx.v1.id)) + + commits = fn fun -> + try do + Repo.transaction(fun) + :committed + rescue + e in Postgrex.Error -> {e.postgres.code, e.postgres.message} + end + end + + # A configuration pointer with no activation receipt. + {:ok, v2} = + DevilsDictionary.Curation.Configurations.create_version(ctx.reviewer, ctx.config.id, %{ + reason: "v2", + max_highlights: 1 + }) + + assert {:integrity_constraint_violation, m1} = + commits.(fn -> + Repo.query!( + "UPDATE curation_configurations SET current_version_id = $1 WHERE id = $2", + [v2.id, ctx.config.id] + ) + end) + + assert m1 =~ "latest activation receipt" + + # An item added to a version after its transaction. + assert {:integrity_constraint_violation, m2} = + commits.(fn -> + fields = + item + |> Map.from_struct() + |> Map.drop([:__meta__, :id]) + |> Map.merge(%{role: :highlight, position: 3}) + + Repo.insert!(struct(CompositionItem, fields)) + end) + + assert m2 =~ "arrangement differs" + + # A scope whose members no longer match its signature and receipt. + assert {:integrity_constraint_violation, m3} = + commits.(fn -> + Repo.query!( + "INSERT INTO editorial_composition_memberships (composition_id, object_id, role, inserted_at) VALUES ($1, $2, 'lexeme', now())", + [ctx.composition.id, ctx.amor.object_id] + ) + end) + + assert m3 =~ "scope does not match" + + # A publication pointer with no receipt. + assert {:integrity_constraint_violation, m4} = + commits.(fn -> + Repo.query!( + "UPDATE editorial_compositions SET current_published_version_id = $1 WHERE id = $2", + [ctx.v1.id, ctx.composition.id] + ) + end) + + assert m4 =~ "latest publication receipt" + + assert Repo.get!(Composition, ctx.composition.id).current_published_version_id == nil + assert length(Compositions.items(ctx.v1.id)) == 1 + end + + test "two provisions of one scope make one composition", ctx do + {:ok, other} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.oats.object_id], + language_tag: "en", + reason: "warm up" + }) + + gate = make_ref() + author = account([:contributor]) + + tasks = + for scope <- [ctx.contributor, author] do + gated(gate, fn -> + Compositions.provision(scope, ctx.config.id, %{ + scope_kind: :lexical_page, + lexeme_ids: [ctx.amor.object_id], + language_tag: "en", + reason: "amor's page" + }) + end) + end + + for _ <- tasks, do: assert_receive({:ready, _pid}) + Enum.each(tasks, &send(&1.pid, {:go, gate})) + + assert [{:ok, %{id: a}}, {:ok, %{id: b}}] = Enum.map(tasks, &Task.await(&1, 15_000)) + assert a == b + refute a == other.id + + assert Repo.aggregate( + from(c in Composition, where: c.scope_kind == :lexical_page), + :count + ) == 1 + end +end diff --git a/test/devils_dictionary/curation/publications_test.exs b/test/devils_dictionary/curation/publications_test.exs new file mode 100644 index 00000000..a7967ae8 --- /dev/null +++ b/test/devils_dictionary/curation/publications_test.exs @@ -0,0 +1,560 @@ +defmodule DevilsDictionary.Curation.PublicationsTest do + @moduledoc """ + R1–R4, R6 and R7 of `docs/curation/persistence-slice-1.md`: human review, + explicit publication, the read that withholds without substitution, and + deletion that is never blocked and keeps nothing prohibited. R5, the race, + is `PublicationRaceTest`, on committed connections. + """ + use DevilsDictionary.DataCase, async: true + + import DevilsDictionary.CurationFixtures + + alias DevilsDictionary.Claims.{Assertion, AssertionReview, AssertionRevision} + + alias DevilsDictionary.Curation.{ + CompositionItem, + CompositionPublication, + CompositionReview, + Compositions, + Configurations, + Published, + Publications, + Reviews + } + + alias DevilsDictionary.{Registry, WordFixtures} + + setup do + world = world!() + {config, config_version} = enabled_test_configuration!(world.reviewer, "test-a") + + {:ok, composition} = + Compositions.provision(world.contributor, config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [world.love.object_id], + language_tag: "en", + reason: "curate love" + }) + + {:ok, version} = + Compositions.create_version(world.contributor, composition.id, %{ + lead: content_spec(world.bierce, world.love), + highlights: [quotation_spec(world.sense, 1), content_spec(world.definition, world.love)], + reason: "first arrangement", + expected_parent: nil + }) + + Map.merge(world, %{ + config: config, + config_version: config_version, + composition: composition, + version: version + }) + end + + defp accept!(ctx, version \\ nil) do + {:ok, review} = + Reviews.decide(ctx.reviewer, (version || ctx.version).id, :accepted, + reason: "reads well", + idempotency_key: key("review") + ) + + review + end + + defp publish(ctx, opts \\ []) do + Publications.publish( + ctx.reviewer, + Keyword.get(opts, :version, ctx.version).id, + reason: "ship it", + idempotency_key: Keyword.get(opts, :key, key("publish")), + expected_pointer: Keyword.get(opts, :expected, nil) + ) + end + + # Row counts of every page table routing has (#194), so a test holds + # whether or not those tables exist in the checkout it runs in. + defp page_counts do + %{rows: tables} = + Repo.query!( + "SELECT tablename FROM pg_tables WHERE schemaname = 'public' AND tablename LIKE 'page%' ORDER BY 1" + ) + + for [table] <- tables, into: %{}, do: {table, Repo.aggregate(table, :count)} + end + + defp claim_counts do + {Repo.aggregate(Assertion, :count), Repo.aggregate(AssertionRevision, :count), + Repo.aggregate(AssertionReview, :count)} + end + + describe "review (R1, R2)" do + test "a bot or a non-reviewer cannot review", ctx do + opts = [reason: "fine", idempotency_key: key()] + + assert {:error, :unauthorized} = + Reviews.decide(ctx.contributor, ctx.version.id, :accepted, opts) + + assert {:error, :unauthorized} = Reviews.decide(account(), ctx.version.id, :accepted, opts) + assert {:error, :unauthorized} = Reviews.decide(nil, ctx.version.id, :accepted, opts) + + revoked = account([:reviewer]) + revoke!(revoked) + assert {:error, :unauthorized} = Reviews.decide(revoked, ctx.version.id, :accepted, opts) + + bot = bot_actor!(ctx.sources["wikidata"]) + + for actor <- [bot, actor!(ctx.contributor)] do + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.insert!(%CompositionReview{ + composition_id: ctx.composition.id, + composition_version_id: ctx.version.id, + reviewer_actor_id: actor.id, + decision: :accepted, + reason: "self-approval", + reviewed_eligibility_fingerprint: ctx.version.eligibility_fingerprint, + idempotency_key: key() + }) + end) + + assert message =~ "reviewer role" + end + end + + test "approval publishes nothing and accepts no claim or page", ctx do + before = {claim_counts(), page_counts()} + review = accept!(ctx) + settle!() + + assert review.decision == :accepted + assert review.reviewed_eligibility_fingerprint == ctx.version.eligibility_fingerprint + assert review.reviewer_actor_id == actor!(ctx.reviewer).id + assert Repo.reload!(ctx.composition).current_published_version_id == nil + assert Repo.aggregate(CompositionPublication, :count) == 0 + assert Published.current(ctx.composition.id) == :unpublished + assert {claim_counts(), page_counts()} == before + + # Binding a composition to a page is #194's (#205), and this slice adds + # no binding table. + assert %{rows: [[false]]} = + Repo.query!("SELECT to_regclass('page_composition_bindings') IS NOT NULL") + end + + test "decisions are append-only and idempotent", ctx do + k = key() + opts = [reason: "reads well", idempotency_key: k] + {:ok, review} = Reviews.decide(ctx.reviewer, ctx.version.id, :accepted, opts) + + assert {:ok, ^review} = Reviews.decide(ctx.reviewer, ctx.version.id, :accepted, opts) + + assert {:error, :idempotency_conflict} = + Reviews.decide(ctx.reviewer, ctx.version.id, :rejected, opts) + + assert {:error, :unknown_decision} = + Reviews.decide(ctx.reviewer, ctx.version.id, :approved, opts) + + for sql <- [ + "UPDATE editorial_composition_reviews SET decision = 'rejected' WHERE id = $1", + "DELETE FROM editorial_composition_reviews WHERE id = $1" + ] do + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> Repo.query!(sql, [review.id]) end) + end + + # An acceptance must be of the version's own fingerprint. + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.insert!(%CompositionReview{ + composition_id: ctx.composition.id, + composition_version_id: ctx.version.id, + reviewer_actor_id: actor!(ctx.reviewer).id, + decision: :accepted, + reason: "of something else", + reviewed_eligibility_fingerprint: "another", + idempotency_key: key() + }) + end) + end + + test "an acceptance is refused once the version no longer stands", ctx do + withdraw_revision!(ctx.definition) + + assert {:error, {:ineligible, [{:highlight, 2, :revision_withdrawn}]}} = + Reviews.decide(ctx.reviewer, ctx.version.id, :accepted, + reason: "x", + idempotency_key: key() + ) + + # Rejection still records, with the fingerprint the reviewer saw. + assert {:ok, %CompositionReview{decision: :rejected} = r} = + Reviews.decide(ctx.reviewer, ctx.version.id, :rejected, + reason: "a definition was withdrawn", + idempotency_key: key() + ) + + refute r.reviewed_eligibility_fingerprint == ctx.version.eligibility_fingerprint + end + end + + describe "publication (R3, R4)" do + test "needs an accepted review, then writes a receipt and moves the pointer atomically", + ctx do + assert {:error, :not_approved} = publish(ctx) + review = accept!(ctx) + before = {claim_counts(), page_counts()} + + assert {:ok, %CompositionPublication{} = receipt} = publish(ctx) + settle!() + + assert %{ + action: :publish, + previous_version_id: nil, + published_version_id: published, + authorizing_review_id: authorizing, + eligibility_fingerprint: fingerprint + } = receipt + + assert {published, authorizing, fingerprint} == + {ctx.version.id, review.id, ctx.version.eligibility_fingerprint} + + assert receipt.actor_id == actor!(ctx.reviewer).id + assert Repo.reload!(ctx.composition).current_published_version_id == ctx.version.id + + # Publishing a composition writes no claim, and no page row: showing it + # on a page is a binding, which is #194's. + assert {claim_counts(), page_counts()} == before + end + + test "is idempotent, checks the expected pointer, and refuses a replay of a different act", + ctx do + accept!(ctx) + k = key() + {:ok, receipt} = publish(ctx, key: k) + + assert {:ok, ^receipt} = publish(ctx, key: k) + assert {:error, :already_published} = publish(ctx, expected: ctx.version.id) + + {:ok, v2} = + Compositions.create_version(ctx.contributor, ctx.composition.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + reason: "lead only", + expected_parent: ctx.version.id + }) + + accept!(ctx, v2) + + assert {:error, :idempotency_conflict} = + publish(ctx, version: v2, key: k, expected: ctx.version.id) + + assert {:error, {:stale_pointer, current}} = publish(ctx, version: v2, expected: nil) + assert current == ctx.version.id + + assert {:ok, second} = publish(ctx, version: v2, expected: ctx.version.id) + assert second.previous_version_id == ctx.version.id + assert Repo.aggregate(CompositionPublication, :count) == 2 + settle!() + end + + test "stale fingerprints, a withdrawn approval and a changed configuration refuse", ctx do + accept!(ctx) + withdraw_revision!(ctx.definition) + assert {:error, {:ineligible, [{:highlight, 2, :revision_withdrawn}]}} = publish(ctx) + + restore_revision!(ctx.definition) + + {:ok, _} = + Reviews.decide(ctx.reviewer, ctx.version.id, :needs_review, + reason: "second look", + idempotency_key: key() + ) + + assert {:error, :not_approved} = publish(ctx) + + accept!(ctx) + + {:ok, _} = + Configurations.disable(ctx.reviewer, ctx.config.id, + reason: "pause", + idempotency_key: key(), + expected: ctx.config_version.id + ) + + assert {:error, :configuration_changed} = publish(ctx) + assert Repo.aggregate(CompositionPublication, :count) == 0 + end + + test "the database refuses a pointer without a receipt, and a receipt without the latest acceptance", + ctx do + review = accept!(ctx) + settle!() + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.query!( + "UPDATE editorial_compositions SET current_published_version_id = $1 WHERE id = $2", + [ + ctx.version.id, + ctx.composition.id + ] + ) + end) + + assert message =~ "latest publication receipt" + + {:ok, _} = + Reviews.decide(ctx.reviewer, ctx.version.id, :withdrawn, + reason: "changed my mind", + idempotency_key: key() + ) + + assert {:refused, :integrity_constraint_violation, message} = + refused(fn -> + Repo.insert!(%CompositionPublication{ + composition_id: ctx.composition.id, + action: :publish, + authority_kind: :operator, + published_version_id: ctx.version.id, + authorizing_review_id: review.id, + actor_id: actor!(ctx.reviewer).id, + reason: "on a superseded acceptance", + eligibility_fingerprint: ctx.version.eligibility_fingerprint, + idempotency_key: key() + }) + end) + + assert message =~ "latest review" + end + end + + describe "reading (R6)" do + test "answers the published arrangement and nothing else", ctx do + accept!(ctx) + {:ok, receipt} = publish(ctx) + + assert {:ok, published} = Published.current(ctx.composition.id) + assert published.version.id == ctx.version.id + assert published.receipt.id == receipt.id + assert published.lead.item_object_id == ctx.bierce.object_id + assert Enum.map(published.highlights, & &1.position) == [1, 2] + assert published.withheld == [] + end + + test "withholds an ineligible item and never substitutes another", ctx do + accept!(ctx) + {:ok, _} = publish(ctx) + withdraw_revision!(ctx.definition) + + assert {:ok, published} = Published.current(ctx.composition.id) + assert Enum.map(published.highlights, & &1.position) == [1] + assert published.withheld == [%{role: :highlight, position: 2, reason: :revision_withdrawn}] + + # Record-level takedown and inactive sources withhold too. + restore_revision!(ctx.definition) + disallow_record!(ctx.bierce) + + assert {:ok, %{lead: nil, withheld: [%{role: :lead, reason: :display_restricted}]}} = + Published.current(ctx.composition.id) + end + + test "withholds the whole version when its approval, configuration or scope moves", ctx do + accept!(ctx) + {:ok, _} = publish(ctx) + + {:ok, _} = + Reviews.decide(ctx.reviewer, ctx.version.id, :withdrawn, + reason: "no longer", + idempotency_key: key() + ) + + assert Published.current(ctx.composition.id) == {:withheld, [:approval_withdrawn]} + + accept!(ctx) + assert {:ok, _} = Published.current(ctx.composition.id) + + {:ok, _} = + Configurations.disable(ctx.reviewer, ctx.config.id, + reason: "pause", + idempotency_key: key(), + expected: ctx.config_version.id + ) + + assert Published.current(ctx.composition.id) == {:withheld, [:configuration_changed]} + end + + test "a scope change withholds until a version for the new scope is published", ctx do + {:ok, page} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexical_page, + lexeme_ids: [ctx.love.object_id], + language_tag: "en", + reason: "love's page" + }) + + {:ok, v} = + Compositions.create_version(ctx.contributor, page.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + reason: "page arrangement", + expected_parent: nil + }) + + accept!(ctx, v) + {:ok, _} = publish(ctx, version: v) + + {:ok, _} = + Compositions.change_scope( + ctx.contributor, + page.id, + [ctx.love.object_id, ctx.amor.object_id], + reason: "amor joins the page" + ) + + assert Published.current(page.id) == {:withheld, [:scope_changed]} + settle!() + end + + test "a Bierce entry that appears later withholds a fallback lead, and does not promote itself", + ctx do + {:ok, c} = + Compositions.provision(ctx.contributor, ctx.config.id, %{ + scope_kind: :lexeme, + lexeme_ids: [ctx.oats.object_id], + language_tag: "en", + reason: "oats" + }) + + {:ok, v} = + Compositions.create_version(ctx.contributor, c.id, %{ + lead: content_spec(ctx.oats_definition, ctx.oats), + reason: "no Bierce entry for oats", + expected_parent: nil + }) + + accept!(ctx, v) + {:ok, _} = publish(ctx, version: v) + assert {:ok, %{lead: %{item_object_id: lead}}} = Published.current(c.id) + assert lead == ctx.oats_definition.object_id + + WordFixtures.entry!(ctx, ctx.oats, "bierce", body: "fixture later bierce entry") + + assert {:ok, %{lead: nil, withheld: [%{role: :lead, reason: :priority_source_available}]}} = + Published.current(c.id) + end + + test "withdrawal is its own receipt, and leaves nothing published", ctx do + accept!(ctx) + {:ok, published} = publish(ctx) + + opts = [reason: "take down", idempotency_key: key(), expected_pointer: ctx.version.id] + + assert {:error, :unauthorized} = + Publications.withdraw(ctx.contributor, ctx.composition.id, opts) + + assert {:ok, receipt} = Publications.withdraw(ctx.reviewer, ctx.composition.id, opts) + assert {:ok, ^receipt} = Publications.withdraw(ctx.reviewer, ctx.composition.id, opts) + + assert %{action: :withdraw, previous_version_id: previous, published_version_id: nil} = + receipt + + assert previous == published.published_version_id + assert Published.current(ctx.composition.id) == :unpublished + settle!() + end + end + + describe "deletion and retention (R7)" do + test "a deleted source revision tombstones the item, and nothing prohibited is kept", ctx do + work = WordFixtures.concept!(nil, "fixture work") + {record, record_revision} = owned_record!(ctx, work) + + {:ok, v2} = + Compositions.create_version(ctx.contributor, ctx.composition.id, %{ + lead: content_spec(ctx.bierce, ctx.love), + highlights: [ + quotation_spec(ctx.sense, 1), + content_spec(ctx.definition, ctx.love), + record_work_spec(work, record_revision, ctx.love) + ], + reason: "with a work", + expected_parent: ctx.version.id + }) + + accept!(ctx, v2) + {:ok, _} = publish(ctx, version: v2) + settle!() + + # The source is deleted: a new revision replaces the pinned one, which + # is removed, and the work's source record goes entirely. + pinned = Registry.current_content_revision(ctx.definition.object_id) + + {:ok, _} = + Registry.add_content_revision(ctx.definition.object_id, %{body: "fixture replacement"}) + + Repo.delete_all(from r in "content_revisions", where: r.id == ^pinned.id) + Repo.delete_all(from r in "source_records", where: r.id == ^record.id) + settle!() + + items = Compositions.items(v2.id) + + assert Enum.find(items, &(&1.position == 2 and &1.role == :highlight)).content_revision_id == + nil + + assert Enum.find(items, &(&1.position == 3)).source_record_revision_id == nil + + assert {:ok, published} = Published.current(ctx.composition.id) + assert Enum.map(published.highlights, & &1.position) == [1] + + assert published.withheld == [ + %{role: :highlight, position: 2, reason: :revision_deleted}, + %{role: :highlight, position: 3, reason: :revision_deleted} + ] + + # No curation row holds a source's words, only ids, hashes and locators. + for table <- + ~w(curation_configurations curation_configuration_versions editorial_compositions + editorial_composition_scope_changes editorial_composition_versions + editorial_composition_items editorial_composition_reviews + editorial_composition_publications) do + %{rows: rows} = Repo.query!("SELECT to_jsonb(t)::text FROM #{table} t") + text = rows |> List.flatten() |> Enum.join("\n") + + for words <- [ + "fixture bierce entry body", + "fixture definition body", + "fixture quotation words", + "record payload" + ] do + refute text =~ words, "#{table} keeps #{inspect(words)}" + end + end + end + + test "an item's references can be nulled by a deletion, and never repointed", ctx do + [lead | _] = Compositions.items(ctx.version.id) |> Enum.filter(&(&1.role == :lead)) + other = Registry.current_content_revision(ctx.definition.object_id) + + assert {:refused, :integrity_constraint_violation, _} = + refused(fn -> + Repo.update_all(from(i in CompositionItem, where: i.id == ^lead.id), + set: [content_revision_id: other.id] + ) + end) + end + end + + # The registry's own lifecycle and takedown switches, as a rights change or + # a source withdrawal would set them. + defp withdraw_revision!(content) do + Repo.update_all( + from(r in "content_revisions", where: r.content_id == ^content.object_id and r.is_current), + set: [lifecycle_state: "withdrawn"] + ) + end + + defp restore_revision!(content) do + Repo.update_all( + from(r in "content_revisions", where: r.content_id == ^content.object_id and r.is_current), + set: [lifecycle_state: "active"] + ) + end +end diff --git a/test/mix/tasks/dd_curation_seed_test.exs b/test/mix/tasks/dd_curation_seed_test.exs new file mode 100644 index 00000000..7d046bc5 --- /dev/null +++ b/test/mix/tasks/dd_curation_seed_test.exs @@ -0,0 +1,43 @@ +defmodule Mix.Tasks.Dd.Curation.SeedTest do + @moduledoc """ + `mix dd.curation.seed` writes drafts and identities only. It activates, + approves and publishes nothing, and a re-run changes nothing. + """ + + use DevilsDictionary.DataCase, async: false + + alias DevilsDictionary.Curation.{ + CompositionPublication, + CompositionReview, + Configuration, + ConfigurationActivation, + Configurations, + Profile + } + + alias Mix.Tasks.Dd.Curation.Seed + + setup do + Mix.shell(Mix.Shell.Process) + on_exit(fn -> Mix.shell(Mix.Shell.IO) end) + end + + test "seeds a draft default and five proposed identities, twice to the same effect" do + Seed.run([]) + Seed.run([]) + + assert_received {:mix_shell, :info, ["global-default: draft, version 1 " <> _]} + assert_received {:mix_shell, :info, ["profiles: bierce (proposed), voltaire (proposed)" <> _]} + assert_received {:mix_shell, :info, ["resolution: {:unavailable, :no_active_version}"]} + + assert [%Configuration{state: :draft, current_version_id: nil}] = Repo.all(Configuration) + assert Repo.aggregate(Profile, :count) == 5 + assert Repo.all(from p in Profile, where: p.state != :proposed) == [] + + for schema <- [ConfigurationActivation, CompositionReview, CompositionPublication] do + assert Repo.aggregate(schema, :count) == 0 + end + + assert Configurations.resolve_default() == {:unavailable, :no_active_version} + end +end diff --git a/test/support/curation_fixtures.ex b/test/support/curation_fixtures.ex new file mode 100644 index 00000000..26a41008 --- /dev/null +++ b/test/support/curation_fixtures.ex @@ -0,0 +1,283 @@ +defmodule DevilsDictionary.CurationFixtures do + @moduledoc """ + A small curation world for #196/#201 integrity tests, built through the + registry and the curation services themselves. + + Every text is a placeholder. No words are attributed to Bierce or anyone + else, and no profile gets a dossier about a real person: an admission test + uses a test profile with test references. That rule is the task's (never + invent quotations or biography evidence), and it applies to fixtures too. + """ + + import Ecto.Query + + alias DevilsDictionary.Accounts.{Scope, User} + alias DevilsDictionary.AccountsFixtures + alias DevilsDictionary.Curation.{Configurations, Digest} + alias DevilsDictionary.{Fixtures, Registry, Repo, WordFixtures} + alias DevilsDictionary.Sources.Actor + + @doc "An account with the given roles, and its scope." + def account(roles \\ []) do + user = + AccountsFixtures.user_fixture() + |> Ecto.Changeset.change( + reviewer: :reviewer in roles, + internal_contributor: :contributor in roles + ) + |> Repo.update!() + + Scope.for_user(user) + end + + @doc "Revokes an account's roles." + def revoke!(%Scope{user: user}) do + Repo.update_all(from(u in User, where: u.id == ^user.id), + set: [reviewer: false, internal_contributor: false] + ) + end + + @doc "A `bot` actor, for proving a bot can do none of this." + def bot_actor!(source) do + Repo.insert!(%Actor{actor_kind: :bot, bot_source_id: source.id, label: "fixture bot"}) + end + + @doc "The account's own `user` actor." + def actor!(%Scope{user: user}) do + {:ok, actor} = + Repo.transaction(fn -> DevilsDictionary.Claims.Contributions.account_actor!(user) end) + + actor + end + + @doc """ + The world: the source catalog, a reviewer and a contributor, and words. + + * `love` carries a Bierce entry, a Wiktionary definition, and a Wiktionary + sense with a quotation; + * `oats` carries only a Wiktionary definition (no Bierce entry); + * `amor` is another English word, for scope changes. + """ + def world! do + ctx = Fixtures.seed_catalog!() + reviewer = account([:reviewer]) + contributor = account([:contributor]) + + love = WordFixtures.word!(ctx, "love", ["bierce", "wiktionary"], scope: nil) + oats = WordFixtures.word!(ctx, "oats", ["wiktionary"], scope: nil) + amor = WordFixtures.word!(ctx, "amor", ["wiktionary"], scope: nil) + + bierce = WordFixtures.entry!(ctx, love, "bierce", body: "fixture bierce entry body") + definition = WordFixtures.entry!(ctx, love, "wiktionary", body: "fixture definition body") + oats_definition = WordFixtures.entry!(ctx, oats, "wiktionary", body: "fixture oats body") + + sense = + WordFixtures.sense!(ctx, love, "wiktionary", + examples: [ + %{"type" => "example", "text" => "fixture example"}, + %{"type" => "quotation", "text" => "fixture quotation words", "ref" => "fixture ref"} + ] + ) + + Map.merge(ctx, %{ + reviewer: reviewer, + contributor: contributor, + love: love, + oats: oats, + amor: amor, + bierce: bierce, + definition: definition, + oats_definition: oats_definition, + sense: sense + }) + end + + @doc "Seeds and activates the global default, returning `{configuration, version}`." + def enabled_default!(reviewer) do + %{configuration: c, version: v} = Configurations.seed!() + + {:ok, _} = + Configurations.activate(reviewer, c.id, v.id, + reason: "fixture activation", + idempotency_key: "fixture-activate-#{c.id}-#{v.id}", + expected: nil + ) + + {Repo.reload!(c), v} + end + + @doc "An internal test configuration with one activated version, returning `{configuration, version}`." + def enabled_test_configuration!(reviewer, slug, attrs \\ %{}) do + {:ok, c} = Configurations.create_test_configuration(reviewer, slug, "Test #{slug}") + + {:ok, v} = + Configurations.create_version( + reviewer, + c.id, + Map.merge(%{reason: "fixture version"}, attrs) + ) + + {:ok, _} = + Configurations.activate(reviewer, c.id, v.id, + reason: "fixture activation", + idempotency_key: "fixture-activate-#{slug}", + expected: nil + ) + + {Repo.reload!(c), v} + end + + @doc "A content item spec for its current revision, meaning a lexeme." + def content_spec(content, lexeme) do + %{ + kind: :content, + object_id: content.object_id, + content_revision_id: Registry.current_content_revision(content.object_id).id, + meaning: {:lexeme, lexeme.object_id} + } + end + + @doc "A quotation spec for example `index` of a sense's current revision, meaning that sense." + def quotation_spec(sense, index) do + revision = Registry.current_sense_revision(sense.object_id) + %{"text" => text} = Enum.at(revision.examples, index) + + %{ + kind: :sense_quotation, + object_id: sense.object_id, + sense_revision_id: revision.id, + locator: "quotation:#{index}", + words_sha256: Digest.sha256(text), + meaning: {:sense_revision, revision.id} + } + end + + @catalog "wikidata-famous-v1" + @catalog_row "Q8777422" + + @doc "A work spec pinned to a committed catalog manifest row." + def catalog_spec(lexeme, overrides \\ %{}) do + path = Application.app_dir(:devils_dictionary, "priv/artworks/manifests/#{@catalog}.json") + checksum = path |> File.read!() |> Jason.decode!() |> Map.fetch!("checksum") + + %{ + kind: :work, + catalog: + Map.merge(%{manifest: @catalog, checksum: checksum, identity: @catalog_row}, overrides), + meaning: {:lexeme, lexeme.object_id} + } + end + + @doc """ + A registry work that *is* the committed catalog row: an entity carrying the + row's QID as a verified Wikidata identifier. + """ + def catalog_work!(label \\ "Cupid and Psyche (fixture)") do + WordFixtures.concept!(@catalog_row, label) + end + + @doc """ + A source record that materialized `object`, and its revision id: the + evidence a source-record work item names. + """ + def owned_record!(ctx, object, source_slug \\ "wikidata") do + record = WordFixtures.record!(ctx, source_slug, raw: %{"fixture" => "record payload"}) + now = DateTime.utc_now() + + Repo.insert_all("source_materialized_outputs", [ + %{ + source_record_id: record.id, + output_role: "work", + output_key: "fixture:#{object.object_id}", + output_object_id: object.object_id, + inserted_at: now, + updated_at: now + } + ]) + + revision_id = + Repo.one!( + from r in "source_record_revisions", + where: r.source_record_id == ^record.id, + select: r.id + ) + + {record, revision_id} + end + + @doc "A work spec backed by a source-record revision." + def record_work_spec(object, revision_id, lexeme) do + %{ + kind: :work, + object_id: object.object_id, + source_record_revision_id: revision_id, + meaning: {:lexeme, lexeme.object_id} + } + end + + @doc "The current `defines` claim revision from a content item to a lexeme." + def defines_claim!(content, lexeme) do + Repo.one!( + from r in DevilsDictionary.Claims.AssertionRevision, + join: p in assoc(r, :predicate), + where: + r.subject_object_id == ^content.object_id and r.object_object_id == ^lexeme.object_id and + r.is_current and p.key == "defines" + ) + end + + @doc "Takes down the source record behind a content item's current revision." + def disallow_record!(content) do + revision = Registry.current_content_revision(content.object_id) + + Repo.update_all( + from(rec in "source_records", + join: rev in "source_record_revisions", + on: rev.source_record_id == rec.id, + where: rev.id == ^revision.source_record_revision_id + ), + set: [display_allowed: false] + ) + end + + @doc "Unique idempotency keys." + def key(prefix \\ "key"), do: "#{prefix}-#{System.unique_integer([:positive])}" + + @doc """ + Runs every pending deferred check now, as COMMIT would, then defers them + again. A sandboxed test never commits, so without this it would never see + what the database checks at COMMIT. No constraint in the schema is + `DEFERRABLE INITIALLY IMMEDIATE`, so `ALL DEFERRED` restores the initial + state exactly. + """ + def settle! do + Repo.query!("SET CONSTRAINTS ALL IMMEDIATE") + Repo.query!("SET CONSTRAINTS ALL DEFERRED") + :ok + end + + @doc """ + What the database answers to `fun`, run in a (sub)transaction whose + deferred checks run before it ends: `:accepted`, or `{:refused, code, + detail}`. + """ + def refused(fun) do + result = + try do + Repo.transaction(fn -> + fun.() + Repo.query!("SET CONSTRAINTS ALL IMMEDIATE") + end) + + :accepted + rescue + e in Postgrex.Error -> {:refused, e.postgres.code, e.postgres.message} + e in Ecto.ConstraintError -> {:refused, e.type, e.constraint} + end + + # A savepoint that commits keeps its SET CONSTRAINTS; one that rolls back + # does not. Defer again either way, so the rest of the test is unchanged. + Repo.query!("SET CONSTRAINTS ALL DEFERRED") + result + end +end