From 540f45ea782958fd49ad77c7a8897f2b7b5e1b56 Mon Sep 17 00:00:00 2001 From: Holden Thomas Date: Thu, 17 Sep 2026 18:30:31 +0200 Subject: [PATCH 1/2] Record the Met Open Access P0 probe for #99 185 of a 300-request ceiling, no image bytes, no credentials involved. The crosswalk is the headline: 84% of sampled objects name a QID, and all 81 of those QIDs carry P3634 pointing back at the same object, with zero disagreements. Every tag row carries a Wikidata QID. Three corrections to #99's contract table. The Met emits transient 403s at ~6.7 req/s, far below its documented 80/s, and 17 of 21 recovered on retry at 1.5s -- which collides with Transport's terminal 403 to "authentication_failed" for a provider that has no authentication. rightsAndReproduction is empty across the sample so it cannot cross-check isPublicDomain; the image gate fails closed on its own instead. And v1.1 search does take offset and limit, so the GET path #104 built is right. The finding that should shape P2: the tag vocabulary is depictive, not conceptual. None of the ten meanings has an exact tag. "war" is servable through broader mappings onto World War I and American Civil War, but abstract senses like nepotism and grief are not tag-servable at all. Co-Authored-By: Claude Opus 5 --- docs/integrations/met-probe-2026-09-17.md | 167 ++++++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 docs/integrations/met-probe-2026-09-17.md diff --git a/docs/integrations/met-probe-2026-09-17.md b/docs/integrations/met-probe-2026-09-17.md new file mode 100644 index 00000000..a11de9e7 --- /dev/null +++ b/docs/integrations/met-probe-2026-09-17.md @@ -0,0 +1,167 @@ +# Met Open Access — #99 P0 bounded live probe + +**Date:** 2026-09-17 · **Issue:** #99 (P0), tracked by #102 Phase 2 +**Ceiling:** 300 requests (#99 P0) · **Spent:** 185 · **Image bytes downloaded:** none +**Credentials:** none exist for this provider; nothing to redact. +**Verdict: GO for P1, with three contract corrections below.** + +## Request ledger + +| Stage | Endpoint | Requests | +|---|---|---:| +| Catalogue size + five coverage searches | `/v1/objects`, `/v1.1/search` | 6 | +| Search response shape | `/v1.1/search` | 1 | +| Pagination parameters | `/v1.1/search?offset=&limit=` | 3 | +| Sample hydration (100 requested) | `/v1.1/search`, `/v1/objects/{id}` | 101 | +| Retry of the 21 failures | `/v1/objects/{id}` | 21 | +| Ten-meaning tag probe | `/v1.1/search`, `/v1/objects/{id}` | 38 | +| `war` tag vocabulary | `/v1.1/search`, `/v1/objects/{id}` | 15 | +| **Total** | | **185** | + +**Declared sample:** the first 100 object IDs from +`/public/collection/v1.1/search?q=*&isHighlight=true&hasImages=true` (reported total 2,290), +hydrated individually. 96 hydrated successfully; 4 never did (below). Percentages are over +those 96. **This sample is highlights-only and is therefore biased upward** on images, +crosswalks and tags; P1's 200-object pilot must include non-highlights before these rates are +treated as catalogue-wide. + +## Contract corrections — things #99's table has wrong + +### 1. The practical rate limit is far below the documented 80/s, and it fails as 403 + +#99 records "80 requests per second documented; not exercised" and "Authentication: None — Verified". +At a **0.15 s interval (~6.7 req/s)**, **21 of 101** object hydrations returned **HTTP 403 Forbidden**. +Re-running those 21 at a **1.5 s interval recovered 17**. Four remained 403 across four attempts: +`194243`, `19434`, `194432`, `194622`. + +So 403 from the Met is **mostly a throttling signal, not an authentication verdict**, and it is +not accompanied by a 429 or a `Retry-After`. + +**Direct consequence for the code that just landed in #104.** `Discovery.Transport` maps 403 to +a terminal error with no retry: + +```elixir +{:ok, %Req.Response{status: status}} when status in [401, 403] -> + {:error, "authentication_failed"} +``` + +For a keyless provider that emits transient 403s, that is both a **false diagnosis** and a +**hard failure where one retry would have succeeded**: a Met run would fail ~20% of its object +fetches and report `authentication_failed` for a provider that has no authentication. This must be +resolved before the Met provider is written. Options, in preference order: + +1. Let a provider classify its own HTTP statuses (a `retryable_status?/1`-shaped optional callback), + keeping the current behaviour as the default. Fits the Phase 1 contract direction. +2. Treat 403 as retryable **only** when the provider declares no credentials. +3. Run the Met through the shared coordinator at a conservative fixed interval and accept the loss. + +Recommend (1). It is small, it is the same "providers own their differences" principle Phase 1 +was built on, and (3) alone does not fix the misleading error code. + +**Revised guidance for the ledger:** ~1 request/second sustained. 80/s is not achievable and should +be struck from the table. + +### 2. `rightsAndReproduction` is empty everywhere, so it cannot cross-check `isPublicDomain` + +#99 P0 asks to "confirm `isPublicDomain` semantics against `rightsAndReproduction`". On this sample +`rightsAndReproduction` was **non-empty on 0 of 96 objects**, so the cross-check is not available. +What is available is cleaner: + +- `isPublicDomain` true: **66/96 (68%)** +- of those, carrying `primaryImage`: **66/66 (100%)** +- **not** public domain **and** carrying a `primaryImage` URL: **0** + +The gate fails closed on its own. `creditLine` is present on **96/96 (100%)** and is the paired +attribution. Strike the `rightsAndReproduction` cross-check; assert the `isPublicDomain` ⇒ image, +`not isPublicDomain` ⇒ no image invariant instead. + +### 3. `/v1.1/search` takes `offset` and `limit` — the Phase 1 GET path is the right shape + +Confirmed live. `limit=20` returns 20; `offset=100` returns a different first ID +(`20121` vs `10065`) against a stable `total` of 2,290. Default page size is 100. +This is exactly the GET + offset transport #104 built, so no further transport work is needed +beyond the 403 question above. + +## Measurements + +### Identity crosswalk — the strongest result in the probe + +| | Rate | +|---|---:| +| `objectWikidata_URL` present | **81/96 (84%)** | +| `artistWikidata_URL` present | 83/96 (86%) | +| ≥1 constituent with `constituentWikidata_URL` | 87/96 (90%) | +| Those 81 QIDs carrying **P3634** on Wikidata | **81/81 (100%)** | +| P3634 value **agrees** with the Met object ID | **81/81 (100%)** | +| Disagreements opening reconciliation | **0** | + +Both directions agree perfectly on this sample. For comparison, the whole Wikidata universe of +**P11005 (Artsy)** is **48 items**; the Met crosswalk is a working identity spine, Artsy's is a +curiosity. The 15 objects with no QID are the source-supported-entry case #99 says must be +exercised — it is real and it is ~16% even among highlights. + +### Tags + +| | Rate | +|---|---:| +| Objects carrying ≥1 tag | **65/96 (68%)** | +| Tag rows total | 215 | +| Tag rows carrying `Wikidata_URL` | **215/215 (100%)** | +| Tag rows carrying `AAT_URL` | 210/215 (97%) | +| Objects where *every* tag has a QID | 65/65 (100%) | +| Distinct tag terms in the sample | 123 | + +**Every tag carries a Wikidata QID.** #99's rule that "a tag's Wikidata QID is the mapping key, +never its English term" is not just safe, it is free. This is a materially better vocabulary than +Artsy genes, which have no QIDs at all. + +### The ten meanings — the finding that should shape P2 + +Probed per #99 through tags rather than text search. **Exact tag matches: 0 of 10.** + +| Meaning | Met hits (`hasImages`) | Exact tag | Closest tags observed | +|---|---:|---|---| +| nepotism | 1 | no | Men, Profiles | +| war | 4,826 | no | **World War I (Q361)**, **American Civil War (Q8676)**, Victory (Q308902) | +| grief | 190 | no | Female Nudes, Men | +| love | 1,757 | no | Flowers | +| power | 3,773 | no | Abstraction, Trains | +| family | 10,249 | no | Architecture, Buildings | +| solitude | 33 | no | Lovers, Trees | +| justice | 557 | no | Female Nudes, Globes | +| bank | 4,624 | no | Birds | +| Pop Art | 96 | no | Flowers, Food | + +The Met's tag vocabulary is **depictive, not conceptual**: it says what is *shown* (Men, Women, +Portraits, Boats, Flowers, Violins, Eagles), not what a work is *about*. Abstract senses — +nepotism, grief, justice, solitude, power — have no tag and will not get one. + +This is not fatal, and it is not the Artsy failure repeated. Hydrating 14 highlight-with-image +results for `war` (87 such hits) produced no generic "War" tag but did produce +**World War I (Q361)** and **American Civil War (Q8676)**. So the `war` sense is servable by a +**`broader`/`related` mapping onto several specific-conflict QIDs**, which is precisely the mapping +type #99 already specifies. What P2 cannot promise is one exact tag per abstract sense. + +**Recommendation for P2:** budget for sense mappings that carry a *set* of tag QIDs with an explicit +relation type, and accept that concrete nouns will be well served long before abstract ones. +Set that expectation in #102 now rather than discovering it at the shelf. + +## Scale, for the record + +| | Artsy route (Wikidata P11005) | Met Open Access | +|---|---:|---:| +| Addressable universe | 48 items | 502,782 objects | +| `war` | 0 | 4,826 with images; 87 highlight-with-image | +| Rembrandt / Van Gogh / Warhol / Hockney | 1 / 0 / 0 / 0 | 1,095 / 436 / 302 / 116 | + +## Go / no-go + +**GO for P1**, conditional on the 403 handling being settled first, because P1's 200-object pilot +will hit it ~20% of the time and would otherwise report `authentication_failed` throughout. + +Open before P1: +1. Decide the 403 handling (recommend the optional provider callback). +2. Re-measure crosswalk, tag and image rates on a **non-highlight** sample; the numbers here are + an upper bound. +3. Record in #102 that abstract senses are not tag-servable, so Phase 2's acceptance is not written + against words like `nepotism`. From 4efcb877fd2284ab690cf6066fae4d828697e2e1 Mon Sep 17 00:00:00 2001 From: Holden Thomas Date: Thu, 17 Sep 2026 19:13:15 +0200 Subject: [PATCH 2/2] Correct three overstatements in the Met probe report The hydration denominator is 100, not 101 -- the ledger's 101 counts the one search that produced the IDs alongside the 100 object fetches. "Four attempts" for the four IDs that never recovered contradicted the ledger's single 21-request retry row and would have put the total at 193, not the 185 claimed throughout. Each of those IDs was tried twice. The image finding was promoted to a contract invariant off a sample drawn with hasImages=true and highlights-only, which is the one place the bias bites hardest -- and 30 objects matched hasImages=true with no primaryImage anyway, so hasImages is not that gate. It stays a hypothesis until P1's non-highlight sample reproduces it. P1 must not filter records on it yet. Co-Authored-By: Claude Opus 5 --- docs/integrations/met-probe-2026-09-17.md | 27 +++++++++++++++++------ 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/docs/integrations/met-probe-2026-09-17.md b/docs/integrations/met-probe-2026-09-17.md index a11de9e7..993b42c2 100644 --- a/docs/integrations/met-probe-2026-09-17.md +++ b/docs/integrations/met-probe-2026-09-17.md @@ -30,9 +30,12 @@ treated as catalogue-wide. ### 1. The practical rate limit is far below the documented 80/s, and it fails as 403 #99 records "80 requests per second documented; not exercised" and "Authentication: None — Verified". -At a **0.15 s interval (~6.7 req/s)**, **21 of 101** object hydrations returned **HTTP 403 Forbidden**. -Re-running those 21 at a **1.5 s interval recovered 17**. Four remained 403 across four attempts: -`194243`, `19434`, `194432`, `194622`. +At a **0.15 s interval (~6.7 req/s)**, **21 of 100** object hydrations returned **HTTP 403 Forbidden**. +(The 101 in the ledger row is those 100 hydrations plus the one `/v1.1/search` that produced the IDs.) +Re-running those 21 once each at a **1.5 s interval recovered 17**. Four were still 403 after both +attempts — the initial fetch and that single retry — and were not tried again: +`194243`, `19434`, `194432`, `194622`. No ID was requested more than twice; the ledger's 21-request +retry row is the complete retry spend. So 403 from the Met is **mostly a throttling signal, not an authentication verdict**, and it is not accompanied by a 429 or a `Retry-After`. @@ -65,15 +68,25 @@ be struck from the table. #99 P0 asks to "confirm `isPublicDomain` semantics against `rightsAndReproduction`". On this sample `rightsAndReproduction` was **non-empty on 0 of 96 objects**, so the cross-check is not available. -What is available is cleaner: +What the sample shows instead: - `isPublicDomain` true: **66/96 (68%)** - of those, carrying `primaryImage`: **66/66 (100%)** - **not** public domain **and** carrying a `primaryImage` URL: **0** -The gate fails closed on its own. `creditLine` is present on **96/96 (100%)** and is the paired -attribution. Strike the `rightsAndReproduction` cross-check; assert the `isPublicDomain` ⇒ image, -`not isPublicDomain` ⇒ no image invariant instead. +`creditLine` is present on **96/96 (100%)** and is the paired attribution. + +**These are sample observations, not an established invariant, and P1 must not filter records on +them yet.** Two reasons to hold them at arm's length. The sample was drawn with `hasImages=true`, +so the first line is close to tautological by construction — what it does not explain is why 30 +objects matched `hasImages=true` and still carried no `primaryImage`, which suggests `hasImages` +covers `additionalImages` too and is not the same gate. And the sample is highlights-only, which +biases image coverage upward exactly where this claim lives. + +Strike the `rightsAndReproduction` cross-check — it is unavailable, and that much is settled. +Record `isPublicDomain` ⇒ `primaryImage` and `not isPublicDomain` ⇒ no image as a **hypothesis to +confirm**, promotable to a contract invariant only once P1's non-highlight sample reproduces it or +the Met's own published terms state it. ### 3. `/v1.1/search` takes `offset` and `limit` — the Phase 1 GET path is the right shape