diff --git a/docs/integrations/met-probe-2026-09-17.md b/docs/integrations/met-probe-2026-09-17.md new file mode 100644 index 00000000..993b42c2 --- /dev/null +++ b/docs/integrations/met-probe-2026-09-17.md @@ -0,0 +1,180 @@ +# Met Open Access — #99 P0 bounded live probe + +**Date:** 2026-09-17 · **Issue:** #99 (P0), tracked by #102 Phase 2 +**Ceiling:** 300 requests (#99 P0) · **Spent:** 185 · **Image bytes downloaded:** none +**Credentials:** none exist for this provider; nothing to redact. +**Verdict: GO for P1, with three contract corrections below.** + +## Request ledger + +| Stage | Endpoint | Requests | +|---|---|---:| +| Catalogue size + five coverage searches | `/v1/objects`, `/v1.1/search` | 6 | +| Search response shape | `/v1.1/search` | 1 | +| Pagination parameters | `/v1.1/search?offset=&limit=` | 3 | +| Sample hydration (100 requested) | `/v1.1/search`, `/v1/objects/{id}` | 101 | +| Retry of the 21 failures | `/v1/objects/{id}` | 21 | +| Ten-meaning tag probe | `/v1.1/search`, `/v1/objects/{id}` | 38 | +| `war` tag vocabulary | `/v1.1/search`, `/v1/objects/{id}` | 15 | +| **Total** | | **185** | + +**Declared sample:** the first 100 object IDs from +`/public/collection/v1.1/search?q=*&isHighlight=true&hasImages=true` (reported total 2,290), +hydrated individually. 96 hydrated successfully; 4 never did (below). Percentages are over +those 96. **This sample is highlights-only and is therefore biased upward** on images, +crosswalks and tags; P1's 200-object pilot must include non-highlights before these rates are +treated as catalogue-wide. + +## Contract corrections — things #99's table has wrong + +### 1. The practical rate limit is far below the documented 80/s, and it fails as 403 + +#99 records "80 requests per second documented; not exercised" and "Authentication: None — Verified". +At a **0.15 s interval (~6.7 req/s)**, **21 of 100** object hydrations returned **HTTP 403 Forbidden**. +(The 101 in the ledger row is those 100 hydrations plus the one `/v1.1/search` that produced the IDs.) +Re-running those 21 once each at a **1.5 s interval recovered 17**. Four were still 403 after both +attempts — the initial fetch and that single retry — and were not tried again: +`194243`, `19434`, `194432`, `194622`. No ID was requested more than twice; the ledger's 21-request +retry row is the complete retry spend. + +So 403 from the Met is **mostly a throttling signal, not an authentication verdict**, and it is +not accompanied by a 429 or a `Retry-After`. + +**Direct consequence for the code that just landed in #104.** `Discovery.Transport` maps 403 to +a terminal error with no retry: + +```elixir +{:ok, %Req.Response{status: status}} when status in [401, 403] -> + {:error, "authentication_failed"} +``` + +For a keyless provider that emits transient 403s, that is both a **false diagnosis** and a +**hard failure where one retry would have succeeded**: a Met run would fail ~20% of its object +fetches and report `authentication_failed` for a provider that has no authentication. This must be +resolved before the Met provider is written. Options, in preference order: + +1. Let a provider classify its own HTTP statuses (a `retryable_status?/1`-shaped optional callback), + keeping the current behaviour as the default. Fits the Phase 1 contract direction. +2. Treat 403 as retryable **only** when the provider declares no credentials. +3. Run the Met through the shared coordinator at a conservative fixed interval and accept the loss. + +Recommend (1). It is small, it is the same "providers own their differences" principle Phase 1 +was built on, and (3) alone does not fix the misleading error code. + +**Revised guidance for the ledger:** ~1 request/second sustained. 80/s is not achievable and should +be struck from the table. + +### 2. `rightsAndReproduction` is empty everywhere, so it cannot cross-check `isPublicDomain` + +#99 P0 asks to "confirm `isPublicDomain` semantics against `rightsAndReproduction`". On this sample +`rightsAndReproduction` was **non-empty on 0 of 96 objects**, so the cross-check is not available. +What the sample shows instead: + +- `isPublicDomain` true: **66/96 (68%)** +- of those, carrying `primaryImage`: **66/66 (100%)** +- **not** public domain **and** carrying a `primaryImage` URL: **0** + +`creditLine` is present on **96/96 (100%)** and is the paired attribution. + +**These are sample observations, not an established invariant, and P1 must not filter records on +them yet.** Two reasons to hold them at arm's length. The sample was drawn with `hasImages=true`, +so the first line is close to tautological by construction — what it does not explain is why 30 +objects matched `hasImages=true` and still carried no `primaryImage`, which suggests `hasImages` +covers `additionalImages` too and is not the same gate. And the sample is highlights-only, which +biases image coverage upward exactly where this claim lives. + +Strike the `rightsAndReproduction` cross-check — it is unavailable, and that much is settled. +Record `isPublicDomain` ⇒ `primaryImage` and `not isPublicDomain` ⇒ no image as a **hypothesis to +confirm**, promotable to a contract invariant only once P1's non-highlight sample reproduces it or +the Met's own published terms state it. + +### 3. `/v1.1/search` takes `offset` and `limit` — the Phase 1 GET path is the right shape + +Confirmed live. `limit=20` returns 20; `offset=100` returns a different first ID +(`20121` vs `10065`) against a stable `total` of 2,290. Default page size is 100. +This is exactly the GET + offset transport #104 built, so no further transport work is needed +beyond the 403 question above. + +## Measurements + +### Identity crosswalk — the strongest result in the probe + +| | Rate | +|---|---:| +| `objectWikidata_URL` present | **81/96 (84%)** | +| `artistWikidata_URL` present | 83/96 (86%) | +| ≥1 constituent with `constituentWikidata_URL` | 87/96 (90%) | +| Those 81 QIDs carrying **P3634** on Wikidata | **81/81 (100%)** | +| P3634 value **agrees** with the Met object ID | **81/81 (100%)** | +| Disagreements opening reconciliation | **0** | + +Both directions agree perfectly on this sample. For comparison, the whole Wikidata universe of +**P11005 (Artsy)** is **48 items**; the Met crosswalk is a working identity spine, Artsy's is a +curiosity. The 15 objects with no QID are the source-supported-entry case #99 says must be +exercised — it is real and it is ~16% even among highlights. + +### Tags + +| | Rate | +|---|---:| +| Objects carrying ≥1 tag | **65/96 (68%)** | +| Tag rows total | 215 | +| Tag rows carrying `Wikidata_URL` | **215/215 (100%)** | +| Tag rows carrying `AAT_URL` | 210/215 (97%) | +| Objects where *every* tag has a QID | 65/65 (100%) | +| Distinct tag terms in the sample | 123 | + +**Every tag carries a Wikidata QID.** #99's rule that "a tag's Wikidata QID is the mapping key, +never its English term" is not just safe, it is free. This is a materially better vocabulary than +Artsy genes, which have no QIDs at all. + +### The ten meanings — the finding that should shape P2 + +Probed per #99 through tags rather than text search. **Exact tag matches: 0 of 10.** + +| Meaning | Met hits (`hasImages`) | Exact tag | Closest tags observed | +|---|---:|---|---| +| nepotism | 1 | no | Men, Profiles | +| war | 4,826 | no | **World War I (Q361)**, **American Civil War (Q8676)**, Victory (Q308902) | +| grief | 190 | no | Female Nudes, Men | +| love | 1,757 | no | Flowers | +| power | 3,773 | no | Abstraction, Trains | +| family | 10,249 | no | Architecture, Buildings | +| solitude | 33 | no | Lovers, Trees | +| justice | 557 | no | Female Nudes, Globes | +| bank | 4,624 | no | Birds | +| Pop Art | 96 | no | Flowers, Food | + +The Met's tag vocabulary is **depictive, not conceptual**: it says what is *shown* (Men, Women, +Portraits, Boats, Flowers, Violins, Eagles), not what a work is *about*. Abstract senses — +nepotism, grief, justice, solitude, power — have no tag and will not get one. + +This is not fatal, and it is not the Artsy failure repeated. Hydrating 14 highlight-with-image +results for `war` (87 such hits) produced no generic "War" tag but did produce +**World War I (Q361)** and **American Civil War (Q8676)**. So the `war` sense is servable by a +**`broader`/`related` mapping onto several specific-conflict QIDs**, which is precisely the mapping +type #99 already specifies. What P2 cannot promise is one exact tag per abstract sense. + +**Recommendation for P2:** budget for sense mappings that carry a *set* of tag QIDs with an explicit +relation type, and accept that concrete nouns will be well served long before abstract ones. +Set that expectation in #102 now rather than discovering it at the shelf. + +## Scale, for the record + +| | Artsy route (Wikidata P11005) | Met Open Access | +|---|---:|---:| +| Addressable universe | 48 items | 502,782 objects | +| `war` | 0 | 4,826 with images; 87 highlight-with-image | +| Rembrandt / Van Gogh / Warhol / Hockney | 1 / 0 / 0 / 0 | 1,095 / 436 / 302 / 116 | + +## Go / no-go + +**GO for P1**, conditional on the 403 handling being settled first, because P1's 200-object pilot +will hit it ~20% of the time and would otherwise report `authentication_failed` throughout. + +Open before P1: +1. Decide the 403 handling (recommend the optional provider callback). +2. Re-measure crosswalk, tag and image rates on a **non-highlight** sample; the numbers here are + an upper bound. +3. Record in #102 that abstract senses are not tag-servable, so Phase 2's acceptance is not written + against words like `nepotism`.