Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 180 additions & 0 deletions docs/integrations/met-probe-2026-09-17.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Met Open Access — #99 P0 bounded live probe

**Date:** 2026-09-17 · **Issue:** #99 (P0), tracked by #102 Phase 2
**Ceiling:** 300 requests (#99 P0) · **Spent:** 185 · **Image bytes downloaded:** none
**Credentials:** none exist for this provider; nothing to redact.
**Verdict: GO for P1, with three contract corrections below.**

## Request ledger

| Stage | Endpoint | Requests |
|---|---|---:|
| Catalogue size + five coverage searches | `/v1/objects`, `/v1.1/search` | 6 |
| Search response shape | `/v1.1/search` | 1 |
| Pagination parameters | `/v1.1/search?offset=&limit=` | 3 |
| Sample hydration (100 requested) | `/v1.1/search`, `/v1/objects/{id}` | 101 |
| Retry of the 21 failures | `/v1/objects/{id}` | 21 |
| Ten-meaning tag probe | `/v1.1/search`, `/v1/objects/{id}` | 38 |
| `war` tag vocabulary | `/v1.1/search`, `/v1/objects/{id}` | 15 |
| **Total** | | **185** |

**Declared sample:** the first 100 object IDs from
`/public/collection/v1.1/search?q=*&isHighlight=true&hasImages=true` (reported total 2,290),
hydrated individually. 96 hydrated successfully; 4 never did (below). Percentages are over
those 96. **This sample is highlights-only and is therefore biased upward** on images,
crosswalks and tags; P1's 200-object pilot must include non-highlights before these rates are
treated as catalogue-wide.

## Contract corrections — things #99's table has wrong

### 1. The practical rate limit is far below the documented 80/s, and it fails as 403

#99 records "80 requests per second documented; not exercised" and "Authentication: None — Verified".
At a **0.15 s interval (~6.7 req/s)**, **21 of 100** object hydrations returned **HTTP 403 Forbidden**.
(The 101 in the ledger row is those 100 hydrations plus the one `/v1.1/search` that produced the IDs.)
Re-running those 21 once each at a **1.5 s interval recovered 17**. Four were still 403 after both
attempts — the initial fetch and that single retry — and were not tried again:
`194243`, `19434`, `194432`, `194622`. No ID was requested more than twice; the ledger's 21-request
retry row is the complete retry spend.

So 403 from the Met is **mostly a throttling signal, not an authentication verdict**, and it is
not accompanied by a 429 or a `Retry-After`.

**Direct consequence for the code that just landed in #104.** `Discovery.Transport` maps 403 to
a terminal error with no retry:

```elixir
{:ok, %Req.Response{status: status}} when status in [401, 403] ->
{:error, "authentication_failed"}
```

For a keyless provider that emits transient 403s, that is both a **false diagnosis** and a
**hard failure where one retry would have succeeded**: a Met run would fail ~20% of its object
fetches and report `authentication_failed` for a provider that has no authentication. This must be
resolved before the Met provider is written. Options, in preference order:

1. Let a provider classify its own HTTP statuses (a `retryable_status?/1`-shaped optional callback),
keeping the current behaviour as the default. Fits the Phase 1 contract direction.
2. Treat 403 as retryable **only** when the provider declares no credentials.
3. Run the Met through the shared coordinator at a conservative fixed interval and accept the loss.

Recommend (1). It is small, it is the same "providers own their differences" principle Phase 1
was built on, and (3) alone does not fix the misleading error code.

**Revised guidance for the ledger:** ~1 request/second sustained. 80/s is not achievable and should
be struck from the table.

### 2. `rightsAndReproduction` is empty everywhere, so it cannot cross-check `isPublicDomain`

#99 P0 asks to "confirm `isPublicDomain` semantics against `rightsAndReproduction`". On this sample
`rightsAndReproduction` was **non-empty on 0 of 96 objects**, so the cross-check is not available.
What the sample shows instead:

- `isPublicDomain` true: **66/96 (68%)**
- of those, carrying `primaryImage`: **66/66 (100%)**
- **not** public domain **and** carrying a `primaryImage` URL: **0**

`creditLine` is present on **96/96 (100%)** and is the paired attribution.

**These are sample observations, not an established invariant, and P1 must not filter records on
them yet.** Two reasons to hold them at arm's length. The sample was drawn with `hasImages=true`,
so the first line is close to tautological by construction — what it does not explain is why 30
objects matched `hasImages=true` and still carried no `primaryImage`, which suggests `hasImages`
covers `additionalImages` too and is not the same gate. And the sample is highlights-only, which
biases image coverage upward exactly where this claim lives.

Strike the `rightsAndReproduction` cross-check — it is unavailable, and that much is settled.
Record `isPublicDomain` ⇒ `primaryImage` and `not isPublicDomain` ⇒ no image as a **hypothesis to
confirm**, promotable to a contract invariant only once P1's non-highlight sample reproduces it or
the Met's own published terms state it.

### 3. `/v1.1/search` takes `offset` and `limit` — the Phase 1 GET path is the right shape

Confirmed live. `limit=20` returns 20; `offset=100` returns a different first ID
(`20121` vs `10065`) against a stable `total` of 2,290. Default page size is 100.
This is exactly the GET + offset transport #104 built, so no further transport work is needed
beyond the 403 question above.

## Measurements

### Identity crosswalk — the strongest result in the probe

| | Rate |
|---|---:|
| `objectWikidata_URL` present | **81/96 (84%)** |
| `artistWikidata_URL` present | 83/96 (86%) |
| ≥1 constituent with `constituentWikidata_URL` | 87/96 (90%) |
| Those 81 QIDs carrying **P3634** on Wikidata | **81/81 (100%)** |
| P3634 value **agrees** with the Met object ID | **81/81 (100%)** |
| Disagreements opening reconciliation | **0** |

Both directions agree perfectly on this sample. For comparison, the whole Wikidata universe of
**P11005 (Artsy)** is **48 items**; the Met crosswalk is a working identity spine, Artsy's is a
curiosity. The 15 objects with no QID are the source-supported-entry case #99 says must be
exercised — it is real and it is ~16% even among highlights.

### Tags

| | Rate |
|---|---:|
| Objects carrying ≥1 tag | **65/96 (68%)** |
| Tag rows total | 215 |
| Tag rows carrying `Wikidata_URL` | **215/215 (100%)** |
| Tag rows carrying `AAT_URL` | 210/215 (97%) |
| Objects where *every* tag has a QID | 65/65 (100%) |
| Distinct tag terms in the sample | 123 |

**Every tag carries a Wikidata QID.** #99's rule that "a tag's Wikidata QID is the mapping key,
never its English term" is not just safe, it is free. This is a materially better vocabulary than
Artsy genes, which have no QIDs at all.

### The ten meanings — the finding that should shape P2

Probed per #99 through tags rather than text search. **Exact tag matches: 0 of 10.**

| Meaning | Met hits (`hasImages`) | Exact tag | Closest tags observed |
|---|---:|---|---|
| nepotism | 1 | no | Men, Profiles |
| war | 4,826 | no | **World War I (Q361)**, **American Civil War (Q8676)**, Victory (Q308902) |
| grief | 190 | no | Female Nudes, Men |
| love | 1,757 | no | Flowers |
| power | 3,773 | no | Abstraction, Trains |
| family | 10,249 | no | Architecture, Buildings |
| solitude | 33 | no | Lovers, Trees |
| justice | 557 | no | Female Nudes, Globes |
| bank | 4,624 | no | Birds |
| Pop Art | 96 | no | Flowers, Food |

The Met's tag vocabulary is **depictive, not conceptual**: it says what is *shown* (Men, Women,
Portraits, Boats, Flowers, Violins, Eagles), not what a work is *about*. Abstract senses —
nepotism, grief, justice, solitude, power — have no tag and will not get one.

This is not fatal, and it is not the Artsy failure repeated. Hydrating 14 highlight-with-image
results for `war` (87 such hits) produced no generic "War" tag but did produce
**World War I (Q361)** and **American Civil War (Q8676)**. So the `war` sense is servable by a
**`broader`/`related` mapping onto several specific-conflict QIDs**, which is precisely the mapping
type #99 already specifies. What P2 cannot promise is one exact tag per abstract sense.

**Recommendation for P2:** budget for sense mappings that carry a *set* of tag QIDs with an explicit
relation type, and accept that concrete nouns will be well served long before abstract ones.
Set that expectation in #102 now rather than discovering it at the shelf.

## Scale, for the record

| | Artsy route (Wikidata P11005) | Met Open Access |
|---|---:|---:|
| Addressable universe | 48 items | 502,782 objects |
| `war` | 0 | 4,826 with images; 87 highlight-with-image |
| Rembrandt / Van Gogh / Warhol / Hockney | 1 / 0 / 0 / 0 | 1,095 / 436 / 302 / 116 |

## Go / no-go

**GO for P1**, conditional on the 403 handling being settled first, because P1's 200-object pilot
will hit it ~20% of the time and would otherwise report `authentication_failed` throughout.

Open before P1:
1. Decide the 403 handling (recommend the optional provider callback).
2. Re-measure crosswalk, tag and image rates on a **non-highlight** sample; the numbers here are
an upper bound.
3. Record in #102 that abstract senses are not tag-servable, so Phase 2's acceptance is not written
against words like `nepotism`.