Skip to content

MET-WP7-PREREQ-02: durable CloudFront unresolved-DOI quarantine - #933

Merged
ja573 merged 2 commits into
feature/metricsfrom
feature/metrics--wp7-prereq-02-doi-quarantine
Sep 21, 2026
Merged

ja573 merged 2 commits into
feature/metricsfrom
feature/metrics--wp7-prereq-02-doi-quarantine

Conversation

@ja573

@ja573 ja573 commented Sep 21, 2026

Copy link
Copy Markdown
Member

MET-WP7-PREREQ-02 — durable CloudFront unresolved-DOI quarantine

Implements the approved and frozen specification in #930, under CTO implementation authorization #930 comment 5759028300.

Risk: CRITICAL. Status: DRAFT. It needs an independent exact-head CRITICAL source review and a separate CTO merge authorization. Do not merge.

Binding Value
Base feature/metrics @ 15b6efd089dee25d988cbb316c082be1780bf56f (tree 53411810ad68014598a32376f6e4e929465002e8)
Branch feature/metrics--wp7-prereq-02-doi-quarantine
Migration thoth-api/migrations/20260921_v1.9.0 (additive; guarded down)
Write budget 13 of the 14 authorized paths; conditional path 13 (GraphQL lifecycle tests) not used

What changes

  • Migration. Creates only metric_identifier_quarantine:

    • one row per REJECTED / UNKNOWN_DOI provenance row (UNIQUE record_provenance_id);
    • four non-cascading FKs;
    • locale-independent nonblank checks and period_end > period_start;
    • no DOI-format CHECK, no request identity, no seed or backfill.

    down.sql locks the table and refuses while any row exists.

  • Coordinator. RequestScope retains the source it already locks FOR SHARE. A quarantine row is written only when the locked driver_key is exactly cloudfront and the observation carries none of publication_isbn, publication_type, institution_ror, source_record_id or source_row_number. It is written in the same transaction as the unchanged rejected provenance, sanitized import error and invalid counter.

    • The row stays REJECTED / UNKNOWN_DOI, with no record, revision or delta.
    • The DOI is stored byte for byte.
    • A replay writes nothing.
  • Checkpoint update. Locks are taken in the order checkpoint FOR UPDATE, then import, source account and source, each FOR SHARE. For CloudFront COMPLETED_WITH_ERRORS imports only, the three-way partition is derived from provenance and quarantine rows:

    • inconsistent: fails as INTERNAL_STATE_INCONSISTENCY, with no write or release;
    • consistent but ineligible: the ordinary release, with no cursor;
    • quarantine-only with exact one-day COMPLETE coverage: records or replaces the cursor entry and releases, while last_successful_period_end stays unadvanced.
  • Docs. Contract register §3.4 (the narrow exception only), CHANGELOG.md, and the implementation report.

No GraphQL operation, field, input or enum changes: the regenerated SDL is byte-identical to the base, 6a61ba36…f74d6f. No role, policy, entitlement, thoth-errors, Cargo, workflow or Sphinx change.

Local validation (commit 12a2a48e, clean tree, disposable Postgres 17 UTF8/C + Redis)

Command Result
cargo fmt --all -- --check exit 0
cargo test -p thoth-api --features backend 1941 passed, 0 failed, 1 ignored. Baseline was 1912; 29 new tests.
cargo test --workspace exit 0, 0 failed
cargo check --workspace exit 0
cargo clippy --all --all-targets --all-features -- -D warnings exit 0
cargo build (forced SDL regeneration) exit 0
git diff --check 15b6efd0...HEAD clean

Migration CLI (thoth migrate / --revert, disposable DB):

  • apply, empty revert and reapply all pass, and the schema dumps are identical;
  • running this migration's own down leaves a pg_dump schema identical to the base binary's;
  • a populated revert is refused with rollback refused … holds 1 row(s), and the schema and row are unchanged.

Mutation checks: six targeted regressions (coverage gate, A1 field, inconsistency, cursor flag, lock order, driver key) were all caught.

Production migration execution: NONE. Provider access: NONE.

Refs #930 (programme #766). Downstream thoth-pub/thoth-sphinx#18 is unchanged and remains on HOLD.

Add migration 20260921_v1.9.0 creating metric_identifier_quarantine: one
row per REJECTED / UNKNOWN_DOI provenance row, with locale-independent
nonblank checks, an ordered-period check, four non-cascading foreign keys,
no DOI-format check and no request identity. Its downgrade locks the
table and refuses while any row exists.

The canonical coordinator retains the source it already locks and, only
for driver_key exactly "cloudfront" and an observation carrying none of
the five excluded optional fields, writes one quarantine row in the same
transaction as the unchanged rejection, import error and invalid counter.

updateMetricSourceCheckpoint locks checkpoint, import, source account and
source in that order and, for CloudFront COMPLETED_WITH_ERRORS imports,
derives rejection and quarantine evidence from provenance: contradictions
fail closed as INTERNAL_STATE_INCONSISTENCY with no write; consistent but
ineligible imports keep the ordinary release; a quarantine-only import
with exact one-day COMPLETE coverage records its period-manifest cursor
entry without advancing last_successful_period_end.

Refs #930
Add the implementation report for the durable CloudFront unresolved-DOI
quarantine: bindings, preflight, write-budget compliance, decisions,
migration effects, validation, CI and remaining gates.

Refs #930
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-21T15:06:22.997202Z 983a988 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ja573
ja573 merged commit 186c75e into feature/metrics Sep 21, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant