Repository navigation
The operator's view: one preflight for every provider, the ledger in a table, and the freshness a browser shelf can claim (#144 Phase 4) - #153
Conversation
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
|
Warning Review limit reached
This review includes 17 billable files and costs up to $4.25.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Or wait 15 minutes for your next included review. View limit detailsLimit details: You’ve used the included review currently available. Your 72 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Essentials Run ID: ⛔ Files ignored due to path filters (4)
📒 Files selected for processing (17)
Comment |
…eader `mix dd.discovery.check` had checked CineGraph, by name, since the second provider was registered. Eleven providers could have had a missing credential, an endpoint pasted whole out of a dashboard with its userinfo still in it, or a `source_policies` override `Policy` cannot read, and the preflight would still have printed *CineGraph configuration ready*. And the other half of the operator's question — what has each provider spent, what is it holding, what is stale, what does retention take next — had no reader at all: it was four tables and a `SELECT`. One module answers both. `Discovery.Status.preflight/0` reads the registry, each provider's own `enabled?/0`, its stanza and its policy; `Discovery.Status.rows/0` reads `discovery_runs`, `discovery_results`, `discovery_request_attempts` and the source row's provider-wide backoff. `mix dd.discovery.check`, `mix dd.discovery.status` and a new `/ops/discovery` render what those two return and hold no numbers of their own — the arrangement `Health.records/1` has had with `mix dd.health` and `/ops/imports` since #69 §6, for the reason that page states: two implementations of one number are two numbers. **Nothing names a provider.** Both functions map `Providers.all()`, so the thirteenth gets a row in both tasks and both tables the day it is registered, and three tests prove it by registering a fake and finding its row. Two conventions the twelve already follow carry the rest: a provider's stanza is its slug with hyphens as underscores, and its credentials are the `allowed_provider_env` names starting with its shell-safe prefix — the same `env_prefix` `runtime.exs` already derives the per-provider policy variables with. A name no registered provider claims is printed as *unclaimed* rather than dropped, which is what Artsy's two have been since it left the registry. **No credential value is read to report one.** `runtime.exs` is the only scope holding both the exported environment and the development `.env`, so that is where presence is decided; what leaves it is one boolean per admitted name, and `:provider_credentials` is a map of them. A task asking `System.get_env/1` would have called every `.env` key missing in dev. The check task's suite configures a distinctive value for every keyed provider and greps the task's whole output for each one; the status task's and the page's do the same. A misconfiguration fails the preflight. **A missing credential does not**: a keyless checkout is a deployment this system is designed to have, and a gate against it would be a gate against the Met. A registry `validate!/0` refuses is checked first and on its own — asking a module that does not export `enabled?/0` whether it is enabled raised `UndefinedFunctionError` out of the preflight, which is a worse report of the same fault than the sentence the boot check already writes. Measured, on this machine, 2026-09-22: * keyed: `mix dd.discovery.check` prints 12 rows, `12/12 ready`; keyless (the `.env` symlink moved aside), the same 12 rows, `6/12 ready · 0 switched off · 6 waiting for a credential`, exit 0. * `mix dd.discovery.status` prints 12 rows on both, and on the development database reads 164 runs, 1,202 results held, 39 of 154 display roots stale and 0 past retention — the acceptance line's *nine rows, not one*, at the count the registry has now. * `/ops/discovery` at 1280 and 375 px, light and dark, in `docs/discovery/issue-144-phase4-*`. At 375 the page's `scrollWidth` is 375: the two tables scroll inside their own boxes and the page does not. * `mix precommit` — 1,722 tests, 0 failures, from 1,692 on `main`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…aid nothing Checking the issue's fourth acceptance line rather than assuming it: *every shelf on `/define/war` says when it was fetched or since when it is held*. Six do, since Phase 2 — film, artworks, images, text, news and music read *Fetched 4 days ago · refreshes on your next visit after…*, and the artwork shelf's corpus half adds *catalog held since 17 Sep 2026*. The GIFs shelf said nothing at all. It said nothing because `freshness/1` reads a display root and a browser shelf has none: the request is the reader's browser's and nothing it returns is stored, which is K10 and is the point of that transport. But *now, and not kept* is an answer to when this was fetched, and a shelf that stays silent reads as one whose age nobody knows rather than one that has no age to have. One line, in the shared browser shelf, where it names no provider: *Fetched by your browser on this visit; nothing is stored here.* — the same sentence the on-demand definition card already uses for the same reason, in the same place the server shelves put theirs. Its id follows the browser shelf's own rule and carries the provider as well as the type, because two browser providers on one content type are two shelves. Measured: `/define/war` on the development database now answers the acceptance line for all seven shelves; `culture_chrome_test.exs` asserts the line and its id, beside the Phase 2 cases for a fetched shelf, a corpus shelf and a shelf that has fetched nothing and rightly says nothing. 27 tests in that file, 0 failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
55137b1 to
ad3e826
Compare
|
Re-pushed after GitGuardian flagged two findings on the first push. Both were the test fixtures' own sentinel values — Gate re-run after the rename: |
Closes #144.
Phase 4, the last one: the operator's view.
mix dd.discovery.checkwalks theregistry instead of naming CineGraph,
mix dd.discovery.statusis new,/ops/discoveryrenders both, and the discovery README gains the section that says what this issue made
true. With it, the six acceptance boxes are all true or accounted for, so the issue closes.
What changed
lib/devils_dictionary/discovery/status.expreflight/0(registry,enabled?/0, stanza, endpoints, policy, credential presence) androws/0(runs, failures, last success, results held, display roots and stale roots, retention due, budget used in this source's window, provider-wide backoff). The one implementation the two tasks and the page renderlib/mix/tasks/dd.discovery.check.exProviders.validate!/0is called rather than re-implemented, and first — a module that does not exportenabled?/0used to raiseUndefinedFunctionErrorfrom inside the preflightlib/mix/tasks/dd.discovery.status.exlib/devils_dictionary_web/live/discovery_live.ex/ops/discovery: the ledger table, the preflight table, four stats and the two commands. A sibling of/ops/imports, not a section of it — that page is the corpus, this one the culture shelves, and they share an eyebrow and nothing elseconfig/runtime.exsprovider_credential_env, and:provider_credentialspublishes one boolean per admitted name. That scope is the only one holding both the exported environment and the development.env, so it is the only one that can answer present honestly — and the only thing that leaves it is the booleanrouter.ex,layouts.ex,ops.exculture.exdocs/discovery/README.md,adding-a-provider.md,README.mdProvider-agnostic, and asserted: three tests register a fake provider and find its row —
in
Status, in each task, and on the page. Two conventions the twelve already follow do thework: a stanza is the slug with hyphens as underscores, and a credential is an
allowed_provider_envname starting with the slug's shell-safe prefix — the sameenv_prefixruntime.exsalready derives policy variables with. Names no provider claims are printed asunclaimed rather than dropped, which is what
ARTSY_CLIENT_IDandARTSY_CLIENT_SECRETare.Never a key:
dd_discovery_check_test.exsconfigures a distinctive value for every keyedprovider and greps the task's whole output for each one; the status task's suite and the page's
do the same. Endpoints are printed — they are in
config.exsin the open — after the checkthat they carry no userinfo, query or fragment.
The six acceptance boxes
allowed_provider_env, nothing elsea04530athat brought both ontoProvider.Helpersgrep -c 'defp word_pattern' …providers/*.exreturns 0retention_seconds: 86_400holds nothing older than a day, includingsource_recordsretention_test.exs(25 h under a short policy takes the result and its source record; the same age under the default keeps both), plus the Guardian's 23 h hold. 65 tests, 0 failures/define/warsays when it was fetched or since when it is held/define/war:culture-freshness-{film,artwork,image,text,news,music}andculture-freshness-gif-giphymix dd.discovery.statusruns on a keyless checkout and prints a row per provider.envmoved aside, exit 0conformance_coverage_test.exs, every registered provider can either run or render — there is no third statemix dd.discovery.checkKeyed:
Keyless — the same twelve rows, the
.envsymlink moved aside, exit 0:mix dd.discovery.statusKeyed, against the development database:
Keyless — twelve rows, not one:
/ops/discovery1280 px, light and dark:
375 px, light and dark. Measured rather than eyeballed:
document.documentElement.scrollWidthis 375 and
window.innerWidthis 375 — the two tables scroll inside their own boxes and thepage does not scroll sideways.
Gate
MIX_TEST_PARTITION=p4ops mix precommit— 1,722 tests, 0 failures (1,692 onmain).Shared files touched:
config/runtime.exs,router.ex,layouts.ex,components/ops.ex,components/culture.ex(one line), and the two discovery documents. No provider module changed.🤖 Generated with Claude Code