feat: version registry that keeps release lists between runs - #119
Open
roxblnfk wants to merge 2 commits into
Open
feat: version registry that keeps release lists between runs#119roxblnfk wants to merge 2 commits into
roxblnfk wants to merge 2 commits into
Conversation
fix: request every releases page once with `per_page=100` fix: stop `destroy()` from loading the remaining release pages Every run asked GitHub or GitLab for the release list and spent the API rate limit on it. Releases are now kept in a local version registry: a provider-neutral database of the releases and assets of every known repository, one JSON file per repository, on by default in the per-user cache directory. Versions never expire; only the last check of a repository has a TTL (`cache-ttl`, default 600 s), within which `dload get` costs no API request. A stale check fetches only the newest pages until a stored release is reached. Pages stay lazy: the first run loads what the requested version needs, older releases are fetched on demand and appended. A failed check falls back to the stored releases. `dload get --refresh` ignores the TTL once, `dload cache:clear [software...]` drops records. The page loader used to build a paginator per page and probe the next one, so every page but the first was requested twice; `destroy()` iterated the whole lazy collection and loaded every remaining page after each download. Co-Authored-By: Dmitriy Derepko <xepozz@list.ru> Assisted-By: Claude Fable 5.1 <noreply@anthropic.com>
…them A 404 for a release asset was reported as a missing repository, with advice about tokens and addresses, and the deleted release stayed in the registry until a later check happened to overwrite it. Asset URLs now raise `AssetNotFoundException`; when every matching asset of a release is gone, the downloader drops the release from the registry, marks the repository for a check, and fetches the list once more before giving up. Assisted-By: Claude Fable 5.1 <noreply@anthropic.com>
roxblnfk
force-pushed
the
feat/version-registry
branch
from
September 12, 2026 20:17
e85e96b to
ea36d7e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔍 What was changed
DLoad now keeps a local version registry: a provider-neutral database of the releases and assets of every known repository, one readable JSON file per repository under the per-user cache directory (
cache-dir/DLOAD_CACHE_DIRto move it,cache-ttl=0to disable).cache-ttl, default 600 s); within itdload getmakes no API request at all.dload get --refreshignores the TTL once; newdload cache:clear [software...]drops the whole registry or the repositories a software package is served from.AssetNotFoundExceptioninstead of "repository not found, check your token", the release is dropped from the registry, and the list is fetched again in the same run before giving up.per_page=100, anddestroy()no longer iterates the lazy collection, which used to load every remaining page after each download.How it works
--refresh), the newest pages are fetched until one contains a stored release; the first page always overwrites what is stored, so assets attached later are picked up.Why?
Every run asked GitHub or GitLab for the release list, and every request counted against the API rate limit; unauthenticated CI matrices hit the 60 requests/hour limit quickly, and nothing was remembered between runs. Caching raw HTTP responses (#118) would have kept credentials-bound answers keyed by URL; storing release metadata keeps only tags, names and download links, so the directory can be shared between machines or a CI cache freely.
Checklist
dolthub/dolt(644 releases) the first run makes one API request, the second run none,dolt:~1.20.0loads the tail on demand, and a registry entry with poisoned asset links is dropped and the next release is downloadedcomposer test,composer psalm,composer cs:diffpass locallyDocumentation
README (and the ru/es/zh translations) gained a "Version Registry" section;
dload.xsddocumentscache-dirandcache-ttl.