Conversation
Operators can POST /papers/pdf/recover to re-download an arXiv PDF into pdfs/, refresh manifest/DB/catalog fields, and use a Recover PDF button on /papers when the file is missing. Co-authored-by: Daniel <znsoft@163.com>
SEARCH_TOKEN already authenticates GET /papers via the query string, but the catalog and recover XHRs omitted it and 401'd. Co-authored-by: Daniel <znsoft@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When an Agent Papers catalog entry is an arXiv paper but has no file under
pdfs/(UI: “无本地 PDF” / “No local PDF”), operators can recover it without a full sync.Endpoint
POST /papers/pdf/recover{ "id": "2609.03747" }(also acceptsarxiv_id){ "all_missing": true }— walks catalog entries with no local PDF and anarxiv_idorpdf_url; concurrency capped at 2{ "ok": true, "paper": { has_local, local_pdf, ... }, "already_present": false }{ "ok": false, "error": "...", "code": "fetch"|"papers"|"bad_request" }with a clear message (arXiv HTTP 404, HTML-only, network, not in catalog)already_presentwithout re-downloadingAuth: same as
/papersand/settings— Hub global admin cookie, admin Bearer, or operatorSEARCH_TOKEN(including?token=on the page and its XHRs). Unauthenticated POST is JSON 401.URL strategy: try the stored
pdf_url, thenhttps://arxiv.org/pdf/{id}.pdf(and the un-suffixed form). Follows redirects. Rejects non-PDF bodies (HTML 404 / withdrawn / HTML-only). Does not markhas_localunless a real%PDFfile is written.Persistence: writes
pdfs/, updatesmanifest.json(pdf_path,download_status, optionalpage_count), invalidates the in-memory catalog snapshot, and best-effort patchespapers.dbviapython3sqlite if that file exists.This tree has no
hf_daily.go/ import-translate pipeline. Download checks match the existing Pythondownload_pdfhelper (magic bytes, arXiv?download=1retry).auto_translateis not present here, so recover does not enqueue translation.UI
On
/papers, whencan_manage(authenticated catalog) and!has_localandarxiv_idorpdf_urlis set, a 恢复 PDF / Recover PDF button appears next to the muted “无本地 PDF” label. Click calls the API, shows progress/error on that card, and swaps in View/Download without a full page reload. A toolbar 恢复缺失 PDF action runs the batch path.Tests
internal/api/papers_recover_test.go: fake PDF server success, arXiv 404, HTML-only, already-present file, corrupt re-download, batch counts, auth 401, optional SQLite update.