Speed up papers catalog first paint (gzip, ETag cache, client SWR)

[znsoftm]

Problem

Public /papers paints the shell (header, chips, filters) then sits on 「正在加载目录…」 until GET /papers/api/catalog finishes. On the nginx → search-proxy → tunnel → search-service path that body is ~1 MB uncompressed, caching was not CDN/browser-friendly for a static snapshot, and the client always waited on a full download before the first card.

What changed

Three complementary fixes; live progress / manage / visits stay on the tiny GET /papers/api/progress poll.

1. Lean static snapshot — catalog-snapshot.json now truncates abstracts to 400 EN / 220 ZH runes (cards still show 280 / 140). query_hits dropped. Same card ZH/EN brief, search still matches the prefix.
2. Gzip + cacheable HTTP — writer also emits catalog-snapshot.json.gz. GET /papers/api/catalog (and GET /papers/static/catalog-snapshot.json) serves:
   • Cache-Control: public, max-age=60, stale-while-revalidate=300
   • ETag + If-None-Match → 304
   • Content-Encoding: gzip when Accept-Encoding includes gzip
   • no request-time can_manage / visit overlay (those stay on /progress) so the bytes are actually static
3. Client SWR — last good catalog is stored in localStorage (rs_papers_catalog_v1) and painted immediately on repeat visits. Catalog + progress fetch in parallel. Empty-body retry only when the first 200 is empty and nothing is on screen. No cache: "no-store" on the catalog GET.

Relay http.Client now sets DisableCompression: true so gzip Content-Encoding survives the tunnel (Go’s default transport would otherwise inflate it).

Before / after (measured)

Unit fixture (TestLeanCatalogSizeReduction, 80 papers, unique-ish abstracts):

payload	bytes	vs full
full abstracts (old snapshot shape)	237,691	100%
lean snapshot	124,756	52%
lean + gzip (wire)	24,514	10%

Scaled to the public ~1 MB catalog of similar uniqueness, first download is on the order of ~100 KB gzipped instead of ~1 MB uncompressed. Repeat visits: 304 (0 body) or localStorage paint.

Local 40-paper fixture (this agent’s browser/curl check):

request	result
GET /papers/api/catalog	71,664 bytes, public, max-age=60, stale-while-revalidate=300, ETag
same + Accept-Encoding: gzip	1,780 bytes, Content-Encoding: gzip
If-None-Match	304, 0 bytes
reload in Chrome	catalog 304 Not Modified; cards + Chinese briefs painted; no stuck empty list
tag chip agent安全	20 / 40, filters still work

Public catalog stays far under the ~2 MiB tunnel safety margin. Streaming for /papers/api/catalog is unchanged.

Snapshot regenerate / invalidate

Operators do not run a separate generator.

• Auto-rewrite every PAPERS_CATALOG_SNAPSHOT_INTERVAL (default 60s)
• Immediate rewrite on import / invalidateCatalog
• Files: $PAPERS_DIR/catalog-snapshot.json and .gz
• Delete those files to force a live lean fallback; the writer recreates them
• Do not hand-edit the JSON

Documented in README.md and agent-papers/README.md.

Test plan

• go test ./...
• Snapshot write/serve, gzip Content-Encoding, If-None-Match 304, lean truncation + size ratio
• Frontend wires hydrateCatalogCache / fetchCatalogJSON / If-None-Match; no no-store on catalog fetch
• Relay preserves gzip through the tunnel
• Local /papers: cards + Chinese briefs on first paint; reload is 304 / instant
• Search and tag chips still filter the in-memory snapshot
• Progress remains no-store and still carries snapshot_etag / can_manage / visits

Stacked on cursor/papers-review-crosslinks-1557 (current papers tip). Do not merge onto main until that stack lands.