design(modal): best use of Modal credits — $30/mo recurring + $100 one-time grant, for Atlas embedding and GPU batch #6069

Design (2026-11-04, Drew's directive): best use of our Modal balance — $30/mo Starter credits (recurring) + $100 one-time hackathon grant = $130 today. Full design doc: ~/workspace/scratch/modal-gpu-design-2026-11-04/design.md (sandbox scratch; move into repo docs when implemented).

Verified facts

• Modal Starter: $30/mo included compute, 10 GPU concurrency, per-second billing. L4 = $0.80/hr, A10 = $1.10/hr, H100 = $3.95/hr. Granted credits expire 12 months after grant → the $100 is spend-by ~Oct 2027.
• Access already exists and is proven: ~/git/fleet/modal-transcribe/ (Whisper on L4, 2026-10-31). No new account/credential.
• Production Atlas model is arctic-embed-m-v2.0 (768-dim, swap completed 2026-11-03) → Modal output is dimension-identical to the live index, no migration.

Headline math (assumptions labeled in the doc; validate with a ~$1 benchmark first)

• Throughput planning numbers for arctic-embed-m-v2.0 on one L4: 15k (pess) / 30k (expected) / 60k (opt) tok/s. Fleet local baseline is a measured 1,529 tok/s.
• Full Atlas internal re-embed (55.05M tokens, measured): ~31–61 min on L4 ≈ $0.60–$1.15 all-in — vs 10.0 h locally (~20× faster).
• $30/mo ≈ 2.9B tokens ≈ 5.4M chunks ≈ 8–9 GB of source text (L4, expected rate, after container overhead).
• Embed-everything-we-own (all repos, ~464 MB tracked measured, ~100M tokens after filtering): ~$1–2 one-shot.

Recommended allocation

• Recurring $30: weekly delta embed window (<$0.15/run) → monthly bulk/experiment window (chunking experiments and bake-offs become ~$1 passes instead of 10-h local jobs) → monthly feedback-normalization + golden-set eval batch → transcription on demand → ~$10/mo reserve as burst fund.
• One-time $100 (one-shot artifacts): embed-everything first pass ($2) → finish the Qwen3-0.6B bake-off that died locally at 37,888/101,471 chunks after 46 h ($2, closes a verdict hanging since Oct 31) → arctic-embed-l-v2.0 1024-d shadow index + parity eval ($3) → comments-corpus embed when the license gate clears (~$5–25) → ~$50 reserve.
• Ledger: track modal as a pool in quota-check (credits granted/used, tokens embedded, reuse ratio), alert on projected PAYG overage and on grant expiry <60 days with >$25 unspent. Our scheduler triggers modal run; Modal's own cron is backup only.

Hard guardrails

• Secret-tier text NEVER goes to Modal (external processor; classification violation regardless of price). Internal tier only. Secret tier stays on the local embedder (638 chunks — laptop rate is fine).
• No OCI anything (stood down 2026-10-31). No Mac in any option (Vertex machine). No always-on GPU. No new vendor accounts.

GPU-after-credits survey (Blacksmith / GitHub Actions GPU / Copilot / OCI / local) is filed separately and linked below — headline: Modal PAYG overage is the next step; Blacksmith has no GPU product; GitHub's native GPU runner is a $3.12/hr T4; we own no discrete GPU (laptop is a Radeon 780M iGPU; the "RTX 3060 Ti" in our records is a Marketplace listing we passed on).

Cross-links: #6018 (Atlas ingestion program), #6019 (ingestion ops + cost — this is its burst-compute answer), #5733 (Jaccard third signal — Modal feeds the vector leg the rescore combines).