test(webui): gate bundled UX, security, and real-model integration

[inureyes]

Parent / dependencies

Parent: #1834. Depends on #1838, #1837, #1839, #1840, #1841, #1842, #1843, #1844, #1845, #1846, #1847. Final integration owner; individual children remain responsible for their own unit/contract tests.

Ownership

Own tests/webui_*, frontend Playwright/accessibility/visual integration suite, shared verification script and CI integration, final cross-surface regression fixes coordinated with module owners. Browser mock fixtures do not replace a real bundled Rust server.

Test matrix and release gate

• Build and launch BOTH server commands in UI-on/UI-off and single-model/empty-router modes, feature-on/off, nonempty API prefix, loopback generated key, explicit key/TLS and supported reverse proxy configuration. Verify root health, public shell/private data, asset/cache/CSP behavior and offline operation outside source tree. Keep required MLX runtime resources available; single bundled WebUI does not imply static linking of MLX.
• Hermetic browser flow against the real router and fake model/downloader traits: login → empty library → rescan/download → inspect → load → chat → cancel → unload/drain → remove. Exercise bad state transitions, stale multi-tab changes, unknown POST result, event gaps, auth expiration, server restart, download error/cancel, and corrupted/missing assets. Security attacks target legacy routes as well as UI adapters.
• Hardware gate with actual checkpoints: one dense plus one hybrid/MoE family from the project's available regression set, one supported VLM image input, and an opt-in small public download. Record binary SHA/features, checkpoint path and revision, hardware/OS, command, generated output, operation order and before/after resource observations. Set MLXCEL_REQUIRE_MODELS=1 or equivalent required-fixture assertion; unavailable hardware/checkpoint is a reported blocker, not a green skip.
• Run Rust workspace/test-fast gate with Metal+accelerate and relevant workspace/all-target clippy/fmt; feature-disabled build and CUDA server smoke on its supported host. Do not parallelize GPU test binaries. No quantization/kernel arithmetic changes belong here; if unavoidable follow the teacher-forced logit trace conditions.
• Playwright Chromium/Firefox/WebKit plus actual Safari on macOS 27 (WebKit CI alone is not Safari validation); 390/1024/1440 widths, light/dark/tinted/opaque/high-contrast, 200% zoom, reduced motion, keyboard/IME and manual VoiceOver. Capture approved deterministic screenshots; failed baselines need review rather than blanket regeneration.
• Initial JS ≤200 KiB gzip, total JS ≤700 KiB gzip, bundle ≤5 MiB; local cold shell usable ≤2 s, action feedback ≤100 ms excluding backend work, no layout shifts from status updates. Record reference host/browser/dataset and startup RSS/time separately from model load. Test 1,000 catalog entries and 10,000-token transcript; observation overhead follows the Activity child's paired-run budget.

Acceptance criteria

• One make verify-webui (or documented equivalent) orchestrates lint/type/unit/contract/bundle/browser checks; actual hardware checks are clearly separate required release evidence, never masked by mock success.
• Compatibility endpoints/CLI manifests and UI-disabled regression pass. CI tests the installed artifact, not only the Vite dev server.
• Shared requirement-to-test matrix accounts for every epic acceptance criterion, including security, state completion and no-model/no-network guarantees. No enabled placeholder or misleading metric survives final review.
• Publish test evidence and remaining limitations; do not close the epic while a required gate remains unverified.