Repository navigation
docs(qa): macOS tray UX + settings-parity audit (ux-audit epic) - #1043
Merged
Merged
Conversation
Read-only sweep of the running v0.60.0 tray (accessibility-tree menu walk) plus a static comparison of the tray's native surfaces and settings catalogue against the Web UI. Parity matrix (feature surfaces, settings fields, per-server actions) and 16 ranked findings: dead menu-bar health badge, unlabelled status item, silent action failures, unreachable Agent Tokens view, and three settings fields that drifted out of the tray catalogue.
Deploying mcpproxy-docs with
|
| Latest commit: |
6740056
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://42dd0ace.mcpproxy-docs.pages.dev |
| Branch Preview URL: | https://audit-ux-macos-tray-sweep.mcpproxy-docs.pages.dev |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
📦 Build ArtifactsWorkflow Run: View Run Available Artifacts
How to DownloadOption 1: GitHub Web UI (easiest)
Option 2: GitHub CLI gh run download 32976599745 --repo smart-mcp-proxy/mcpproxy-go
|
The document read as an open backlog, but #1055 ("fix(tray): close the 16 findings of the macOS tray UX + settings-parity audit", merged 2026-08-25) fixed all 16, with #1056 as a catalogue follow-up. The header's "no version skew, so every finding below is reproducible from source" is no longer true against main. - Header scopes that claim to the time of the walk and carries a status banner. - New §6 Resolution records the two fixes whose obvious implementation is wrong: F1's severity dot is an attributed-title glyph, not a composite image (the status image stays isTemplate and a template image renders monochrome), and F6 added a parity gate rather than just the three missing keys. - Verified on main rather than taken on trust: Menu/TrayIcon.swift is deleted, TokensView is reachable from Views/MainWindow.swift, MCPProxyApp.swift sets isTemplate + accessibilityDescription + toolTip + attributedTitle glyph, and SettingsCatalog.swift carries deep_scan / instructions / code_execution_max_parallel. Note for anyone tracing this later: #1053's body claims to fix these 16 tray findings, but #1053 touches no Swift file — it is the Web UI cluster its own title names. #1055 is the tray PR. The caveat is stated rather than hidden: this pass could not re-walk the live tray, because macOS Accessibility was revoked on the reviewing machine (check_accessibility -> trusted:false), so the resolution is read from the #1055 diff, not from a second walk.
Dumbris
force-pushed
the
audit/ux-macos-tray-sweep
branch
from
August 26, 2026 13:17
19b94a2 to
8cce2f2
Compare
Verified against v0.60.0 rather than taken on trust: - The tray Settings tab is named "Raw", not "Raw JSON" (native/macos/MCPProxy/MCPProxy/Views/SettingsView.swift:38 at v0.60.0). The §1.2 inventory, the §2 parity matrix and F6's body all said "Raw JSON". - The Web UI had no /usage route at v0.60.0 - the router's path list runs /, /servers, /repositories, /search, /settings, /secrets, /sessions, /tools, /activity, /security, /tokens with no usage or overview entry. Usage was a Dashboard tab; /usage became a route later, in #1044. The parity matrix's "Usage report | ⚠ savings % on Dashboard | ✅ /usage | inconsistent" overstated the Web UI side of that comparison. Also noted in §6 that three §1.1 menu labels have changed since the walk as part of the F9 fix: Open MCPProxy Window, Open Web UI in Browser, Launch at Login.
Replaces the "could not re-walk" caveat with the walk itself. macOS Accessibility was granted, so mcpproxy-ui-test works now; the walk reproduces this audit almost verbatim, because the installed tray is still v0.60.0 with "Update 0.61.0 — ready to restart?" sitting in the menu. Reproduced live: F2 (empty status-bar title, no description, no tooltip), F5 (no Agent Tokens entry), F7 (ElevenLabs "Stop" vs cloudflare-graphql "Disable"; and memory showing status Disabled above action Start), F8 (all three quarantined servers offer only Disable/Stop, Restart, View Logs), F11 (research and deploy still offered at 0 tools), F12 (streamable-http beside http and sse), F15 (29 flat entries), F16 (no tool search), and the pre-#1055 menu labels. One drift worth noting: the Clients block now reads "No clients in the last 24h" rather than listing an idle client - the empty state H5 exists to avoid. The consequence for the resolution section: #1055's fixes are real in the diff but remain unverified against a running tray. That needs the pending 0.61.0 restart or the dev bundle-swap, neither of which this pass did - restarting the tray kills the core it manages.
Dumbris
added a commit
that referenced
this pull request
Aug 29, 2026
…inish the ux-audit sweep (#1085) * chore(roadmap): truth-sync ux-audit and upgrade-nudge Two epics claimed a state the repo contradicts. upgrade-nudge was `done` with all four tasks done, but Spec 079 FR-002 — the human-readable "N releases / M weeks behind" delta required on status, doctor, the startup log, the Web UI and both trays — was never built. The four shipped tasks are US-slices and the FR belonged to none of them; spec 079 has only spec.md, so no checkbox surface could catch the gap. It survived only as TODO(spec-079/FR-002) comments in internal/updatecheck/checker.go and cmd/mcpproxy/status_cmd.go, which is how an outside contributor found it (#1081). Epic back to in_progress with an explicit upgrade-nudge-delta task. ux-audit was `todo` with both sweeps todo, but both shipped in August (#1046 Web UI, 36 findings; #1043 macOS tray, 16 findings) along with ten PRs closing every finding and the five defects the re-check pass raised (#1061/#1064/#1065, all closed). Recorded the sweeps and fix waves as done and added the two items that really are open: folding the audit's F6 modal Escape/focus and F1 cross-view total assertions into e2e/web-ui-sweep, and verifying the tray fixes on a running tray (the 2026-08-26 re-walk found the installed tray still on v0.60.0, so #1055 remains unobserved). Also cleared the two --check-github errors, neither introduced here: - tpa-db and mcp-2026-upgrade each linked a docs(specs) PR in `pr:`. That field is implementation evidence, so a spec merge read as "epic done". Both entries' own notes already said implementation had not started; moved the refs into the notes and left the statuses honest at todo. - analytics-default-landing (#1044) and schema-deferred (#1063, tasks.md 89/89) were genuinely finished and are now done. * feat(079): "N releases / M weeks behind" upgrade delta (FR-002) Closes the last outstanding slice of Spec 079. FR-002 requires the current version, the latest version AND a human-readable delta on `mcpproxy status`, `doctor`, the startup log, the Web UI and both trays; what shipped was latest-version-only, and the deferral survived as TODO(spec-079/FR-002) comments that an outside contributor found (#1081). The blocker was data, not rendering: the checker fetched only /releases/latest, so it had the offered release's publish date but not the running one's, and no tag sequence to count. internal/updatecheck/delta.go computes the delta as a pure, network-free, clock-free function, so every spec rule is a table row. Fetching is a best-effort enrichment hung off the existing check — never a second check pipeline (FR-001): - one page of /releases (ETag-conditional, so a repeat costs no rate-limit budget), plus a by-tag lookup only when the running build predates that page and its date is therefore unknown; - both requests share one context deadline. A per-request timeout bounds each hop and nothing bounds the total, and CheckNow runs synchronously inside GET /api/v1/info?refresh=true, so an unbounded delta would stall a request a user is waiting on; - counting is by semver, never by list position: GitHub orders the list by created_at, and the live data has v0.53.0-rc.9 sitting ahead of the later-published rc.10. Three degradation rules matter more than the feature. A delta miss must not set CheckError (statusVersionSuffix returns "" whenever it is non-empty, so that would DELETE the nudge instead of enriching it), must not touch the FR-018 backoff counters, and must leave every surface rendering exactly what it rendered before. There is a regression test per rule. The clause is rendered once in Go (FormatBehindSummary) and printed verbatim by every surface, with the raw numbers alongside for machine consumers. That is deliberate: GuidanceLine is already hand-ported into UpdateBanner.vue with a "keep the two in sync" comment, and FR-002 asks for consistent framing across five surfaces. Fields are additive per FR-021 and had to be threaded through six hand-maintained mirrors — VersionInfo, the InfoResponseUpdate DTO and its copier, internal/contracts (→ make swagger), the contracts.ts generator, status_cmd's map decoding, and the Swift tray's Codable, which turned out never to have decoded the US2 fields either. Two adjacent bugs fixed because the delta sits on the same code: - GetLatestReleaseIncludingPrereleases returned releases[0] from a created_at-ordered list, so an RC user could be offered a release older than one already published. Now max-by-semver. - GetReleaseByTag interpolated an unvalidated tag into the request path and deferred validation to callers by comment only. The delta feeds it the ldflags-stamped version, so it now validates and escapes. Both trays render the delta in the update item's tooltip rather than its title: Spec 092 FR-010 fixes the title's exact shape, and a menu row reading "Update v0.62.0 — ready to restart? — 8 releases / ~14 weeks behind" stops being scannable. Known bound, accepted deliberately: the scan reads one 100-release page, so a build older than that window reports a lower bound ("59+ releases") and a semver-in-range release created long ago can be missed without setting that flag. Spec 079's own assumption (L159) allows an approximation where an exact count is unavailable, and the figure is motivational, not load-bearing. Verified: unit + integration tests against an httptest GitHub (delta, saturation, tag fallback, 304 reuse, budget, malformed tags), `go test -race` across internal and cmd, golangci-lint v2 clean, 962 frontend unit tests, and a live version-stamped instance confirming the degradation path renders the pre-delta output when the check fails. * test(ux-audit): finish the sweep's regression net, and fix what it caught The August Web UI audit closes by asking for four assertions to be folded into the committed sweep. Three were already there (F9 contrast, F30 accessible names + aria-live, F14 responsive); this adds the two that were not, and then fixes the defects they immediately exposed on main. F6 — the Add Server modal takes focus, traps Tab, closes on Escape and restores focus to its trigger. These `<dialog :open>` modals are opened by the `open` attribute rather than showModal(), so none of that comes from the browser; it all comes from useModalA11y, whose own comment delegates the real-browser half of its coverage to "the Playwright sweep" — a promise the sweep did not keep. Escape is dispatched IN-PAGE, never via page.keyboard.press: re-checking F6 during the audit produced a false negative for exactly that reason, because a synthetic key never reaches the document listener under test. F1 — Activity and Usage must report the same 24h numbers as the API. The invariant is not naive equality: Usage folds `blocked` into its error tile while Activity splits them, and /activity/usage is served from a short read cache while /activity/summary counts live, so the assertion polls for convergence and encodes the real relationship. Running the completed sweep turned up three defects already on main, none of them regressions of the audit's 36: 1. The LANDING PAGE failed AA in both themes (3.09-4.23:1) and two of its controls had no accessible name. Usage.vue uses the raw `opacity-*` utility for muted text; #1054 fixed that failure mode for the text-base-content ramp and .stat-* but never covered raw opacity, and nothing caught it because the audit walked `/` before #1044 made Usage the landing page. Moved onto the AA-checked --tone-muted; both selects got an aria-label. 2. The Activity status pill clips at 390px for any status outside the known four. An unmapped status falls through to a ghost pill carrying the raw value, and `tool_auto_approved` measured 145px inside a 93px cell (the table is table-fixed below 640px). The pill now truncates against the cell and keeps the full value in its title, rather than the column being sized to today's longest label — which would hold only until the next status is added. The phone check that finally caught #2 had been passing vacuously: it measures rows.first(), and a `success` row — ~90% of traffic — renders an sr-only label of zero width. It only became a real assertion once the new F1 check generated an error row and put one first. That coupling is worth knowing about: the sweep shares one instance, so one check's traffic is the next check's data, and a newly-failing neighbour after adding a check is more likely a real find than a flake. Both were separated from the change under test by running the same sweep against clean HEAD. Sweep: 25 passed / 3 failed on clean HEAD, 30 passed / 0 failed now. Roadmap: ux-audit was still carrying both sweeps as `todo` although they and every finding shipped in August; recorded here along with what the completed net found.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Read-only UX sweep of the running production macOS tray (v0.60.0) plus a static comparison against the Web UI surface.
SettingsCatalog.swiftvsfrontend/src/views/settings/fields.ts), per-server actionsHeadlines: the Spec 044 menu-bar health badge never renders (
TrayIconis dead code), the status item has no VoiceOver label, tray server actions swallow their errors,TokensViewis unreachable, and three Web UI settings are missing from the tray catalogue with no parity gate.Docs-only:
docs/qa/ux-audit-macos-tray-2026-08.md.