Repository navigation
feat(e2e): run the Tauri e2e on macOS with a CI matrix, a PR report and local-model checkpoints - #573
Open
the-null-crafter wants to merge 4 commits into
Open
the-null-crafter wants to merge 4 commits into
the-null-crafter wants to merge 4 commits into
Conversation
…cOS CI matrix and PR report
…e, pick turns it on)
`speakViaEndpoint` requests `.../chat-messages/<id>/tts/`; the assertion still expected the path without the trailing slash, so the spy comparison failed on the URL argument. Fails on origin/main on its own, with no other change checked out. The hook is the correct side — the DM API wants the trailing slash — so the expectation moves, not the request.
the-null-crafter
added
run-tests
Contributor
PR E2E - unknown
No test report was produced (result: Re-trigger it yourself: comment |
Contributor
PR E2E - FAILEDFull report, traces and screenshots
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Tauri desktop e2e (
e2e-tauri/) now runs on macOS, and on GitHub's macOS runners as a matrix of real Ollama setups, with the same author-facing process the web PR e2e has.@wdio/tauri-service'sembeddedprovider launches the debug binary, which serves W3C WebDriver itself throughtauri-plugin-wdio-webdriver(registered in debug builds only, and only when the service setsWDIO_EMBEDDED_SERVER). Notauri-driver, no platform WebView driver;pnpm test:tauri:macosworks instead of exiting 1..github/workflows/macos-e2e-tauri.yml, rows ine2e-tauri/ci/macos/matrix.mjs): two macOS jobs per run, one per architecture (arm64 and Intel), each building the debug binary on its VM and running the machine setups one after another with a reset in between (run-rows.mjs,reset.sh) — the setups a user's Mac can have — nothing installed, Ollama.app running or stopped, the Homebrew formula as a service or CLI only, a bare CLI in/usr/local/bin, no Homebrew, Intel — plus a fresh Mac for the Code journeys. New Journey 4 drivescheck_ollama_status/install_ollama/ pull / cancel / stop against each setup. Same opt-in as the web e2e: therun-testslabel on a same-repo PR, orgh workflow run macos-e2e-tauri.yml -f scenarios=….tests.iblai.app(runs/pr-os-<pr>-<run>/macos/, 7-day expiry), a PR comment per run, its ownmacOS E2Ecommit status (pr-gate.ymluntouched), a known-failures gate (e2e-tauri/ci/macos/known-failures.json: red only on failures outside the list) and the commands/retest-macos [scenario…],/status-macos,/report-macos,/cancel-macos(pr-e2e-commands.yml, jobmacos; they work once this is onmain, sinceissue_commentruns from the default branch).reusable-oci-test-runner.ymldoes it (thegatejob checks outibl-web-opsand runsdetermine-test-user; passwordTEST_USER_PASSWORD). odm-08 is expected to fail on this first run: today a pick that still needs a download leaves the toggle off and never installs Ollama (the SDK'sLLMProviderModalonly does that for an already-installed model, and the Rust pull only starts an installed Ollama). That red is the evidence for the SDK fix; it is deliberately not in the known-failures list.cfd916f9carries main's one-line fix foruse-speech.test.ts, which the branch's base needed to pass the pre-push suite.Docs:
e2e-tauri/README.md(prerequisites, knobs, the matrix, the PR process, the signed-in checkpoints),e2e-tauri/COVERAGE.md(59 checkpoints, 32 covered, 4 journeys).Test plan
code-39is the known persisted-data-dir red), Journey 4 skipped by design off the matrix;cargo test --offline436 passed;pnpm lint:check,pnpm typecheck, prettier, rustfmt, the coverage ledger gate (--no-regress); the report builder's own test (node --test e2e-tauri/ci/macos/build-report.test.mjs) and a real report built from Journey 1's JSON results; both workflow YAMLs parse.run-testslabel to start it; expected first-run findings are the scenario scripts on real runners (Ollama.app first-launch prompts, the tgz layout, Intel from-source brew time),setup-06underbrew-service(launchd's KeepAlive restarts what the app stops), and odm-08 as described. Product gaps go intoknown-failures.jsonwith a reason; odm-08 stays out until the SDK fix lands.startLocalDownload(record the model, turn Local Models on, awaitinstallOllama()when the manager is not running, then pull); macOS permission rows (Accessibility / Screen Recording for Cowork, launched through launchd so TCC judges the app itself) — planned, not built.🤖 Generated with Claude Code