Repository navigation
setup: unify the Linux development-host and runner setup scripts - #1025
Merged
Pedro Henrique Penna (ppenna) merged 3 commits intoOct 30, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The runner-mode version check can execute a project-controlled Cargo alias as root when setup runs from a runner-writable checkout.
No open issues 🎉
What changed in this PR
Consolidates Linux development-host and GitHub Actions runner provisioning into setup-linux.sh, addressing #128 while retaining separate operating modes.
Changes:
- Shares package, toolchain, device-access, and validation logic across both modes, with explicit KVM or MSHV selection.
- Preserves existing Docker installations and documents migration from the former scripts.
- Updates Azure packaging, quality checks, and tests for the unified entry point.
| File | Description |
|---|---|
scripts/test_tool_versions.py |
Tests shared provisioning, tool pins, and mode-specific options. |
scripts/test_setup_azure.py |
Verifies unified-script packaging and runner-mode invocation. |
scripts/test_nvx_tools.py |
Updates setup references and runner validation tests. |
scripts/test_nightly_ci.py |
Updates the runner setup diagnostic fixture. |
scripts/setup/tool-versions.conf |
Updates comments naming manifest consumers. |
scripts/setup/setup-linux.sh |
Unifies development-host and runner provisioning. |
scripts/setup/setup-linux-mshv.sh |
Removes the separate MSHV development setup script. |
scripts/setup/README.md |
Documents modes, migration, and Docker preservation. |
scripts/setup/azure/README.md |
Documents the unified Linux setup entry point. |
scripts/setup/azure/linux/bootstrap-runner.sh |
Invokes unified setup with --runner-only. |
scripts/setup/azure/build_package.py |
Packages the unified Linux script. |
doc/setup.md |
Updates bootstrap guidance and lint commands. |
doc/project-structure.md |
Lists the consolidated Linux setup script. |
.github/actions/validate-runner/action.yml |
Updates the toolchain remediation hint. |
.github/actions/check-quality/action.yml |
Lints and format-checks the unified script. |
Review effort: Balanced
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-128-fix-debug-test-setup
branch
from
October 29, 2026 22:08
58fe1f5 to
c39d55e
Compare
Contributor
Author
|
Addressed Copilot's "needs a closer look" note: runner-mode setup ran The fix is folded into 25c0e07. The version check runs Rechecked on hosts:
|
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The script reads the runner token before validating the selected mode, causing an invalid development-mode invocation to block on stdin or report a token-read error instead of rejecting the runner-only option.
No open issues 🎉
Review effort: Balanced
setup-linux-mshv.sh prepared a Linux/MSHV development host, and setup-linux-runner.sh a Linux/KVM or Linux/MSHV GitHub Actions runner. Each carried its own package lists, manifest reader, rustup, Rust, and cargo-nextest installation, device access rules, and host checks, and the copies had drifted: the development script's apt and dnf lists lacked diffutils, and only the runner's check mode checked its Rust targets. scripts/setup/setup-linux.sh replaces both, with the mode structure of setup-windows-whp.ps1: - Development-host mode, the default, prepares an existing checkout for the user who runs it, as setup-linux-mshv.sh did: Docker with Buildx, the user's pinned Rust release, musl target, and cargo-nextest, Docker and device group membership with the reconnect exit (status 20), the NVX build and dry run, and --check-only, --skip-build, --workspace, --guest-bundle, and --bundle-only. It may run as root or prompt for sudo, as before. - --runner-only selects runner mode, which keeps everything that setup-linux-runner.sh did: the nvx-runner account without sudo or Docker access, the root-owned trusted toolchain with the pinned sccache, registration with exactly one role and the additional labels, the service and its environment, migration of a legacy runner, writable state confined to _work, and check mode. It needs no checkout, still refuses to run as root, and never lets sudo prompt. - --backend kvm or mshv is required in both modes, so development-host mode now prepares Linux/KVM hosts too, and its dry run uses the selected hypervisor. - Each mode rejects the other's options, and the options are validated before the script reads a registration token from standard input, reads the manifest, or inspects the host, so a misplaced option never waits for a token. - One implementation installs the packages, to which each mode adds its own: Docker for a development host, or the runner's .NET dependencies on Azure Linux. One downloads and verifies release artifacts; one installs rustup, Rust, each mode's targets, and cargo-nextest, as the user or as root into the trusted toolchain; one writes the device's udev rule and grants the group; and one checks the platform, commands, Python, Rust, targets, cargo-nextest, and device access, as the user or as nvx-runner. Runner mode runs cargo as root only to install cargo-nextest, whose configuration discovery starts at CARGO_HOME. It checks the installed release by running cargo-nextest itself, as the administrator, so no configuration from the current directory, such as a cargo alias, runs as root. The output markers stay as they were: NVX_SETUP_COMPLETE, NVX_SETUP_CHECK, NVX_SETUP_RECONNECT_REQUIRED, NVX_GUEST_BUNDLE, and NVX_RUNNER_SETUP_COMPLETE, which runner check mode prints as before. Development-host check mode now also requires the musl target that the mode installs. The development script's second `tdnf install docker-cli` is gone: the first transaction already installs docker-cli, which provides moby-cli on Azure Linux 3. A missing --workspace now names the path that the script could not find instead of an empty one. The Azure VMSS bootstrap packages setup-linux.sh and runs it with --runner-only. check-quality lints the new script, and validate-runner names it in its hint. The setup README describes both modes with examples and maps the former invocations to the new ones: `setup-linux-mshv.sh ARGS` becomes `setup-linux.sh --backend mshv ARGS`, and `setup-linux-runner.sh ARGS` becomes `setup-linux.sh --runner-only ARGS`. Its runner examples now run the staged script with sh, which the non-executable file requires. The tests follow the single script: they run its manifest reader, read each mode's pins and Rust targets, select runner labels and roles, defer unnamed runner checks, and read the manifest before the host. New tests check that both modes install the same build dependencies plus their own packages, that each mode rejects the other's options before it reads the manifest, and that root runs cargo only to install. Fixes #128 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Development-host mode installed the distribution's Docker packages on every host, as setup-linux-mshv.sh did. On Ubuntu, docker.io conflicts with Docker's own packages, so apt removes docker-ce, docker-ce-cli, and containerd.io to install it: a simulated `apt-get install docker-buildx docker.io` on an Ubuntu 26.04 host that runs docker-ce plans exactly those removals. Azure Linux has no docker-ce, so the Linux/MSHV development hosts never met this, but development-host mode now prepares Linux/KVM hosts too, which commonly run Docker's own packages. Install Docker with Buildx only on a development host that has no docker command, and keep any Docker installation that the host has. The mode still enables the Docker service and grants the docker group, and its check still requires Docker and Buildx, so a kept installation without Buildx stops setup with "Docker Buildx is not installed". Runner mode installs no Docker, as before. The package test also runs each mode on a host that has Docker, and the usage text and the setup README describe the condition. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Runner mode runs five Python programs as root: three read the runner's .runner file, one sets its disableUpdate, and one writes svc.sh from the runner's template. They ran as `python3 -c` and `python3 -`, which put the current directory first on sys.path, so a json.py or pathlib.py there replaced the standard module. setup-linux-runner.sh did the same. When the administrator runs setup from a directory that another user can write, such as a checkout in a runner's _work directory, a job could plant such a module and run code as root: with a json.py that records its uid in the current directory, check mode of the former script ran it as uid 0 on a Linux/MSHV test host. Run them with -I, Python's isolated mode, which leaves the current directory, the user's site directory, and the PYTHON* variables out of the search. The same check mode no longer imports the planted module. A test requires -I for every Python that runs as root, as another requires root to run cargo only to install. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
force-pushed
the
agents/issue-128-fix-debug-test-setup
branch
from
October 29, 2026 22:22
c39d55e to
5d06cba
Compare
Contributor
Author
|
Addressed Copilot's second "needs a closer look" note, folded into 3f97257: The option now only records the request, and the script reads the token right after
While checking the runner code for similar root-in-current-directory exposures, I found one that predates this PR. The five |
Pedro Henrique Penna (ppenna)
assigned
Huzaifa Danish (huzaifa-d) and
Pedro Henrique Penna (ppenna) and unassigned
Huzaifa Danish (huzaifa-d)
Pedro Henrique Penna (ppenna)
deleted the
agents/issue-128-fix-debug-test-setup
branch
October 30, 2026 02:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces
scripts/setup/setup-linux-mshv.shandscripts/setup/setup-linux-runner.shwith one canonical Linux entry point,scripts/setup/setup-linux.sh, which follows the mode structure ofsetup-windows-whp.ps1:Development-host mode (the default) prepares an existing checkout for the current user. It preserves everything
setup-linux-mshv.shdid:--check-only,--skip-build,--workspace,--guest-bundle, and--bundle-only.Runner mode (
--runner-only) keeps everythingsetup-linux-runner.shdid:nvx-runneraccount, without sudo or Docker access;_work;It needs no checkout, refuses to run as root, and never lets sudo prompt.
--backend kvm|mshvis required in both modes, so development-host mode now also covers Linux/KVM hosts.Each mode rejects the other's options before the manifest is read or the host inspected.
One implementation each now covers:
nvx-runner.All output markers and exit statuses are kept:
NVX_SETUP_COMPLETE,NVX_SETUP_CHECK,NVX_SETUP_RECONNECT_REQUIRED(exit 20),NVX_GUEST_BUNDLE, andNVX_RUNNER_SETUP_COMPLETE.The Azure VMSS bootstrap now packages
setup-linux.shand runs it with--runner-only. check-quality lints the new script, andvalidate-runnernames it in its hint. The setup README documents both modes with examples and a migration table:setup-linux-mshv.sh [OPTIONS]setup-linux.sh --backend mshv [OPTIONS]setup-linux-runner.sh --backend BACKEND [OPTIONS]setup-linux.sh --runner-only --backend BACKEND [OPTIONS]Existing runners need no migration; runner mode manages the same account, toolchain, installation, and service.
Testing surfaced a latent bug that KVM support makes reachable. The development script always installed the distribution's Docker packages, and on Ubuntu
apt-get install docker.ioremovesdocker-ce,docker-ce-cli, andcontainerd.io. The second commit therefore installs Docker only on development hosts that have nodockercommand.Fixes #128
Commits
setup: merge the Linux setup scripts into setup-linux.sh: the unified script (a 64% rename of the runner script), tests, CI wiring, Azure packaging, and documentation. After review, it also guarantees two properties:cargo install; the nextest version check runs the binary directly, as the invoking user.--runner-token-stdinonly once the options are valid, so a misplaced option never waits on stdin.setup: keep the Docker that a Linux development host has: the Docker fix, with its test and documentation.setup: run the Linux runner setup's Python as root in isolated mode: hardening of code that predates this PR, found while addressing the review. The fivepython3programs that runner mode runs as root used to search the current directory first. With ajson.pyplanted there, the former runner script's check mode executed it as uid 0 on nvx-mshv-test. Withpython3 -I, it no longer does.Validation
Local:
test_tool_versions,test_setup_azure, andtest_nightly_ci, and the touchedtest_nvx_toolstests, pass on Windows. They also pass on Azure Linux 3 (bash) and Ubuntu 26.04, where the shell harnesses run under dash,bash --posix, and BusyBox.validate-nvxcommand set passes on Linux except twotest_nvx_rs_mutantstests. Those fail ondevitself since ci: fix nvx-rs mutants summarize miscounting shards #952; scripts: test the mutants merge against the shards' indexed outcomes #1023 and scripts: test the mutants merge as #952 changed it #1024 fix them.Remote hosts, with the user's approval (old and new scripts side by side where both apply):
nvx.py build --guest all:NVX_SETUP_COMPLETE=1. Guest bundles from old and new byte-identical, and validated by--check-only --guest-bundle./dev/kvmrule); runner check: old = new. Runner package installed up to the registration-token requirement: old = new. Development provisioning: exit 20 to reconnect, then complete. Check with nvx-rs build, provenance, and--hypervisor kvmdry run.apt-get,docker-ce)docker-ce. Fresh and repeated runner dependency passes; checks: old = new.--check-only, unnamed and named: identical results from old and new.Not exercised: registration against GitHub (
config.sh), which needs a live token and would add a runner to the pool. That code moved unchanged, and the Azure bootstrap's use of it is covered bytest_setup_azure.Separately: both production runners' local
.nvx-labelspredate thegeneralrole, so a named--check-onlyreports "GitHub Actions runner labels are not validated" with both the old and the new script. As the README describes, their next provisioning needs a registration token.