[Pedro Henrique Penna (ppenna)]

Closes #298. Part of #290 (requirement 2) and #281.

What

AWF's Cloud Hypervisor mount policies need a read-write /workspace share and a read-only /opt/hostedtoolcache share attached to one sandbox at the same time, each with its own guest mount point, device tag, and access mode enforced by the host. The microVM had a single virtio-fs slot. This adds a second slot in OpenVMM and uses it end to end.

OpenVMM pin

The openvmm gitlink moves from 4cb5777fa to a41cf7351, the head of nanvix/openvmm#113: two signed commits directly on nanvix/openvmm main, the feature and a review follow-up. With the new pin:

Slot	Stable ID	Tag	MMIO	IRQ	Shared status
First (unchanged)	fs:microvm0	microvm	0xd0001000	6	0x30004
Second (new)	fs:microvm1	microvm1	0xd0008000	13	0x30020

• Repeatable --mount. Attachments fill the slots in order. Each adds a virtfs_dir=, virtfs_tag=, virtfs_mode= triplet with its slot's tag. The second slot exists only with a filesystem attached, so a machine with one share or none keeps its device inventory, command line, and snapshot contract.
• Host-enforced, per-share policy. Each slot is a separate HostFs server with its own access mode and denied paths, so the host rejects writes to a read-only share whichever guest mount, tag, or link reaches it.
• Overlap rules. Guest targets that equal or contain one another, and host directories that do, are rejected before boot. Host directories are compared after canonicalization and by root object identity and, on Linux, by the filesystem sources that each reaches through the mount that contains it and every mount below it (from /proc/self/mountinfo), so a bind mount or nested mount can't expose part of one share inside the other. With two shares, each --mount-deny must be absolute and applies to the share whose root contains it. --mount-owner applies to both shares.
• Snapshots. The contract records both shares and requires each slot's discovery token exactly once and the bootstrap triplets exactly, in slot order. A restore must supply them again in capture order with the same paths, identities, targets, modes, denied paths, and owner. Earlier OpenVMM releases reject a two-share snapshot.
• ttrpc still binds only the first slot and rejects a two-share snapshot.

NVX

• Kernel. 0003-microvm-virtio-mmio-shared-status.patch maps the second slot's MMIO base to its status word. A kernel without the mapping refuses to probe the device, so the share fails closed.
• CLI. sandbox run and provision accept --mount twice. Each --mount-deny applies to the --mount before it, and a relative path resolves in that share; with one share, every --mount-deny still applies to it. NVX forwards absolute denied paths to OpenVMM. Overlapping guest targets, overlapping host directories (compared by resolved path and by the file identities of each directory and of the other's ancestors, which also catches one directory reached through two paths and a share inside a bind mount of the other share's directory), a denied path outside its share, and a third share are rejected before OpenVMM starts. OpenVMM's Linux mount-source check also rejects a bind mount that exposes part of one share inside the other. Both shares' bootstrap tokens count toward the 1024-byte command-line budget. run forwards a repeated --mount too.
• Managed sandboxes. Two shares are persisted under mounts in configuration format 4, which earlier NVX releases reject instead of starting without the second share. One share keeps formats 2 and 3.
• Guest. nvx-init-agent parses every triplet before it mounts any, fails closed on an incomplete or out-of-order triplet, a repeated tag, or overlapping targets, mounts the shares in slot order, and unmounts them in reverse order before the overlay on a one-shot exit, a managed stop, or fatal. nvx-hostmount mounts every share outside the sandbox.
• nvx-sandbox-smoke checks repeated TARGET MODE pairs and, given an rw and an ro share, requires a link in the rw share to fail to write into the ro share.
• test-microvm. The new filesystem-shares scenario joins the default set on every backend. It attaches a read-write workspace and a read-only tool cache, each with its own denied path, and requires:
  • guest writes to reach the workspace;
  • every mutation of the tool cache to fail, also through a link and, with the microvm1 tag remounted read-write inside the guest, with EROFS from OpenVMM;
  • three shares, nested guest targets, nested host directories, and a relative denied path to be rejected before boot;
  • both shares to round-trip through a snapshot whose restore requires them in capture order (a missing or swapped share is rejected).
• CI. New Linux (KVM, MSHV) and Windows (WHP) steps attach a read-write /workspace share and a read-only /opt/hostedtoolcache share to one Ubuntu sandbox, one-shot and managed. The guest's writes must reach the workspace, the tool cache and the denied files must stay unchanged, the managed configuration must use format 4, and stop must leave a clean scratch journal.
• Docs. run.md, usage.md, ci.md, and the design notes.

Acceptance criteria

Criterion	Coverage
A sandbox can attach an rw /workspace share and an ro /opt/hostedtoolcache share at the same time	The new concurrent-share CI step, one-shot and managed, on Linux/KVM, Linux/MSHV, and Windows/WHP; filesystem-shares; unit tests for parsing, forwarding, persistence, and the guest agent
Writes to the ro share fail with EROFS/EACCES; writes to the rw share reach the host	The smoke script's ro/rw checks and cross-share link check, with host-side verification of the workspace, the unchanged tool cache, and the denied files; filesystem-shares
Access modes are enforced by the VMM, not only by guest mount flags	filesystem-shares remounts the microvm1 tag read-write inside the guest and requires touch, mkdir, append, rm, mv, ln -s, and chmod to fail with EROFS; each slot is its own HostFs server
CI covers concurrent ro and rw shares on KVM, MSHV, and WHP	The new Linux and Windows CI steps and the default filesystem-shares scenario in nvx-microvm-tests-{kvm,mshv,whp}
Reject overlapping or nested guest targets, and tags that collide	OpenVMM, NVX, and nvx-init-agent reject equal or nested targets; tags are fixed per slot, and the agent rejects a repeated tag

Validation

The bare-metal and local runs used NVX 490ed705 with OpenVMM e37086001. The final heads, NVX c2200425 and OpenVMM a41cf7351, differ from those only in the gitlink: a41cf7351 rebases e37086001 onto main, and its tree, b47313c7, is identical, so the tested OpenVMM source is exactly the pinned one. On c2200425, nvx.py verify and the validate-nvx unit tests pass locally, and CI runs the full matrix. The bare-metal runs used a release OpenVMM built from the pin and guest artifacts built with Docker from this tree, whose provenance inputs are unchanged by the rebase.

• Local Windows 11: ruff check, ruff format --check, compileall, pyright for Linux and Windows, test_adversarial.py, test_hosts.py, and the validate-nvx unit test set (740 tests; all pass except the Windows curl-shim test, whose precondition needs rustc on PATH).
• Bare-metal Linux/KVM (Ubuntu 26.04, kernel 7.0) and Linux/MSHV (Azure Linux 3.0, kernel 6.6), each with a release OpenVMM built for its backend:
  • nvx.py verify;
  • the five Linux Ubuntu sandbox CI steps extracted verbatim from the workflow: layer smoke, live-share, managed live-share, caller-owned live-share, and the new concurrent shares;
  • the full default test-microvm suite (34 scenarios), including every filesystem-shares case, and the Ubuntu and Azure Linux guest smoke runs from the workflow.
• Bare-metal Windows/WHP (Windows Server 2025): nvx.py verify, the four Windows Ubuntu sandbox CI steps, including the new concurrent shares, the full default test-microvm suite (34 scenarios), and the Ubuntu and Azure Linux guest smoke runs.
• Bind-mount aliases, end to end on Linux/MSHV, in a private user and mount namespace: nvx.py run with an rw workspace and an ro share that is a bind mount of a workspace subdirectory, or with an ro directory bind-mounted inside the workspace, fails before boot with OpenVMM's reach the same files through a mount error, and the same layout without the bind mount boots; NVX's pre-check rejects an ro share inside a bind mount of the workspace and accepts the same layout without it.
• OpenVMM: clippy, doc, nextest (502 tests on Linux, 459 on Windows), and cargo xtask fmt; see microVM: attach a second virtio-fs share with its own slot nanvix/openvmm#113.

Notes

• Two slots. The issue needs two shares. The slot table is data-driven, so a third share (for example, the future diagnostics directories, or Support multiple filesystem mappings with separate access modes #281's third mapping) needs only another slot with a free interrupt and a kernel shared-status entry. ISA IRQs 14 and 15 remain free.
• Pin. microVM: attach a second virtio-fs share with its own slot nanvix/openvmm#113 landed as a fast-forward of nanvix/openvmm main to a41cf7351, so the pin is main's head.
• Runner-dependent performance gate. Three runs of this PR placed the MSHV platform job on the newly added runner azure-mshv-scus-1, and each time the gate flagged guest-exit teardown and multi-vCPU snapshot restore, by +26%, +41%, and +78% at 2, 4, and 8 vCPUs. That runner has the same CPU as the baseline runners but a different host kernel, 6.6.137.mshv2-2.azl3; azure-mshv-1, -6, and -7 run 6.6.148-200.azl3. Re-running the same build on azure-mshv-7 matched the baseline (teardown 24.63 vs 24.58 ms and 8.53 vs 8.16 ms; 8-vCPU restore 53.30 vs 52.80 ms), and same-host A/B runs on bare-metal MSHV showed no difference from dev. Until the baselines account for it, azure-mshv-scus-1 may fail the gate for any PR.

[Copilot]

Copilot review overview

🟡 Changes recommended

Bind-mount aliases bypass NVX pre-launch validation, and WHP CI omits the claimed scratch-journal check.

Review effort: Balanced
Findings: 2

Open (2)

• Windows share test omits ext4 scratch journal cleanliness check · New
• Bind-mounted root aliases bypass pre-launch path validation · New

What changed in this PR

Adds end-to-end support for two concurrent virtio-fs shares with independent read/write policies.

Changes:

• Adds a second OpenVMM virtio-fs slot and kernel mapping.
• Extends CLI, guest mounting, managed configuration, and snapshots for two shares.
• Adds cross-platform tests, CI coverage, and documentation.

File	Description
.github/​workflows/​run-nvx-microvm-tests.yml	Adds concurrent-share CI coverage.
doc/​ci.md	Documents CI validation.
doc/​design/​integrated-implementation-status.md	Records the ninth MMIO slot.
doc/​design/​machine-and-device-abi.md	Documents the second filesystem slot.
doc/​design/​validation.md	Describes concurrent-share validation.
doc/​run.md	Documents multi-share operation.
doc/​usage.md	Updates CLI reference.
guest/​common/​nvx-hostmount	Mounts repeated host shares.
guest/​common/​nvx-init-agent	Parses, mounts, and unmounts multiple shares.
guest/​common/​nvx-sandbox-smoke	Tests isolated read/write policies.
kernel/​patches/​0003-microvm-virtio-mmio-shared-status.patch	Maps the second slot’s status word.
openvmm	Advances the OpenVMM pin.
scripts/​nvx.py	Accepts and attributes repeated mount options.
scripts/​nvx_tools/​microvm_test_scripts/​filesystem-shares-snapshot.sh	Tests two-share snapshot resume.
scripts/​nvx_tools/​microvm_test_scripts/​filesystem-shares.sh	Exercises host-enforced access modes.
scripts/​nvx_tools/​microvm_tests.py	Adds the filesystem-shares scenario.
scripts/​nvx_tools/​sandbox.py	Models and validates multiple mounts.
scripts/​nvx_tools/​sandbox_lifecycle.py	Persists format-4 multi-mount configurations.
scripts/​test_microvm_tests.py	Tests the new scenario harness.
scripts/​test_nvx_tools.py	Expands CLI, agent, and lifecycle tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Copilot review overview

🟡 Changes recommended

The OpenVMM gitlink update is absent, and bind-mounted descendants can bypass read-only share isolation.

Review effort: Balanced
Findings: 1 · 2

Open (3)

• Reject bind-mounted descendants that bypass read-only share isolation · New
• Update OpenVMM gitlink to support the second mount slot · New
• Bind-mounted root aliases bypass pre-launch path validation

Resolved since last review (1)

• Windows share test omits ext4 scratch journal cleanliness check

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes a security-sensitive cross-platform device ABI, guest lifecycle, snapshots, and an OpenVMM pin that remains open and blocked.

Review effort: Balanced
Findings: 1 · 2

Open (3)

• Reject bind-mounted descendants that bypass read-only share isolation
• Update OpenVMM gitlink to support the second mount slot
• Bind-mounted root aliases bypass pre-launch path validation

[Copilot]

Copilot review overview

🔵 Needs a closer look

It changes a security-sensitive guest/VMM ABI across three backends while the pinned OpenVMM dependency remains unmerged.

Review effort: Balanced
Findings: None

Resolved since last review (3)

• Reject bind-mounted descendants that bypass read-only share isolation
• Update OpenVMM gitlink to support the second mount slot
• Bind-mounted root aliases bypass pre-launch path validation

[Copilot]

Copilot review overview

🔵 Needs a closer look

The cross-repository device ABI and snapshot changes still depend on in-progress exact-head backend validation.

Review effort: Balanced
Findings: None