openvmm: serve virtio-fs requests after a gated restore by ppenna · Pull Request #918 · microsoft/nvx

Pedro Henrique Penna (ppenna) · 2026-10-27T01:11:13Z

Fixes #904. Pins microsoft/openvmm#4762 at e68871cc6f14fceb6d38a899504ae8105f89e310, whose microVM virtio-fs queue workers wait for request admission to reopen instead of ending, so a share that a clone uses after a tiered restore, such as an export bound to the dormant filesystem slot, answers its requests instead of hanging them. It adds a microVM scenario that covers that case on KVM, MSHV, and WHP.

Based on dev at 7d4df3571. microsoft/openvmm#4762 is two commits directly on a12d9167f, the head of feature/microvm/dev and the revision that dev pins, so this branch's three commits are its whole diff.

Change

Commit	Files	Change
`88cc8943e` openvmm	`openvmm`	`a12d9167f` → `e68871cc6`: the two commits of microsoft/openvmm#4762
`b4c0204aa` doc	`doc/openvmm-upstream-roadmap.md`	Records them as positions 344 and 345, PR18 fixups to fold into position 111, which added the request admission; moves the NVX pin; and records that microsoft/openvmm#4749 merged positions 341-343 into `feature/microvm/dev`
`864a5f020` scripts	`scripts/nvx_tools/microvm_tests.py`, `scripts/nvx_tools/microvm_test_scripts/platform-filesystem.sh`, `scripts/test_microvm_tests.py`, `scripts/nvx_tools/microvm_shards.json`, `doc/ci.md`, `doc/usage.md`	New default-suite scenario `platform-filesystem`; shards replanned

OpenVMM changes

Root cause. A tiered restore arms the post-restore input gate, so OpenVMM quiesces device input before it starts the VM's state units, which closes request admission on the microVM virtio-fs device. Each queue worker ended as soon as it found admission closed, task_control recorded it as done, and nothing started it again when the guest acknowledged the restore and release_snapshot_boundary reopened admission. Every FUSE request of the clone, from its FUSE_INIT on, then waited forever; the mount still succeeded because it does not wait for FUSE_INIT. The rollback of a failed save, which starts the queues again behind the gate, ended the same way.
Test harness (a04cc8a00). Factors the virtio-fs integration test harness without changing what its tests do, so that a test can drive the microVM device, restart a queue from saved state, and await a reply later.
Fix (e68871cc6). A worker now waits for admission to reopen, through an event_listener::Event that resume() notifies, in a wait that a stop cancels at once and that leaves nothing registered when it is dropped. Admission still precedes queue.next(), so a closed gate leaves descriptors on the virtqueue and a save still finds no request in flight. The snapshot guide and the virtio-fs reference now say that a tiered restore gates filesystem input too, so post-restore repair must not wait for a virtio-fs reply. Unit and integration tests cover it; without the fix, both integration tests time out.

Tests

platform-filesystem captures a platform snapshot without --mount, which leaves the filesystem slot dormant, and restores it with a --mount of a host directory, as a clone's export binds the slot. After the platform repair acknowledges the restore and the gate releases, the clone mounts the share, reads the file that the host wrote, and writes one that the host then reads; the restore may not change the snapshot. A failed guest check prints NVX-PLATFORM-FILESYSTEM-FAIL, which fails the restore at once. It runs with the Alpine sandbox-control guest only, like platform-zero-fill, and joins shard 2 on KVM and shard 1 on MSHV and WHP with a 6 s estimate.
Unit tests cover the guest script, the capture and restore commands, the required markers, the guest's result on the host, a failure that the guest reports, and the snapshot's integrity.

Evidence

On a local Windows/WHP host, with OpenVMM built from e68871cc6 and the guest artifacts of the development release v0.1.0-dev.ef7c177c66d1:

Check	OpenVMM `e68871cc6` (this pin)	OpenVMM `a12d9167f` (previous pin)
`platform-filesystem`	Passes (4.0 s, and 2.5 s in a second run)	The clone prints `NVX-POST-RESTORE-OK: tier=platform`, then hangs on its first read until the 40 s phase timeout, which reproduces #904
`filesystem-snapshot`, `filesystem-shares`, `filesystem-policy`, `filesystem-comma-paths`, `denied-filesystem-paths`, `scratch-snapshot`, `platform-zero-fill`, `snapshot-tiers`	Pass	Not run

KVM and MSHV are left to CI.

Equivalence and validation

Pin. The gitlink is the exact head of [DONT REVIEW] virtiofs: wait for request admission instead of ending a queue worker openvmm#4762, whose two signed commits have a12d9167f, the previous pin and the head of feature/microvm/dev, as their parent. The OpenVMM protocol vectors and built-in CPU profiles that nvx-rs copies are unchanged: openvmm_vectors and openvmm_cpu_profiles pass with --ignored.
OpenVMM, on [DONT REVIEW] virtiofs: wait for request admission instead of ending a queue worker openvmm#4762's head, on Windows. For virtiofs: cargo clippy --all-targets -- -D warnings, cargo doc --no-deps, cargo nextest run (112 passed), the new tests under --stress-count 30 (30/30), and cargo xtask fmt --fix (clean).
NVX, at this head, on Windows. nvx.py verify, ruff check, ruff format --check, pyright for Linux and Windows, the Markdown link check, compileall, test_adversarial.py, test_hosts.py, and the validate-nvx unittest list (1,286 tests), whose only failures are the unrelated ones below.
Failing locally, unrelated. 18 subtests of test_setup_azure.py and, intermittently, test_host_loopback_allow_script_checks_publication_and_traffic, which run Linux shell scripts under Git's sh.exe here; test_setup_azure.py fails the same way without this branch's script changes. CI runs them on its runners.
Not run locally. KVM and MSHV, the rest of the microVM suite, shellcheck and shfmt (no checked shell script changed), the Specula wiring tests, and the OpenVMM VMM tests. CI runs them.

Exact NVX head submitted to CI: 864a5f020cc40da7fba33d7503e39e9dc46ed529.

Pin microsoft/openvmm#4762 at e68871cc6f14fceb6d38a899504ae8105f89e310, two commits on a12d9167f, the previous pin, which is the head of feature/microvm/dev. A tiered restore starts the VM's devices behind the post-restore input gate, which closes request admission on the microVM virtio-fs device. Its queue workers ended as soon as they found admission closed, and nothing started them again when the guest acknowledged the restore and input resumed, so every FUSE request of the clone, from its FUSE_INIT on, waited forever. A share attached at the restore, such as a clone's export bound to the dormant filesystem slot, could be mounted but never used. The first commit factors the virtio-fs integration test harness. The second makes each worker wait for admission to reopen instead of ending, and a stop still cancels the wait at once. That also fixes the rollback of a failed save, which starts the queues again behind the gate, and a worker that completes a request behind a snapshot boundary, which ended the same way. The snapshot guide and the virtio-fs reference now say that the guest's virtio-fs requests wait until the gate's release, so post-restore repair must not wait for their replies. Part of #904 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Record the two commits that the previous commit pins as positions 344 and 345 of the microVM chain, and move the NVX pin to e68871cc6. Position 344 factors the virtio-fs integration test harness, and position 345 makes a microVM virtio-fs queue worker wait for request admission to reopen instead of ending. They fix the request admission that position 111, in PR18, added: its worker ended for good when it found admission closed, as it does in the queues that a gated restore, or the rollback of a failed save, starts. Both are therefore PR18 fixups, to fold into position 111, rather than new work. Add them to PR18's row of the PR summary and to its commit mapping, the chain, and the table of the commits after the mapped tip, and extend the scope. The pin is now 188 commits past the mapped tip. Positions 341 to 343 are no longer only a proposal: microsoft/openvmm#4749 merged them, so position 343, a12d9167f, is the head of feature/microvm/dev. Say so in the scope, and name microsoft/openvmm#4762 as the proposal of positions 344 and 345. Part of #904 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

No test used a virtio-fs share after a tiered restore. The filesystem-snapshot scenario restores untiered snapshots, which arm no post-restore input gate, so it passed while every microVM virtio-fs queue that a gated restore started ended at once and left the clone's FUSE requests waiting forever. Add the platform-filesystem microVM scenario to the default suite. The scenario captures a platform snapshot without --mount, which leaves the filesystem slot dormant, and restores it with a --mount of a host directory, as a clone's export binds the slot. After the guest's platform repair acknowledges the restore and the gate releases, the clone's post-restore script mounts the share, reads the file that the host wrote, and writes one that the host then reads. The guest reports a failed check with NVX-PLATFORM-FILESYSTEM-FAIL, which fails the restore at once, and the restore may not change the snapshot. The platform capture needs sandbox blocks, so the scenario runs only with the Alpine sandbox-control guest, like platform-zero-fill. On a local Windows/WHP host, with an OpenVMM built from the pinned revision, it passed in 4.0 s; with the previous pin, a12d9167f, the clone acknowledged the restore and then hung on its first read until the 40 s phase timeout. The shards are planned again with an estimate of 6 s, which CI's duration reports replace once CI has run it. Unit tests cover the guest script, the capture and restore commands, the required markers, the guest's result on the host, a failure that the guest reports, and the snapshot's integrity. doc/ci.md describes the scenario, and doc/usage.md lists it among the scenarios that need the sandbox-control guest. Part of #904 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

🔵 Needs a closer look

The OpenVMM pin changes snapshot-sensitive queue concurrency and needs human review with confirmed KVM and MSHV validation.

No open issues 🎉

What changed in this PR

This PR updates NVX’s OpenVMM pin to address #904, where virtio-fs queue workers stopped during gated restores and left filesystem requests hanging. It adds regression coverage for attaching and using a share after a platform snapshot restore.

Changes:

Pins the OpenVMM fix that waits for request admission to reopen instead of ending queue workers.
Adds the platform-filesystem scenario and unit tests covering file exchange, failure reporting, and snapshot integrity.
Updates CI shard assignments, scenario documentation, and the upstream roadmap.

File	Description
`scripts/test_microvm_tests.py`	Tests scenario commands, markers, file results, failures, and snapshot integrity.
`scripts/nvx_tools/microvm_tests.py`	Registers and implements the platform filesystem restore scenario.
`scripts/nvx_tools/microvm_test_scripts/platform-filesystem.sh`	Mounts the restored share and checks guest read/write operations.
`scripts/nvx_tools/microvm_shards.json`	Adds the scenario and rebalances platform shards.
`doc/usage.md`	Documents the scenario’s Alpine sandbox-control requirement.
`doc/openvmm-upstream-roadmap.md`	Records the new pin and maps its two commits to PR18 fixups.
`doc/ci.md`	Describes the regression scenario and its expected behavior.

_{Review effort: Balanced}

This branch exists only to run CI. It is never a pull request's head, and this commit never goes onto a pull request's branch. It is #958's head, 4a614d9, which holds #955, #956, and #958, with the fixes that dev carries for the failures that their review bases, pinned to OpenVMM 79a501b6, show in CI, and that the pull requests do not change: - #885's test-only 3fbaf8a, cherry-picked before this commit. On GitHub's hosted Windows runners an open with backup semantics bypasses the test's access list. - This commit, which moves the openvmm gitlink 79a501b6 -> e68871cc, the virtio-fs fix of #904 that dev pins since #918, so that a clone's export serves requests after a gated restore. The platform tier's allowlist there admits maxcpus=, so the copy of it in nvx-rs/tests/vectors/openvmm, which the CLI jobs compare with the submodule, takes that line too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 34e6ffe3-3ebf-42db-bfca-e0ecfade6873

The nvx-openvmm-promote skill described a promotion as one commit that changes only the openvmm gitlink, and required exactly one commit on the target. Each of the twelve promotions from #801 to #1042 instead followed its gitlink commit with a doc: commit that maps the pinned commits in doc/openvmm-upstream-roadmap.md, as 6d3442d followed 70adf89 in #1042. Nothing said which of the roadmap's sections a promotion updates; #918, for one, left PR18's node in the dependency graph at 11 commits while the PR summary gave 13. A new section of the skill makes the roadmap commit part of the procedure. It takes the previous pin from the target's gitlink, and stops unless the pinned revision descends from it with no merge since the fork base. It tells an OpenVMM pull request that branched from an older head of feature/microvm/dev, which needs a rebase but leaves the roadmap valid, from a rebase of the whole chain, which shifts every position and needs the roadmap remapped. It derives the pin's position, its distance from the mapped tip, and the new positions from Git rather than by counting, and lists what a promotion updates: the Scope's pin and chain of OpenVMM pull requests, whose previous proposal has usually merged since; the branch DAG, whose final step grows only when it is unassigned; and the table of the commits after the mapped tip, whose remarks it backs with commands that find the earlier commits that change the same files. A fixup of mapped work also updates its PR's row, graph node, and commit mapping. The section ends by checking that the branch holds exactly the two commits and that the second changes only the roadmap. The pull-request section rebuilds both commits when the target moves, and the report covers both. Vectors: since the skill described the openvmm_vectors comparison, the test has come to check three excerpts of OpenVMM's sources and four versions of nvx::contract against the submodule too, while the skill still had a failure copy "both files". It now gives each kind of failure its fix, and runs the check-nvx-rs checks, whose runtime adapter compiles the excerpts. Landing: .gitmodules tracks feature/microvm/dev, and microsoft/openvmm allows only squash merges, which would land a commit other than the one that NVX pins. OpenVMM pull requests land by fast-forwarding that branch to the pinned revision, as microsoft/openvmm#4818 and microsoft/openvmm#4820 did. The skill now says so, pushes only when the user asks, and has the NVX pull request state that landing order. doc/contribute.md and .github/copilot-instructions.md described a one-commit, gitlink-only promotion too; they now describe both commits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The nvx-openvmm-promote skill described a promotion as one commit that changes only the openvmm gitlink, and required exactly one commit on the target. Each of the twelve promotions from #801 to #1042 instead followed its gitlink commit with a doc: commit that maps the pinned commits in doc/openvmm-upstream-roadmap.md, as 6d3442d followed 70adf89 in #1042. Nothing said which of the roadmap's sections a promotion updates; #918, for one, left PR18's node in the dependency graph at 11 commits while the PR summary gave 13. A new section of the skill makes the roadmap commit part of the procedure. It takes the previous pin from the target's gitlink, and stops unless the pinned revision descends from it with no merge since the fork base. It tells an OpenVMM pull request that branched from an older head of feature/microvm/dev, which needs a rebase but leaves the roadmap valid, from a rebase of the whole chain, which shifts every position and needs the roadmap remapped. It derives the pin's position, its distance from the mapped tip, and the new positions from Git rather than by counting, and lists what a promotion updates: the Scope's pin and chain of OpenVMM pull requests, whose previous proposal has usually merged since; the branch DAG, whose final step grows only when it is unassigned; and the table of the commits after the mapped tip, whose remarks it backs with commands that find the earlier commits that change the same files and, separately, check with git merge-tree whether the new commits apply to upstream main as they are. A fixup of mapped work also updates its PR's row, graph node, and commit mapping. The section ends by checking that the branch holds exactly the two commits and that the second changes only the roadmap. The pull-request section rebuilds both commits when the target moves, and the report covers both. Vectors: since the skill described the openvmm_vectors comparison, the test has come to check three excerpts of OpenVMM's sources and four versions of nvx::contract against the submodule too, while the skill still had a failure copy "both files". It now gives each kind of failure its fix, and runs the check-nvx-rs checks, whose runtime adapter compiles the excerpts. Landing: .gitmodules tracks feature/microvm/dev, and microsoft/openvmm allows only squash merges, which would land a commit other than the one that NVX pins. OpenVMM pull requests land by fast-forwarding that branch to the pinned revision, as microsoft/openvmm#4818 and microsoft/openvmm#4820 did. The skill now says so, pushes only when the user asks, and has the NVX pull request state that landing order. doc/contribute.md and .github/copilot-instructions.md described a one-commit, gitlink-only promotion too; they now describe both commits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The nvx-openvmm-promote skill described a promotion as one commit that changes only the openvmm gitlink, and required exactly one commit on the target. Each of the twelve promotions from #801 to #1042 instead followed its gitlink commit with a doc: commit that maps the pinned commits in doc/openvmm-upstream-roadmap.md, as 6d3442d followed 70adf89 in #1042. Nothing said which of the roadmap's sections a promotion updates; #918, for one, left PR18's node in the dependency graph at 11 commits while the PR summary gave 13. A new section of the skill makes the roadmap commit part of the procedure. It takes the previous pin from the target's gitlink, and stops unless the pinned revision descends from it with no merge since the fork base. It tells an OpenVMM pull request that branched from an older head of feature/microvm/dev, which needs a rebase but leaves the roadmap valid, from a rebase of the whole chain, which shifts every position and needs the roadmap remapped. It derives the pin's position, its distance from the mapped tip, and the new positions from Git rather than by counting, after unshallowing a submodule that the Copilot setup's --depth 1 left shallow, whose boundary would cut the counts short without an error. It lists what a promotion updates: the Scope's pin and chain of OpenVMM pull requests, whose previous proposal has usually merged since; the branch DAG, whose final step grows only when it is unassigned; and the table of the commits after the mapped tip, whose remarks it backs with commands that find the earlier commits that change the same files and, separately, check with git merge-tree whether the new commits apply to upstream main. Only a type-check of their packages on that merged tree lets the remarks say that they can go upstream on their own. A fixup of mapped work also updates its PR's row, graph node, and commit mapping. The section ends by checking that the branch holds exactly the two commits and that the second changes only the roadmap. The pull-request section rebuilds both commits when the target moves, and the report covers both. Vectors: since the skill described the openvmm_vectors comparison, the test has come to check three excerpts of OpenVMM's sources and four versions of nvx::contract against the submodule too, while the skill still had a failure copy "both files". It now gives each kind of failure its fix, and runs the check-nvx-rs checks, whose runtime adapter compiles the excerpts. Landing: .gitmodules tracks feature/microvm/dev, and microsoft/openvmm allows only squash merges, which would land a commit other than the one that NVX pins. OpenVMM pull requests land by fast-forwarding that branch to the pinned revision, as microsoft/openvmm#4818 and microsoft/openvmm#4820 did. The skill now says so, pushes only when the user asks, and has the NVX pull request state that landing order. doc/contribute.md and .github/copilot-instructions.md described a one-commit, gitlink-only promotion too; they now describe both commits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The nvx-openvmm-promote skill described a promotion as one commit that changes only the openvmm gitlink, and required exactly one commit on the target. Each of the twelve promotions from #801 to #1042 instead followed its gitlink commit with a doc: commit that maps the pinned commits in doc/openvmm-upstream-roadmap.md, as 6d3442d followed 70adf89 in #1042. Nothing said which of the roadmap's sections a promotion updates; #918, for one, left PR18's node in the dependency graph at 11 commits while the PR summary gave 13. A new section of the skill makes the roadmap commit part of the procedure. It takes the previous pin from the target's gitlink, and stops unless the pinned revision descends from it with no merge since the fork base. It tells an OpenVMM pull request that branched from an older head of feature/microvm/dev, which needs a rebase but leaves the roadmap valid, from a rebase of the whole chain, which shifts every position and needs the roadmap remapped. It derives the pin's position, its distance from the mapped tip, and the new positions from Git rather than by counting, after unshallowing a submodule that the Copilot setup's --depth 1 left shallow, whose boundary would cut the counts short without an error. It lists what a promotion updates: the Scope's pin and chain of OpenVMM pull requests, whose previous proposal has usually merged since; the branch DAG, whose final step grows only when it is unassigned; and the table of the commits after the mapped tip, whose remarks it backs with commands that find the earlier commits that change the same files and, separately, check with git merge-tree whether the new commits apply to upstream main. Only a type-check of their packages on that merged tree lets the remarks say that they can go upstream on their own. A fixup of mapped work also updates its PR's row, graph node, and commit mapping. The section ends by checking that the branch holds exactly the two commits and that the second changes only the roadmap. The pull-request section rebuilds both commits when the target moves, and the report covers both. Vectors: since the skill described the openvmm_vectors comparison, the test has come to check three excerpts of OpenVMM's sources and four versions of nvx::contract against the submodule too, while the skill still had a failure copy "both files". It now gives each kind of failure its fix, reruns the comparison until it passes, since each test stops at its first mismatch and only the NVX CLI jobs run these ignored tests, and runs the check-nvx-rs checks, whose runtime adapter compiles the excerpts. Landing: .gitmodules tracks feature/microvm/dev, and microsoft/openvmm allows only squash merges, which would land a commit other than the one that NVX pins. OpenVMM pull requests land by fast-forwarding that branch to the pinned revision, as microsoft/openvmm#4818 and microsoft/openvmm#4820 did. The skill now says so, pushes only when the user asks, and has the NVX pull request state that landing order. doc/contribute.md and .github/copilot-instructions.md described a one-commit, gitlink-only promotion too; they now describe both commits. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Pedro Henrique Penna (ppenna) and others added 3 commits October 26, 2026 18:09

Copilot AI balanced review requested due to automatic review settings October 27, 2026 01:11

Pedro Henrique Penna (ppenna) mentioned this pull request Oct 27, 2026

[DONT REVIEW] virtiofs: wait for request admission instead of ending a queue worker microsoft/openvmm#4762

Merged

Copilot started reviewing on behalf of Pedro Henrique Penna (ppenna) October 27, 2026 01:11

View session

Copilot AI reviewed Oct 27, 2026

View reviewed changes

Pedro Henrique Penna (ppenna) self-assigned this Oct 27, 2026

Pedro Henrique Penna (ppenna) merged commit 239efb9 into dev Oct 27, 2026
45 checks passed

Pedro Henrique Penna (ppenna) deleted the agents/issue-904-logical-commits-pr-guidelines branch October 27, 2026 15:46

Enrique Saurez (esaurez) mentioned this pull request Oct 28, 2026

runtime-adapter: start and run from a template, and run the stage's scenarios as one suite #963

Merged

Pedro Henrique Penna (ppenna) mentioned this pull request Oct 30, 2026

doc: describe the roadmap commit of an OpenVMM pin promotion #1052

Merged

Navigation Menu

Repository navigation

openvmm: serve virtio-fs requests after a gated restore - #918

openvmm: serve virtio-fs requests after a gated restore#918
Pedro Henrique Penna (ppenna) merged 3 commits into
devfrom
agents/issue-904-logical-commits-pr-guidelines

Pedro Henrique Penna (ppenna) commented Oct 27, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Pedro Henrique Penna (ppenna) commented Oct 27, 2026

Change

OpenVMM changes

Tests

Evidence

Equivalence and validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants