Overlay upper: support virtio-blk scratch or tmpfs end to end, and align docs #558

Goal

Make the implementation and the docs agree on this statement:

For the read-write top of the overlay, NVX supports either a scratch space (a virtio-blk device) or a tmpfs in RAM.

Today only the scratch half is true end to end. OpenVMM accepts a RAM-backed overlay layout, but no NVX guest, launcher, or test uses it, and the docs say the guest agents always use scratch. Found while reviewing the architecture guide in #529.

Current state

Code links are pinned to dev at 19a359b, design-guide links to the #529 head at 1574629, and OpenVMM links to the pinned gitlink nanvix/openvmm@7f225e2.

Implementation

• Guest. nvx-init-agent stops with a scratch device is required, accepts only ext4, and mounts the overlay with upperdir=$scratch/upper. It has no tmpfs or ramfs path.
• Launchers. sandbox.py requires an ext4 scratch image and always passes a scratch block and nvx_scratch=...,ext4. Nothing in scripts/, nvx-rs, or aci_edge_sandboxes emits nvx_overlay_upper.
• VMM. OpenVMM's ramfs_overlay_requested accepts exactly one nvx_overlay_upper=ramfs token. The token only relaxes block validation: exactly one read-only distro block, no scratch, and no snapshot or restore. OpenVMM doesn't control what the guest mounts.
• Tests. nvx-sandbox-smoke fails when the upper isn't /run/nvx/scratch/upper, and test_nvx_tools.py pins that check. OpenVMM unit-tests the token, but no NVX microVM scenario boots the RAM layout.

Docs

• Design guide (doc: replace doc/design with an architecture-only design guide #529). 4.1 says "The guest agents always keep the overlay upper layer on scratch, so the machine's RAM-backed overlay layout, which has no scratch, serves only guests that declare it." Table 4.1 and Figure 4.1 show only scratch, and the 1.1 goals list only scratch. 2.1, 3.4, and 5.1 describe the RAM layout as a VMM allowance and call it ramfs or "in RAM", never tmpfs.
• Old doc/design/, which doc: replace doc/design with an architecture-only design guide #529 removed. cold-boot.md says "The NVX init agent does not use this layout." sandbox-filesystem-and-agent-architecture.md argues against tmpfs for the workload: "Runtime tmpfs is still appropriate for small agent state, not the workload's general writable layer."

Related work

• Shared image-backed sandbox runtime #535 tracks an image-backed runtime with "a tmpfs or ext4-scratch writable root".
• nvx-rs: factor minimal guest rootfs and descriptor-rooted host mapping #539 plans Upper::Tmpfs and Upper::Ext4Scratch in the nvx-rs guest core, and asks not to change the existing shell profile before its replacement is accepted.
• nvx-rs: provide a standalone image-backed sandbox cold-start path #541 requires both upper modes on the standalone cold-start path.
• nvx-rs: prove cross-image platform restore with pre-ready activation RPC #542 plans a blockless RAM-upper platform template.

This issue tracks what those share: one name and token for the RAM mode, launchers and tests that exercise it, and docs that describe what ships. If the guest side lands only through #539 and #541, close this issue once they land and the docs match.

Proposed work

1. Name and token. Settle tmpfs versus ramfs. tmpfs takes a size= limit and fails writes with ENOSPC when full; ramfs has no limit and can grow until the guest runs out of memory. Either change the OpenVMM token (for example, to nvx_overlay_upper=tmpfs) in nanvix/openvmm and promote the pin, or keep ramfs as the layout name and document that the guest mounts tmpfs. The layout rejects capture and restore, so no snapshot records the token today.
2. Guest. When the host declares the RAM layout, mount a size-bounded tmpfs with nosuid,nodev for the upper and work directories instead of resolving scratch, and unmount it at teardown. Validate the overlay options NVX uses (metacopy=on, xino=on) on that upper with the pinned kernel. Refuse a RAM layout that also attaches scratch, and a scratch layout without scratch.
3. Launchers. Let nvx.py sandbox select the upper. In RAM mode, emit the token, attach no scratch block, accept only the distro layer, and reject snapshot options, matching OpenVMM. Do the same for nvx-rs through nvx-rs: provide a standalone image-backed sandbox cold-start path #541.
4. Resources. A tmpfs upper consumes guest RAM, and its pages are charged to the memory cgroup of the task that writes them, so workload writes count against nvx_memory_max. Choose the default size and who sets it, and document the trade-off: scratch fails with ENOSPC on host storage, while a tmpfs upper adds guest memory pressure.
5. Tests. Add a microVM scenario that boots one read-only distro block with a tmpfs upper and no scratch. Make nvx-sandbox-smoke and its test_nvx_tools.py vectors check the declared upper instead of scratch only. Add negative cases for mismatched layouts and for snapshot options in RAM mode.
6. Docs. Describe both modes, their restrictions, and their trade-offs, and remove the "always on scratch" statements:
   • The architecture guide in doc/design/: 1.1, 2.1.3, 2.5 Table 2.7, 3.4 (intro and Table 3.11), 4.1 (Table 4.1, Figure 4.1, and the 4.1.1 assembly and teardown steps), 4.3 (memory accounting), and 5.1.2.
   • doc/run.md and doc/usage.md for the launcher option, and the OpenVMM Guide's run.md and cli.md if the token changes.

Acceptance criteria

• An NVX guest boots a layered sandbox with either an ext4 scratch upper on virtio-blk or a size-bounded tmpfs upper, chosen by the host, on Linux/KVM, Linux/MSHV, and Windows/WHP.
• OpenVMM, the launchers, the guest, and the docs use one name and one token for the RAM mode.
• At least one NVX launcher can request each mode, and the RAM mode attaches no scratch device.
• Smoke checks and unit tests accept the declared upper and reject a mismatch.
• The architecture guide in doc/design/ describes both modes, the RAM mode's restrictions (one read-only distro block, no capture or restore), and their resource trade-offs.

Open questions

• Keep ramfs as the token value, switch to tmpfs, or accept both?
• Should guest support land first in the shell nvx-init-agent, or only in the nvx-rs guest core from nvx-rs: factor minimal guest rootfs and descriptor-rooted host mapping #539?
• What is the default tmpfs size, and is it a host token or a fraction of guest RAM?
• Should a tmpfs upper share the workload's memory limit or get its own?

Out of scope

• Capturing or restoring a populated tmpfs upper. The nvx-rs: prove cross-image platform restore with pre-ready activation RPC #542 RAM-upper template is captured before overlay assembly.
• A separate mismatch: the architecture guide describes one read-only lower layer (distro) and calls 0xd0004000..0xd0005fff unused, while OpenVMM and nvx-init-agent still define runtime and custom layers there. microVM: remove the runtime and custom sandbox block roles #556 tracks it.