Skip to content

nvx-rs: replace the golden CLI scenarios with structured scenario tests #728

Description

Goal

Replace tests/fixtures/cli.json and its replay, nvx_rs_reproduces_the_recorded_scenarios in tests/cli.rs, with scenario tests that check what callers rely on: exit statuses, error codes and stable fragments, the state directory and its parsed JSON files, file modes, and the arguments that OpenVMM receives, rather than transcripts recorded from Python.

Background

  • cli.json holds 141 POSIX and 140 Windows scenarios, in 426 steps, recorded from nvx.py sandbox, egress compile, and artifacts, with nvx-fake-openvmm as OpenVMM. Each step compares the exit status, standard output and error, and state files byte for byte, after replacing the root, process IDs, and random identifiers with placeholders.
  • On Windows, only the replay reaches 533 lines and 130 branches, mostly in src/host/cli/sandbox/ and src/testing/fake_openvmm.rs, and all of egress compile.
  • nvx.py sandbox passes the streams and status of nvx-rs through, and the hypervisor suites match message fragments and parse runtime.json and outcome reports.

Tasks

Replacement tests, in a pull request that doesn't change behavior:

Retirement, in the next pull request:

  • Delete cli.json, the replay and its normalization helpers, and the mapping test.
  • Write only the newest config.json format, and keep reading formats 1 to 6. Drop the README's promise that an older nvx.py can operate a sandbox that nvx-rs provisioned.
  • Docs: replace the README's "Golden scenarios" section, and update its Testing section and doc/project-structure.md.

Gate

Coordination

Acceptance criteria

  • cli.json is gone, and the scenario tests pass on Linux and Windows.

Activity

  1. 33 remaining items

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions