Repository navigation
Support AMD CPUs in the time ABI's CPU profiles #396
Description
Summary
Since #325, every microVM cold boot selects a CPU profile, and only Intel profiles exist. On an AMD host, nvx.py run fails before the VM starts. --cpu-profile auto, the default, fails with E_PROFILE_HOST_UNKNOWN. --cpu-profile host, the opt-in for development hosts that #394 adds, fails with the same code, because host profiles support only Intel CPUs. AMD hosts booted microVMs before #325.
An AMD fingerprint cannot simply become a profile, as the Alder Lake one did for #390: the profile crate, the topology code, and the Hyper-V backends assume Intel's CPUID layout. A profile derived from an AMD fingerprint fails validation today. Once validation accepts it, every cold boot fails with E_CPU_SURFACE, because OpenVMM writes AMD topology fields that the profile pins.
This issue tracks what an investigation of #390 for AMD found. Nothing ran on AMD hardware: neither the configured development hosts nor the CI microVM runners are AMD. Line references are permalinks to nanvix/openvmm main at 974f356, the base of nanvix/openvmm#111, and to NVX dev at 3617108.
What blocks AMD
| # | Where | Problem | Suggested fix |
|---|---|---|---|
| 1 | validate_identity(), and derive_host_profile() and supports_host_profiles() in nanvix/openvmm#111 |
Only GenuineIntel passes, and the host profile's brand string is Intel's. supports_host_profiles() also decides where OpenVMM's auto error suggests --cpu-profile host |
Map AuthenticAMD to amd (Hygon is optional), and give each vendor its own generic brand string |
| 2 | vm_owned_bits() |
topology_cpuid() writes AMD's core count in 0x8000_0008 ECX and the topology leaf 0x8000_001E, but the profile pins those bits, so every AMD cold boot fails with E_CPU_SURFACE |
Make the AMD topology fields VM-owned in AMD profiles only: the Intel profiles pin 0x8000_0008 ECX under mask 0xffffffff, so a vendor-neutral change would change their digests |
| 3 | DATA_LEAVES and apply_policy() |
0x8000_0005, 0x8000_001D, 0x8000_001E, and 0x8000_0021 lose their data. Nothing in 0x8000_0001 ECX or 0x8000_0008 EBX is cleared, so a profile can expose SVM (KVM reports it with nested virtualization enabled), IBS, LWP, the core performance counter extensions, MONITORX, RDPRU, PPIN, and CPPC wherever a backend offers them |
Add an AMD section to the policy. Keep 0x8000_0021, which carries AutoIBRS, LFENCE serialization, SBPB, SRSO_NO, and the TSA bits |
| 4 | expected_subleaves() and register_class() |
A profile lists 0x8000_001D only at subleaf 0, rather than up to its null cache type as for leaf 4, and derivation combines its cache descriptors as feature flags instead of by majority |
Treat 0x8000_001D as leaf 4 is treated, and classify it as informational |
| 5 | HV_FEATURES |
The Hyper-V IBRS, IBPB, STIBP, SSBD (mdd), PSFD, and nested-virtualization bits map only to Intel's CPUID locations; AMD enumerates them in 0x8000_0008 EBX and 0x8000_0001 ECX. On WHP and MSHV, an AMD profile would likely clear those bits, hide CPUID bits that the profile pins (E_CPU_SURFACE), and take the speculation-control MSRs from the guest. The AMD-only bits (srso_no, sbpb, btc_no, and tsa_*_no) are not mapped |
Map each feature to its vendor's CPUID bit, and run the WHP hardware test features_control_the_mapped_cpuid_bits on an AMD host |
| 6 | MSHV's is_per_vp_leaf() |
Only 0xB and 0x1F get per-VP results. If the hypervisor does not fill 0x8000_001E for each VP, Linux logs APIC ID mismatch, which fails the guest check G_APIC_ID_MISMATCH. KVM and WHP already set it |
Give 0x8000_001E per-VP results on MSHV, after checking what the hypervisor presents |
| 7 | topology_cpuid() |
The AMD path writes no cache-sharing counts, so a guest would see the host's L3 grouping in 0x8000_001D instead of one L3 per socket, as on Intel |
Write AMD's cache sharing in 0x8000_001D, and test it as pinned_profiles_share_the_l3_cache_across_the_socket tests leaf 4 |
| 8 | AMD MSRs, for example in KVM's identity_msr_filter() |
No backend defines what a guest reads from HWCR (0xC0010015) or DE_CFG (0xC0011029). On a CPU with a constant TSC, Linux's bsp_init_amd() reads HWCR with an unchecked rdmsr: a fault fails the guest check G_UNCHECKED_MSR, and the kernel warns unless HWCR bit 24 (the TSC counts at the P0 frequency) is set |
Define and test the AMD MSR contract on each backend, and confirm the 200 MHz LAPIC timer rate on Hyper-V AMD hosts |
Decision: TOPOEXT
CPUID.0x80000001:ECX[22], TOPOEXT, enumerates 0x8000_001D and 0x8000_001E. Keeping it mirrors the Intel contract, in which every VP has its own L1 and L2 caches and the socket shares the L3, but it needs rows 6 and 7. Clearing it makes row 6 moot, but Linux then finds no shared cache, and its scheduler treats every CPU as having its own last-level cache.
Already fine for AMD
HostCpuSignaturedecodes AMD's family and model (the extended family only for base family0xF), and fingerprints enumerate0x8000_001Dup to its null cache type.- Profile selection and profile IDs,
<vendor>.<generation>.v<revision>, are vendor-neutral. - KVM emulates
IA32_ARCH_CAPABILITIESon AMD hosts too, so the time ABI's read of KVM's feature MSR works there. - WHP's XSAVE component list covers Zen's components.
- The guest checks C2 and C5 hold: the time ABI sets ARAT and the invariant TSC by policy, and KVM's out-of-range CPUID results for an AMD guest are zeros, which the identity contract allows.
- NVX's guest kernel enables
CONFIG_CPU_SUP_AMDand the AMD mitigations (SRSO,IBPB_ENTRY, andUNRET_ENTRY), anddoctor's host probes decode AMD signatures and the invariant TSC.
Plan
- 1. Rows 1 to 4 in the profile crate, tested with a synthetic AMD fingerprint. No hardware is needed, and this makes
amd.host.v1available on AMD development hosts. - 2. Row 7, AMD cache sharing, in
virt::x86::topology. - 3. Row 5, the Hyper-V feature banks.
- 4. Rows 6 and 8 on AMD hosts, and the TOPOEXT decision.
- 5. Fingerprint AMD hosts on every backend that a profile should serve: Azure Dasv5 (Milan) or Dasv6 (Genoa) VMs for KVM, MSHV, and WHP, and developer Ryzen machines for WHP.
--cpu-fingerprintwrites the fingerprint before it fails, so any AMD host can contribute one today. - 6. Pin one profile per microarchitecture, such as
amd.milan.v1for family0x19, model0x01, add it to NVX's generation table, and qualify its hosts withnvx.py doctorandnvx.py test-microvm. - 7. In NVX, update
PROFILE_VENDORS,HOST_PROFILE_VENDORSand therunguidance from time ABI: boot developer CPUs with an Alder Lake profile and opt-in host profiles #394 (which link this issue on AMD hosts), the time ABI specification (the deferral of AMD profiles, the Intel-only host profiles, and the failure codes), and the CPU profiles section ofdoc/usage.mdanddoc/setup.md. Give AMD runners their own performance series: the platform names do not include the CPU vendor. - 8. Check whether the GitHub-hosted
ubuntu-latestrunners, where Copilot cloud agent sessions runtest-microvm --backend kvm, are AMD; GitHub-hosted runners commonly report AMD EPYC CPUs, andcopilot-setup-stepsdoes not log the CPU. If they are, those sessions cannot boot microVMs until a profile serves their CPU.
Activity
openvmm_core: name --cpu-profile host when no profile serves the host
run: add --cpu-profile and explain E_PROFILE_HOST_UNKNOWN
openvmm_core: name --cpu-profile host when no profile serves the host
run: add --cpu-profile and explain E_PROFILE_HOST_UNKNOWN
run: add --cpu-profile and explain E_PROFILE_HOST_UNKNOWN
run: add --cpu-profile and explain E_PROFILE_HOST_UNKNOWN
run: add --cpu-profile and explain E_PROFILE_HOST_UNKNOWN
openvmm: serve AMD EPYC Milan hosts and AMD host profiles
doc: keep AMD hosts out of the Intel performance series
copilot-setup-steps: report the runner's CPU
cpu_profile: pin AMD Genoa and Turin CPU profiles
openvmm: serve AMD EPYC Genoa and Turin hosts
copilot-instructions: name the runner CPUs that boot microVMs
copilot-instructions: name the runner CPUs that boot microVMs
cpu_profile: pin AMD Genoa and Turin CPU profiles
openvmm: serve AMD EPYC Genoa and Turin hosts
copilot-instructions: name the runner CPUs that boot microVMs
cpu_profile, virt: accept AMD CPUs in CPU profiles and host profiles
cpu_profile: map the Hyper-V feature banks to AMD's CPUID bits
virt: serve AMD's HWCR and DE_CFG and per-VP 0x8000001E under AMD pro…
cpu_profile: pin an AMD Milan CPU profile, amd.milan.v1
openvmm: check MSHV's --cpu-fingerprint unlisted CPUID like a cold boot
Merge pull request microsoft#412 from microsoft/fix/issue-396-amd-gen…