From b5528d605eae00a82a6140b9358582add712c351 Mon Sep 17 00:00:00 2001 From: ZeroDot1 Date: Fri, 23 Oct 2026 13:55:11 +0200 Subject: [PATCH] Fix HIP VMM access updates for growing pools When the HIP VMM pool grows, grant access across the fully mapped contiguous prefix instead of applying the update only to the newest subrange. Some HIP runtimes reject the subrange update with an invalid argument even though the region has just been mapped. The next allocation then aborts during model inference. Reapplying permissions to the mapped prefix keeps the VMM pool usable as it grows. CUDA keeps its existing per-mapping access call. The HIP path does not disable VMM or suppress errors; failures from hipMemSetAccess remain fatal. Validation on CachyOS with ROCm 7.2.4 and an AMD Radeon RX 9070 XT (gfx1201): an ACE-Step Base Q8_0 build with GGML_HIP_NO_VMM=OFF reproduced the original invalid-argument failure during planner flash-attention before this change. After the change, the same VMM-enabled build completed two ACE-Step generations and wrote valid 48 kHz stereo PCM WAV files: 5.12 seconds and 8.00 seconds. The HIP backend reported VMM: yes. Built audiocpp_cli successfully. --- external/ggml/src/ggml-cuda/ggml-cuda.cu | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/external/ggml/src/ggml-cuda/ggml-cuda.cu b/external/ggml/src/ggml-cuda/ggml-cuda.cu index f20c18b65..a92a19aa7 100644 --- a/external/ggml/src/ggml-cuda/ggml-cuda.cu +++ b/external/ggml/src/ggml-cuda/ggml-cuda.cu @@ -554,7 +554,14 @@ struct ggml_cuda_pool_vmm : public ggml_cuda_pool { access.location.type = CU_MEM_LOCATION_TYPE_DEVICE; access.location.id = device; access.flags = CU_MEM_ACCESS_FLAGS_PROT_READWRITE; - CU_CHECK(cuMemSetAccess((CUdeviceptr)((char *)(pool_addr) + pool_size), reserve_size, &access, 1)); +#if defined(GGML_USE_HIP) + // Some HIP runtimes reject access changes applied to a newly mapped + // subrange inside a larger reservation. Reapply access to the fully + // mapped, contiguous prefix; the unmapped tail is excluded. + CU_CHECK(cuMemSetAccess(pool_addr, pool_size + reserve_size, &access, 1)); +#else + CU_CHECK(cuMemSetAccess(start_ptr, reserve_size, &access, 1)); +#endif // add to the pool pool_size += reserve_size;