Repository navigation
Description
Title: --gpu ignores integrated GPUs (ggml DEVICE_TYPE_IGPU) and silently falls back to CPU
Summary
On a laptop whose only Vulkan device is an integrated GPU, zonos2-cli --gpu prints
zonos2: backend = CPU and runs on the CPU, with no warning. The Vulkan device is detected
(ggml_vulkan: Found 1 Vulkan devices), but it is never selected.
Cause
ggml-vulkan reports integrated GPUs as GGML_BACKEND_DEVICE_TYPE_IGPU
(ggml/src/ggml-vulkan/ggml-vulkan.cpp, at the pinned ggml 3af5f576:
return ctx->is_integrated_gpu ? GGML_BACKEND_DEVICE_TYPE_IGPU : GGML_BACKEND_DEVICE_TYPE_GPU;).
zonos2.cpp only looks up GGML_BACKEND_DEVICE_TYPE_GPU:
src/zonos2.cpp:77andsrc/dac.cpp:53:
if (use_gpu) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_GPU);src/zonos2.cpp:88andsrc/dac.cpp:57:
m.is_gpu = (ggml_backend_dev_type(dev) == GGML_BACKEND_DEVICE_TYPE_GPU);
so an iGPU is skipped and the CPU device is used.
Environment
| zonos2.cpp | v0.5.2 (adce4ffb), prebuilt zonos2-windows-x64-vulkan.zip |
| Model | Zyphra/ZONOS2-GGUF@75c877ec: zonos2-q8_0.gguf, dac.gguf |
| Machine | Intel Core Ultra 5 125U (Meteor Lake) laptop, 32 GB, Windows 11 |
| GPU | Intel(R) Graphics (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: none |
Reproduce
zonos2-cli zonos2-q8_0.gguf --tts "<a 14-word English sentence>" out.wav --dac dac.gguf --gpu --seed 1 --max 800
Release binary:
ggml_vulkan: 0 = Intel(R) Graphics (Intel Corporation) | uma: 1 | ...
zonos2: backend = CPU | Intel(R) Core(TM) Ultra 5 125U
generate(kv): 480 decode steps in 21.033s ... RTF=3.77
dac: H=481 eos=471 -> 241152 samples = 5.47s, rms=0.0241 peak=0.2220
The CPU output is correct (faster-whisper large-v3-turbo transcribes the sentence verbatim) and
identical to the same run on another x86 CPU (eos=471, rms=0.0241).
Experiment with a two-line patch
A local build of adce4ffb (Vulkan SDK 1.4.313.2, MSVC 14.44, -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=OFF) with this change in both zonos2.cpp and dac.cpp:
if (use_gpu) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_GPU);
+ if (use_gpu && !dev) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_IGPU);
if (!dev) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_CPU);
...
- m.is_gpu = (ggml_backend_dev_type(dev) == GGML_BACKEND_DEVICE_TYPE_GPU);
+ m.is_gpu = (ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_CPU);does select the iGPU, but the output is wrong:
zonos2: backend = Vulkan0 | Intel(R) Graphics
generate(kv): 799 decode steps in 43.257s = 18.5 frames/s (real-time = 86.1), RTF=4.66
dac: loaded 128 tensors on GPU
dac: H=800 eos=-1 -> 409600 samples = 9.29s, rms=0.0846 peak=0.5774
No EOS (runs to the frame cap), and faster-whisper recognises only the first two words (and
one of them wrongly), followed by unintelligible audio. This looks like the same Vulkan divergence as #11 (RADV) and #10
(NVIDIA on Windows), now also on an Intel iGPU, so it is probably not specific to one driver.
Suggestions
- Accept
GGML_BACKEND_DEVICE_TYPE_IGPUfor--gpu(or at least print a warning when--gpu
was requested and no GPU device was chosen), so the fallback is not silent. - Until the Vulkan divergence (Vulkan backend: Japanese generation often never emits EOS and runs to the frame cap (CUDA/CPU do not reproduce) #10, **Vulkan backend on RADV/RDNA3: wrong pitch, no EOS, and non-deterministic output — CPU backend is bit-identical** #11) is fixed, it may help to note in the README that the
Vulkan output can differ from CPU, and that CPU is the reference path.
Activity