Skip to content

--gpu ignores integrated GPUs (ggml DEVICE_TYPE_IGPU) and silently falls back to CPU #14

Open

Description

@Selengiks

Title: --gpu ignores integrated GPUs (ggml DEVICE_TYPE_IGPU) and silently falls back to CPU

Summary

On a laptop whose only Vulkan device is an integrated GPU, zonos2-cli --gpu prints
zonos2: backend = CPU and runs on the CPU, with no warning. The Vulkan device is detected
(ggml_vulkan: Found 1 Vulkan devices), but it is never selected.

Cause

ggml-vulkan reports integrated GPUs as GGML_BACKEND_DEVICE_TYPE_IGPU
(ggml/src/ggml-vulkan/ggml-vulkan.cpp, at the pinned ggml 3af5f576:
return ctx->is_integrated_gpu ? GGML_BACKEND_DEVICE_TYPE_IGPU : GGML_BACKEND_DEVICE_TYPE_GPU;).
zonos2.cpp only looks up GGML_BACKEND_DEVICE_TYPE_GPU:

  • src/zonos2.cpp:77 and src/dac.cpp:53:
    if (use_gpu) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_GPU);
  • src/zonos2.cpp:88 and src/dac.cpp:57:
    m.is_gpu = (ggml_backend_dev_type(dev) == GGML_BACKEND_DEVICE_TYPE_GPU);

so an iGPU is skipped and the CPU device is used.

Environment

zonos2.cpp v0.5.2 (adce4ffb), prebuilt zonos2-windows-x64-vulkan.zip
Model Zyphra/ZONOS2-GGUF@75c877ec: zonos2-q8_0.gguf, dac.gguf
Machine Intel Core Ultra 5 125U (Meteor Lake) laptop, 32 GB, Windows 11
GPU Intel(R) Graphics (Intel Corporation) | uma: 1 | fp16: 1 | bf16: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matrix cores: none

Reproduce

zonos2-cli zonos2-q8_0.gguf --tts "<a 14-word English sentence>" out.wav --dac dac.gguf --gpu --seed 1 --max 800

Release binary:

ggml_vulkan: 0 = Intel(R) Graphics (Intel Corporation) | uma: 1 | ...
zonos2: backend = CPU | Intel(R) Core(TM) Ultra 5 125U
generate(kv): 480 decode steps in 21.033s ... RTF=3.77
dac: H=481 eos=471 -> 241152 samples = 5.47s, rms=0.0241 peak=0.2220

The CPU output is correct (faster-whisper large-v3-turbo transcribes the sentence verbatim) and
identical to the same run on another x86 CPU (eos=471, rms=0.0241).

Experiment with a two-line patch

A local build of adce4ffb (Vulkan SDK 1.4.313.2, MSVC 14.44, -DGGML_VULKAN=ON -DBUILD_SHARED_LIBS=OFF) with this change in both zonos2.cpp and dac.cpp:

     if (use_gpu) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_GPU);
+    if (use_gpu && !dev) dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_IGPU);
     if (!dev)    dev = ggml_backend_dev_by_type(GGML_BACKEND_DEVICE_TYPE_CPU);
...
-    m.is_gpu  = (ggml_backend_dev_type(dev) == GGML_BACKEND_DEVICE_TYPE_GPU);
+    m.is_gpu  = (ggml_backend_dev_type(dev) != GGML_BACKEND_DEVICE_TYPE_CPU);

does select the iGPU, but the output is wrong:

zonos2: backend = Vulkan0 | Intel(R) Graphics
generate(kv): 799 decode steps in 43.257s = 18.5 frames/s (real-time = 86.1), RTF=4.66
dac: loaded 128 tensors on GPU
dac: H=800 eos=-1 -> 409600 samples = 9.29s, rms=0.0846 peak=0.5774

No EOS (runs to the frame cap), and faster-whisper recognises only the first two words (and
one of them wrongly), followed by unintelligible audio. This looks like the same Vulkan divergence as #11 (RADV) and #10
(NVIDIA on Windows), now also on an Intel iGPU, so it is probably not specific to one driver.

Suggestions

  1. Accept GGML_BACKEND_DEVICE_TYPE_IGPU for --gpu (or at least print a warning when --gpu
    was requested and no GPU device was chosen), so the fallback is not silent.
  2. Until the Vulkan divergence (Vulkan backend: Japanese generation often never emits EOS and runs to the frame cap (CUDA/CPU do not reproduce) #10, **Vulkan backend on RADV/RDNA3: wrong pitch, no EOS, and non-deterministic output — CPU backend is bit-identical** #11) is fixed, it may help to note in the README that the
    Vulkan output can differ from CPU, and that CPU is the reference path.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions