Skip to content

MiniMax-H3 I2VA/FL2VA: ROCm VAE encode crashes with 0xC0000005 (ACCESS_VIOLATION) on Windows(问题已解决) #1897

Closed

Description

@farawayso

[Bug] MiniMax-H3 I2VA/FL2VA: ROCm VAE encode crashes with 0xC0000005 (ACCESS_VIOLATION) on Windows

sd-cli version: stable-diffusion.cpp version unknown, commit 97d2990 (pre-built Windows ROCm binary, ~2026-08-19)
OS: Windows 11
GPU: AMD Radeon RX 7900 XTX (gfx1100, 24GB VRAM / 23.7GB usable)
ROCm: ROCm 7.14.0 (bundled with the pre-built binary)

Description

MiniMax-H3 I2VA (image-to-video+audio) and FL2VA (first+last frame → video+audio) modes crash with exit code 3221225725 (0xC0000005, STATUS_ACCESS_VIOLATION) when using the ROCm HIP backend (--backend llm=cpu,vae=ROCm0,diffusion=ROCm0).

The crash is deterministic (100% reproduction rate) and occurs during VAE encode of the input image — the very first GPU operation after model loading. The diffusion sampling and VAE decode stages are never reached.

Reproduction

Command line

sd-cli.exe --mode vid_gen \
  -p "<lora:minimax_h3_turbo_v11_sdcpp:1.0> 美女说,树青,听说你去洱海了。" \
  --init-image input.png \
  --width 480 --height 864 \
  --video-frames 175 \
  --sample-steps 5 --flow-shift 6 --cfg-scale 1.0 \
  --seed 42 \
  --backend llm=cpu,vae=ROCm0,diffusion=ROCm0 \
  --offload-to-cpu \
  --eager-load \
  --lora-model-dir ./models/MiniMax-H3/loras \
  --lora-apply-mode at_runtime \
  --diffusion-fa \
  --max-vram ROCm0=22

The crash also occurs without --offload-to-cpu, with --max-vram values 14–22, and with or without --eager-load. The only variable that matters is the backend: vae=ROCm0 always crashes; vae=Vulkan1 always works.

Via menu

Select backend [5] ROCm+VAE乒乓, model pruned, mode I2VA, provide an input image. The auto-route (menu [4]) correctly avoids this by sending I2VA/FL2VA to Vulkan — the crash only happens when ROCm is explicitly selected.

Crash log

[INFO ] stable-diffusion.cpp:6123 - MiniMax-H3 I2VA
[DEBUG] vae.hpp:142  - VAE Tile size: 16x16
[DEBUG] ggml_extend.hpp:884  - num tiles : 2, 5
[DEBUG] ggml_extend.hpp:885  - optimal overlap : 0.125000, 0.406250 (targeting 0.250000)
[DEBUG] ggml_extend.hpp:886  - processing 10 tiles
[INFO ] ggml_graph_cut.cpp:973  - minimax_h3_video_vae build cached graph cut plan done (taking 1 ms)
[DEBUG] model_manager.cpp:506  - model manager staged compute params (344.04 MB, 118 tensors) to ROCm0, taking 0.11s
[DEBUG] ggml_extend.hpp:2244 - minimax_h3_video_vae compute buffer size: 627.13 MB(VRAM)

  [STDERR]   Device 0: AMD Radeon RX 7900 XTX, gfx1100 (0x1100), VMM: no, Wave Size: 32, VRAM: 24560 MiB
[FAILED] sd-cli.exe 退出码: 3221225725

The image preprocessing (crop + resize) completes successfully, and all four model files load and initialize normally. The crash happens at the instant the VAE encode graph is computed on ROCm0 — 118 tensors (344MB) are staged to VRAM, then the compute buffer is allocated (627MB), and immediately the process dies with an access violation. No error message from the HIP runtime, no user-mode exception handling.

Successful Vulkan comparison (same config, same hardware)

Running the identical config with --backend llm=cpu,vae=Vulkan1,diffusion=Vulkan1 completes successfully:

[DEBUG] vae.hpp:142  - VAE Tile size: 16x16
[DEBUG] ggml_extend.hpp:2244 - minimax_h3_video_vae compute buffer size: 627.13 MB(VRAM)
  |==================================================| 10/10 - 19.61it/s
[DEBUG] vae.hpp:166  - computing vae encode graph completed, taking 0.62s
Stage ROCm (crash) Vulkan (works)
VAE encode (image→latent) 0xC0000005 crash during compute 0.62s, 19.6it/s, clean
LLM vision conditioning never reached 68.6s (normal for Qwen3-VL)
Diffusion sampling (5 steps) never reached 122.8s@22.1s/it
VAE decode (4 passes × 10 tiles) never reached 21.9s
Total generation crash 231.1s / 4m17s

What has been ruled out

  • Not the graph-cut bugs (MiniMax-H3: segmented offload (max_vram) deterministically produces an unrelated image (SD1.5 unaffected) #1865/fix: prevent gallocr hash overflow in tiny graph-cut segments #1880/fix: mark graph cuts with both a prefix and a suffix #1883): Those were in the diffusion sampling graph-cut path and were fixed in master-826 (16304cc). The VAE encode path is separate code — it uses its own graph-cut plan (minimax_h3_video_vae), not the diffusion model's graph. The encode path has never been patched.
  • Not an OOM: VRAM during load is 0MB (all weights in RAM via --offload-to-cpu), and the encode only needs 344MB staged + 627MB compute buffer = ~1GB, well within 23.7GB available. The same config runs to completion on Vulkan with the same VRAM budget.
  • Not model-specific: Both minimax_h3_fl2va_pruned-Q4_K_M (11GB) and minimax_h3_fl2va-Q4_K_M (18GB, full) models crash identically when I2VA/FL2VA is used on ROCm.
  • Not input-image-specific: Tested with various input images (768×1368, 2048×2048, diverse content) — all crash at the same point.
  • Not mitigatable: --offload-to-cpu, different --max-vram values (14–22), --eager-load, --vae-on-cpu (deprecated) — none prevent the crash. The only working VAE encode path on this hardware is Vulkan.

Hypothesis

The MiniMax-H3 video VAE (minimax_h3_video_vae_fp16.safetensors, 5.6GB, 562 tensors) is a custom architecture that differs from the standard SD VAE — it has a 24-channel latent space, temporal clip length 17, and uses tiled encoding. The ggml HIP backend for ROCm likely has a bug in one of the operations used by the VAE encoder graph (possibly a 2D convolution, a memory layout/view operation, or the tiled overlap handling) that causes an out-of-bounds memory access. The Vulkan backend handles the same computation graph without issue, suggesting the bug is in the HIP kernel implementation, not in the model itself or the higher-level VAE code.

Additional context: T2V/T2VA works fine on ROCm

Text-only modes (T2V/T2VA, no image input, no VAE encode needed) run flawlessly on the same ROCm binary — including VAE decode during the final video assembly. The VAE decode path uses the same model weights but different graph operations, and it completes successfully. This strongly points to a bug specific to the VAE encode graph's kernel set on the ROCm HIP backend.

Attachments

  • Full crash log (from --verbose output): the log above is the complete relevant portion; the model loading section (before the crash) is unremarkable — all four models load without warnings.
  • Successful Vulkan run log: available on request.

Activity

  1. stduhpf commented on Aug 21, 2026

    @stduhpf
    Contributor

    Did you manage to run any other model with the same ROCm build (are you sure it is specifically a MiniMax VAE issue)?

  2. farawayso commented on Aug 22, 2026

    @farawayso
    Author

    Did you manage to run any other model with the same ROCm build (are you sure it is specifically a MiniMax VAE issue)?

    @stduhpf Yes — I ran other models on the exact same ROCm build (commit 97d2990, the pre-built Windows ROCm binary), and the crash is specific to the MiniMax-H3 video VAE's encode graph, not a general problem with this ROCm build or with VAE encode on HIP:

    Model Mode VAE encode (ROCm0, same build)
    Z-Image Turbo (z-image ae.safetensors) txt2img not exercised (no encode needed) — but full pipeline + VAE decode works
    Boogu Image Edit (img2img, z-image ae.safetensors) img2img 512x512, untiled ✅ works — encode 3.20s, full run OK
    Boogu Image Edit, forced --vae-tiling img2img 1280x1280, 9 tiles ✅ works — encode 3.08s (tiled encode fine on HIP!)
    MiniMax-H3 T2V/T2VA (minimax_h3_video_vae_fp16) video, no encode ✅ works incl. VAE decode (same weights, entire pipeline OK)
    MiniMax-H3 I2VA/FL2VA (same VAE) video + input image, 10 tiles (2x5, overlap 0.125/0.40625) ❌ 0xC0000005, deterministic, at first graph compute

    So: VAE encode in general (including tiled encode with the 2D AE) runs fine on this HIP build on the same GPU. The access violation only happens when computing the MiniMax-H3 video VAE encoder graph — a 3D/temporal VAE (24-channel latent, temporal clip 17, 562 tensors) whose tiled encode path differs from the standard 2D AE.

    One more data point that suggests this is a HIP-kernel-set issue rather than a tiling logic issue: the identical tiled encode of the same MiniMax VAE completes cleanly on Vulkan (0.62s), and the MiniMax VAE decoder (different op set from the encoder, same weights) runs fine on ROCm. The crash happens the instant the encoder graph is computed (118 tensors staged → 627MB buffer allocated → process dies), with no HIP runtime error — consistent with an out-of-bounds access inside a HIP kernel used only by the encoder graph (possibly the 3D conv/layout ops).

    If it helps narrow it down, I can also reproduce with a minimal input (single tile, no overlap) or grab a verbose graph dump — just tell me what would be most useful. For now I've verified the report is not a build-level issue: the same binary+backend handles VAE encode/tiled-encode fine for other models.

  3. stduhpf commented on Aug 22, 2026

    @stduhpf
    Contributor

    That's helpful thanks. Minimax VAE encode works fine on my local HIP builds so that's strange. I don't think the prebuilt releases are compatibe with my hardware so I can't test it myself.

  4. farawayso commented on Oct 1, 2026

    @farawayso
    Author

    I've identified the root cause of this issue. sd.cpp was by default loading an outdated amdhip64_7.dll from Windows System32 — a leftover from a previous AMD driver installation. The newer version of amdhip64_7.dll has actually been moved to a different path, but the old copy in System32 was never cleaned up. As a result, sd.cpp kept picking up the stale DLL instead of the correct one.

    AMD engineers have confirmed and reproduced this issue, and we believe AMD will follow up and fix it soon. As a temporary workaround, you can copy the amdhip64_7.dll from the latest driver into the same directory as sd.exe.

  5. changed the title [-][Bug] MiniMax-H3 I2VA/FL2VA: ROCm VAE encode crashes with 0xC0000005 (ACCESS_VIOLATION) on Windows[/-] [+] MiniMax-H3 I2VA/FL2VA: ROCm VAE encode crashes with 0xC0000005 (ACCESS_VIOLATION) on Windows(问题已解决)[/+] on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions