Skip to content

CUDA: add implicit conv3d - #16948

Closed
bssrdf wants to merge 19 commits into
ggml-org:masterfrom
bssrdf:conv3d-implicit
Closed

bssrdf wants to merge 19 commits into
ggml-org:masterfrom
bssrdf:conv3d-implicit

Conversation

@bssrdf

@bssrdf bssrdf commented Nov 2, 2025 •

Copy link
Copy Markdown
Contributor

This PR adds an implicit conv3d op in CUDA backend, as a complement to IM2COL_3D+GEMM kernel currently used in SD.cpp for video models. It pretty much follows conv2d_implicit.

Using tensor cores, this PR outperforms IM2COL_3D+GEMM. The memory saving is significant, as expected.

ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
(IC, OC, IW, IH, ID, KW, KH, KD) im2col_3D+GEMM TIME im2col_3D+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM pytorch/cudnn TIME
(320, 1280, 26, 38, 8, 3, 3, 3) 1.81 ms 168.85 MB 1.53 ms 38.59 MB 1.42 ms
(1280, 1280, 26, 38, 8, 3, 3, 3) 6.71 ms 559.61 MB 6.01 ms 38.59 MB 5.68 ms
(320, 1280, 52, 76, 8, 3, 3, 3) 7.39 ms 675.39 MB 3.86 ms 154.38 MB 5.64 ms
(1280, 1280, 52, 76, 8, 3, 3, 3) 26.40 ms 2238.44 MB 15.25 ms 154.38 MB 21.62 ms
(320, 1280, 104, 152, 8, 3, 3, 3) 29.36 ms 2701.56 MB 15.42 ms 617.50 MB 22.02 ms
(1280, 1280, 104, 152, 8, 3, 3, 3) 105.70 ms 8953.75 MB 60.83 ms 617.50 MB 82.19 ms
(320, 1280, 208, 304, 4, 3, 3, 3) 59.02 ms 5403.12 MB 30.06 ms 1235.00 MB 43.78 ms
(640, 1280, 208, 304, 4, 3, 3, 3) 109.66 ms 9571.25 MB 60.04 ms 1235.00 MB 83.66 ms

@bssrdf
bssrdf marked this pull request as draft November 2, 2025 17:38
@Green-Sky

Copy link
Copy Markdown
Collaborator

Quick n dirty test perf run:

$ result/bin/test-conv3d
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2070, compute capability 7.5, VMM: yes
(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 1280, 26, 38, 8, 3, 3, 3) 10.30 ms 168.85 MB 34.95 ms 38.59 MB
(1280, 1280, 26, 38, 8, 3, 3, 3) 37.30 ms 559.61 MB 141.30 ms 38.59 MB
(320, 1280, 52, 76, 8, 3, 3, 3) 37.85 ms 675.39 MB 137.66 ms 154.38 MB
(1280, 1280, 52, 76, 8, 3, 3, 3) 156.93 ms 2238.44 MB 558.15 ms 154.38 MB
(320, 1280, 104, 152, 8, 3, 3, 3) 157.53 ms 2701.56 MB 569.28 ms 617.50 MB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 8953.75 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 9388687360
Segmentation fault (core dumped)

@github-actions github-actions Bot added testing Everything test related Nvidia GPU Issues specific to Nvidia GPUs ggml changes relating to the ggml tensor library for machine learning labels Nov 2, 2025
@bssrdf

bssrdf commented Nov 2, 2025

Copy link
Copy Markdown
Contributor Author

Quick n dirty test perf run:

$ result/bin/test-conv3d
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2070, compute capability 7.5, VMM: yes

(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 1280, 26, 38, 8, 3, 3, 3) 10.30 ms 168.85 MB 34.95 ms 38.59 MB
(1280, 1280, 26, 38, 8, 3, 3, 3) 37.30 ms 559.61 MB 141.30 ms 38.59 MB
(320, 1280, 52, 76, 8, 3, 3, 3) 37.85 ms 675.39 MB 137.66 ms 154.38 MB
(1280, 1280, 52, 76, 8, 3, 3, 3) 156.93 ms 2238.44 MB 558.15 ms 154.38 MB
(320, 1280, 104, 152, 8, 3, 3, 3) 157.53 ms 2701.56 MB 569.28 ms 617.50 MB

ggml_backend_cuda_buffer_type_alloc_buffer: allocating 8953.75 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 9388687360
Segmentation fault (core dumped)

Need to get tensors working:)

@Green-Sky

Copy link
Copy Markdown
Collaborator

With that last commit I am catching an illegal memory access.

[DEBUG] ggml_extend.hpp:1588 - wan_vae compute buffer size: 715.16 MB(VRAM)
[ERROR] ggml_extend.hpp:75   - ggml_cuda_compute_forward: IM2COL failed
[ERROR] ggml_extend.hpp:75   - CUDA error: an illegal memory access was encountered
[ERROR] ggml_extend.hpp:75   -   current device: 0, in function ggml_cuda_compute_forward at /build/fr4p6mskfb7pv7d6n87kfpipj5dr1bql-source/ggml/src/ggml-cuda/ggml-cuda.cu:2544
[ERROR] ggml_extend.hpp:75   -   err
/build/fr4p6mskfb7pv7d6n87kfpipj5dr1bql-source/ggml/src/ggml-cuda/ggml-cuda.cu:89: CUDA error

@bssrdf

bssrdf commented Nov 2, 2025 •

Copy link
Copy Markdown
Contributor Author

With that last commit I am catching an illegal memory access.

[DEBUG] ggml_extend.hpp:1588 - wan_vae compute buffer size: 715.16 MB(VRAM)
[ERROR] ggml_extend.hpp:75   - ggml_cuda_compute_forward: IM2COL failed
[ERROR] ggml_extend.hpp:75   - CUDA error: an illegal memory access was encountered
[ERROR] ggml_extend.hpp:75   -   current device: 0, in function ggml_cuda_compute_forward at /build/fr4p6mskfb7pv7d6n87kfpipj5dr1bql-source/ggml/src/ggml-cuda/ggml-cuda.cu:2544
[ERROR] ggml_extend.hpp:75   -   err
/build/fr4p6mskfb7pv7d6n87kfpipj5dr1bql-source/ggml/src/ggml-cuda/ggml-cuda.cu:89: CUDA error

Give it another try. Tensor core path should be on and working.

ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 1280, 26, 38, 8, 3, 3, 3) 1.88 ms 168.85 MB 1.54 ms 38.59 MB
(1280, 1280, 26, 38, 8, 3, 3, 3) 6.76 ms 559.61 MB 6.18 ms 38.59 MB
(320, 1280, 52, 76, 8, 3, 3, 3) 7.58 ms 675.39 MB 3.99 ms 154.38 MB
(1280, 1280, 52, 76, 8, 3, 3, 3) 26.99 ms 2238.44 MB 15.51 ms 154.38 MB
(320, 1280, 104, 152, 8, 3, 3, 3) 30.06 ms 2701.56 MB 15.86 ms 617.50 MB
(1280, 1280, 104, 152, 8, 3, 3, 3) 108.33 ms 8953.75 MB 62.23 ms 617.50 MB
(320, 1280, 208, 304, 4, 3, 3, 3) 60.33 ms 5403.12 MB 30.79 ms 1235.00 MB
(640, 1280, 208, 304, 4, 3, 3, 3) 112.34 ms 9571.25 MB 61.49 ms 1235.00 MB

@Green-Sky

Copy link
Copy Markdown
Collaborator

Looks good!

$ result/bin/test-conv3d
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2070, compute capability 7.5, VMM: yes
(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 1280, 26, 38, 8, 3, 3, 3) 11.35 ms 168.85 MB 6.15 ms 38.59 MB
(1280, 1280, 26, 38, 8, 3, 3, 3) 39.40 ms 559.61 MB 24.59 ms 38.59 MB
(320, 1280, 52, 76, 8, 3, 3, 3) 40.52 ms 675.39 MB 23.23 ms 154.38 MB
(1280, 1280, 52, 76, 8, 3, 3, 3) 166.10 ms 2238.44 MB 91.93 ms 154.38 MB
(320, 1280, 104, 152, 8, 3, 3, 3) 164.08 ms 2701.56 MB 91.41 ms 617.50 MB
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 8953.75 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 9388687360
Segmentation fault (core dumped)

@jeffbolznv

Copy link
Copy Markdown
Contributor

Please add the perf tests to make_test_cases_perf in test-backend-ops.cpp.

@bssrdf

bssrdf commented Nov 2, 2025 •

Copy link
Copy Markdown
Contributor Author

Looks good!

$ result/bin/test-conv3d
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2070, compute capability 7.5, VMM: yes

(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 1280, 26, 38, 8, 3, 3, 3) 11.35 ms 168.85 MB 6.15 ms 38.59 MB
(1280, 1280, 26, 38, 8, 3, 3, 3) 39.40 ms 559.61 MB 24.59 ms 38.59 MB
(320, 1280, 52, 76, 8, 3, 3, 3) 40.52 ms 675.39 MB 23.23 ms 154.38 MB
(1280, 1280, 52, 76, 8, 3, 3, 3) 166.10 ms 2238.44 MB 91.93 ms 154.38 MB
(320, 1280, 104, 152, 8, 3, 3, 3) 164.08 ms 2701.56 MB 91.41 ms 617.50 MB

ggml_backend_cuda_buffer_type_alloc_buffer: allocating 8953.75 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 9388687360
Segmentation fault (core dumped)

Please add the perf tests to make_test_cases_perf in test-backend-ops.cpp.

Will do. Right now, test-backend-op test -o CONV_3D failed to pass.

@Green-Sky , does the WAN video generation work ok?

@Green-Sky

Copy link
Copy Markdown
Collaborator

@Green-Sky , does the WAN video generation work ok?

As far as I can tell yes. I have only tested Wan2.2-TI2V-5B yet (wan2.2 vae).

Single "frame" (aka image gen using wan):
output

I don't have the im2col gemm image ready, but it looked identical. Same for 5 frames.

I was unable to run higher frame counts due to my 8gig vram limit. It should technically just fit if I restart and don't open extra programs. Not gonna do that today.

@Green-Sky

Copy link
Copy Markdown
Collaborator

@Green-Sky , does the WAN video generation work ok?

I tried a smaller resolution and I did get another illegal memory access:

[DEBUG] ggml_extend.hpp:1588 - wan_vae compute buffer size: 5393.97 MB(VRAM)
[ERROR] ggml_extend.hpp:75   - ggml_cuda_compute_forward: CONCAT failed
[ERROR] ggml_extend.hpp:75   - CUDA error: an illegal memory access was encountered
[ERROR] ggml_extend.hpp:75   -   current device: 0, in function ggml_cuda_compute_forward at /build/jj5n0s93mbksd9wmv6vrs8dxmhcvhnik-source/ggml/src/ggml-cuda/ggml-cuda.cu:2544
[ERROR] ggml_extend.hpp:75   -   err

@bssrdf

bssrdf commented Nov 3, 2025

Copy link
Copy Markdown
Contributor Author

@Green-Sky , does the WAN video generation work ok?

I tried a smaller resolution and I did get another illegal memory access:

[DEBUG] ggml_extend.hpp:1588 - wan_vae compute buffer size: 5393.97 MB(VRAM)
[ERROR] ggml_extend.hpp:75   - ggml_cuda_compute_forward: CONCAT failed
[ERROR] ggml_extend.hpp:75   - CUDA error: an illegal memory access was encountered
[ERROR] ggml_extend.hpp:75   -   current device: 0, in function ggml_cuda_compute_forward at /build/jj5n0s93mbksd9wmv6vrs8dxmhcvhnik-source/ggml/src/ggml-cuda/ggml-cuda.cu:2544
[ERROR] ggml_extend.hpp:75   -   err

@Green-Sky, can you retry my latest commit? I see your test failed at CONCAT op. That's weird. Could you also run this case with current main branch? Thanks.

@bssrdf
bssrdf marked this pull request as ready for review November 3, 2025 14:00
@Green-Sky

Green-Sky commented Nov 3, 2025 •

Copy link
Copy Markdown
Collaborator

@leejet I suspect sd.cpp is doing something wrong here, because if width and height are multiples of 32, decoding works. But if they are just multiples of 16 I get that illegal memory access error.

But it is still possible that it is an error with this pr.

@Green-Sky

Copy link
Copy Markdown
Collaborator
$ result/bin/sd -M vid_gen --diffusion-model models/wan/Wan2.2-TI2V-5B-Q8_0.gguf --vae models/wan/Wan2.2_VAE.safetensors --t5xxl models/wan/umt5-xxl-encoder-Q8_0.gguf -p "a lovely cat strolling down a wooden plank" --cfg-scale 6 --sampling-method euler -v -n "色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走" -W 576 -H 352 --diffusion-fa --offload-to-cpu --video-frames 33 --flow-shift 3.0 --steps 48
wan2.2-TI2V-5B-Q8_0-cat-strolling_funkysize.mp4

@bssrdf

bssrdf commented Nov 9, 2025

Copy link
Copy Markdown
Contributor Author

@leejet I suspect sd.cpp is doing something wrong here, because if width and height are multiples of 32, decoding works. But if they are just multiples of 16 I get that illegal memory access error.

But it is still possible that it is an error with this pr.

@Green-Sky, I just fixed a bug which may be related to your crash. Please give it a try. Thanks,

@Green-Sky

Copy link
Copy Markdown
Collaborator

@leejet I suspect sd.cpp is doing something wrong here, because if width and height are multiples of 32, decoding works. But if they are just multiples of 16 I get that illegal memory access error.
But it is still possible that it is an error with this pr.

@Green-Sky, I just fixed a bug which may be related to your crash. Please give it a try. Thanks,

Yes, I can indeed now use multiples of 16, that are not multiples of 32. 🚀

wan2.2-TI2V-5B-Q8_0-cat-strolling_funkysize_16.mp4

@Green-Sky

Copy link
Copy Markdown
Collaborator

I switched to the fastwan 2.2 TI2V 5B model and generated using 6 steps the most frames+res I could fit.

576x352 93frames

fastwan2.2-t2i5b_cat1-steps6.mp4

... in 56 seconds total. On CPU this would take probably over 1 hour.

@bssrdf

bssrdf commented Nov 10, 2025

Copy link
Copy Markdown
Contributor Author

I switched to the fastwan 2.2 TI2V 5B model and generated using 6 steps the most frames+res I could fit.

576x352 93frames

fastwan2.2-t2i5b_cat1-steps6.mp4
... in 56 seconds total. On CPU this would take probably over 1 hour.

@Green-Sky , can you compare the performance IM2COL_3D vs IMPLICIT GEMM? Thanks.

@Green-Sky

Copy link
Copy Markdown
Collaborator

@Green-Sky , can you compare the performance IM2COL_3D vs IMPLICIT GEMM? Thanks.

With sd.cpp that is rather impractical for me, I dont have 20Gigs of vram 😄. I can only run a limited selection of the tests that you provide.

@bssrdf

bssrdf commented Nov 10, 2025

Copy link
Copy Markdown
Contributor Author

@Green-Sky , can you compare the performance IM2COL_3D vs IMPLICIT GEMM? Thanks.

With sd.cpp that is rather impractical for me, I dont have 20Gigs of vram 😄. I can only run a limited selection of the tests that you provide.

Sure. You have done a lot of great jobs testing these PRs in sd.cpp. Many thanks.

@Green-Sky

Copy link
Copy Markdown
Collaborator

@Green-Sky , can you compare the performance IM2COL_3D vs IMPLICIT GEMM? Thanks.

With sd.cpp that is rather impractical for me, I dont have 20Gigs of vram 😄. I can only run a limited selection of the tests that you provide.

Sure. You have done a lot of great jobs testing these PRs in sd.cpp. Many thanks.

I will redo this one later #16948 (comment) , but as you can see it did not complete it.

:)

@Green-Sky

Green-Sky commented Nov 10, 2025 •

Copy link
Copy Markdown
Collaborator

Was experimenting more with wan and got another of those illegal access errors in CONCAT. But I am not latest, so I will check latest.

-W 608 -H 368 --diffusion-fa --offload-to-cpu --video-frames 33 --flow-shift 3.5 --steps 6 (among others)
sample 38x23x9

edit: It has indeed been fixed somewhere in the last few commits: :)

@Green-Sky

Green-Sky commented Nov 11, 2025 •

Copy link
Copy Markdown
Collaborator

You changed the test selection again :P

Anyway, here is that one test:

$ result/bin/test-conv3d
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2070, compute capability 7.5, VMM: yes
(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 320, 104, 158, 8, 3, 3, 3) 110.58 ms 2326.80 MB 34.35 ms 160.47 MB

edit: Can you keep them enabled, and not commit your last selection? Also, would be really cool if they where sorted a bit by vram use by im2col+gemm. That would really help bench this on more hardware.

@bssrdf

bssrdf commented Nov 11, 2025

Copy link
Copy Markdown
Contributor Author

You changed the test selection again :P

Anyway, here is that one test:

$ result/bin/test-conv3d
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2070, compute capability 7.5, VMM: yes

(IC, OC, IW, IH, ID, KW, KH, KD) im2col+GEMM TIME im2col+GEMM VRAM implicit GEMM TIME implicit GEMM VRAM
(320, 320, 104, 158, 8, 3, 3, 3) 110.58 ms 2326.80 MB 34.35 ms 160.47 MB
edit: Can you keep them enabled, and not commit your last selection? Also, would be really cool if they where sorted a bit by vram use by im2col+gemm. That would really help bench this on more hardware.

Sorry, I am just adding more test cases (obtained from sd.cpp wan generation). I want to test these real world cases instead of makeup ones. But in my commit I am going to turn most off because CI will time out otherwise. You can always easily turn them on by uncommenting them.

@CarlGao4

CarlGao4 commented Dec 9, 2025

Copy link
Copy Markdown

This PR hasn't been updated for a month. Any news?

@Green-Sky

Copy link
Copy Markdown
Collaborator

It has been working great for me.
So all green here. :)

@CarlGao4

Copy link
Copy Markdown

Can you fork latest sd.cpp with this patch? I can't build it successfully

@bssrdf

bssrdf commented Dec 12, 2025 •

Copy link
Copy Markdown
Contributor Author

Can you fork latest sd.cpp with this patch? I can't build it successfully

To make it work in SD, you can just copy the two new files(conv3d-implicit.cu and conv3d-implicit.cuh) into ggml-cuda/ and change a few lines in ggml-cuda.cu. That's it.

@Green-Sky

Copy link
Copy Markdown
Collaborator

Can you fork latest sd.cpp with this patch? I can't build it successfully

To make it work in SD, you can just copy the two new files(conv3d-implicit.cu and conv3d-implicit.cuh) into ggml-cuda/ and change a few lines in ggml-cuda.cu. That's it.

With a simple sd.cpp modification I point out here: leejet/stable-diffusion.cpp#872 (comment)
Otherwise it will not be used. (:

@CarlGao4

Copy link
Copy Markdown

I tried these steps before, but finally it telled me something that op not found (I've deleted the files so I can't tell you the exact output)

@bssrdf

bssrdf commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

Close this.

@bssrdf bssrdf closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Nvidia GPU Issues specific to Nvidia GPUs testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants