img2img 'weakened' by diffusion-fa

@Green-Sky

Since ab835f7 , img2img generations need higher strength values to apply changes to the input images.

This is an SDXL test on Vulkan, with the prompt ./sd -m ./cyberrealisticPony_semiRealV35a.safetensors -s 1 -p 'silver car' -i ./base.png , leaving the strength at the default (0.75) for the first gens:

base image	`1896b28`	`1896b28` --diffusion-fa	`ab835f7`	`ab835f7` --diffusion-fa	`ab835f7` --diffusion-fa --strength 0.85

I first noticed this issue on current master 5900ef6 . All binaries built with current ggml 7dee1d6 .

@Green-Sky

Hmm, interesting. Maybe we should really just remove the padding changes again.
#736

The same test on 5900ef6 with:

diff --git a/ggml_extend.hpp b/ggml_extend.hpp
index 22dd88c..f408b79 100644
--- a/ggml_extend.hpp
+++ b/ggml_extend.hpp
@@ -880,7 +880,7 @@ __STATIC_INLINE__ struct ggml_tensor* ggml_nn_attention_ext(struct ggml_context*
 #else
     if (can_use_flash_attn && L_k % 256 != 0) {
         // TODO(Green-Sky): might be worth just padding by default
-        if (L_k == 77 || L_k == 4208 || L_k == 3952) {
+        if (L_k == 4208 || L_k == 3952) {
             kv_pad = GGML_PAD(L_k, 256) - L_k;
         } else {
             can_use_flash_attn = false;

The same test on 5900ef6 with:

diff --git a/ggml_extend.hpp b/ggml_extend.hpp
index 22dd88c..f408b79 100644
--- a/ggml_extend.hpp
+++ b/ggml_extend.hpp
@@ -880,7 +880,7 @@ __STATIC_INLINE__ struct ggml_tensor* ggml_nn_attention_ext(struct ggml_context*
 #else
     if (can_use_flash_attn && L_k % 256 != 0) {
         // TODO(Green-Sky): might be worth just padding by default
-        if (L_k == 77 || L_k == 4208 || L_k == 3952) {
+        if (L_k == 4208 || L_k == 3952) {
             kv_pad = GGML_PAD(L_k, 256) - L_k;
         } else {
             can_use_flash_attn = false;

Yeah that just disables the most common padding case.
The problem with PAD is that it add zeros, and if a zero is not the neutral value, calculations will go off.

Ah, I see. So that's essentially equivalent to switching the #if 0 right above.

WIP: update stable-diffusion.cpp to 5900ef6605c6 (new API) (#1669)

revert the L_k padding, introduced as part of leejet#736

fix: adjust flash attention scale for kv padding

For completeness sake, with the change I've mentioned on #774 (4c8b774), I get:

@wbruna

@wbruna is this still an issue?

Oh, thanks for reminding me of this one. It was fixed by #778 .

Navigation Menu

Repository navigation

img2img 'weakened' by diffusion-fa #756

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

img2img 'weakened' by diffusion-fa #756

Description

Activity

Green-Sky commented on Aug 6, 2025

wbruna commented on Aug 9, 2025

Green-Sky commented on Aug 9, 2025

wbruna commented on Aug 9, 2025

wbruna commented on Aug 29, 2025

Green-Sky commented on Sep 9, 2025

wbruna commented on Sep 9, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions