Skip to content

Commit 42ce76d

Browse files
committed
sycl: decline float -> PTQ1_0/PQ2_0 copies instead of asserting
Adding the two types to ggml_sycl_is_quantized_type() made supports_op accept f32/f16/bf16 -> PTQ1_0/PQ2_0 copies, but there is no quantize-rows kernel for either, so ggml_sycl_cpy() took the float -> quantized branch and died on GGML_ASSERT(ggml_sycl_can_quantize_rows_sycl(...)) at cpy.cpp:1298/1302/1308. Neither format can be produced by a row-at-a-time GPU kernel: the packing assumes the Hadamard rotation the offline converter applies, so the right answer is to decline the pair and let the scheduler place it elsewhere, not to add a kernel. Quantized -> same-quantized copies are unaffected and still take the memcpy path. Reported by @dwymark on an Arc B390 (Panther Lake Xe3); reproduced on an Arc Pro B50. test -b SYCL0 -o CPY -p "ptq1_0|pq2_0" before: abort at cpy.cpp:1298 after: 18/18 tests passed, float -> ternary reports "not supported" No regression: MUL_MAT 204/204, MUL_MAT_ID 150/150, GET_ROWS 8/8 for the two types. The unrelated tq2_0 -> tq2_0 abort at cpy.cpp:1455 and the SET_ROWS abort at set_rows.cpp:549 are both present on prism and untouched here.
1 parent 450c5c3 commit 42ce76d

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

‎ggml/src/ggml-sycl/ggml-sycl.cpp‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6146,6 +6146,16 @@ static bool do_ggml_backend_sycl_device_supports_op(ggml_backend_dev_t dev, cons
61466146
ggml_type src0_type = op->src[0]->type;
61476147
ggml_type src1_type = op->src[1]->type;
61486148

6149+
// Quantizing a float row into PTQ1_0 or PQ2_0 has no kernel: both are
6150+
// produced offline by the converter, which also applies the Hadamard
6151+
// rotation the packing assumes. ggml_sycl_cpy() would take the
6152+
// float -> quantized branch and assert, so decline the pair here and let
6153+
// the scheduler fall back. The quant -> same-quant copies are handled.
6154+
if ((src1_type == GGML_TYPE_PTQ1_0 || src1_type == GGML_TYPE_PQ2_0) &&
6155+
src0_type != src1_type) {
6156+
return false;
6157+
}
6158+
61496159
if (src0_type == GGML_TYPE_F16) {
61506160
if (src1_type == GGML_TYPE_Q2_K ||
61516161
src1_type == GGML_TYPE_Q3_K ||

0 commit comments

Comments
 (0)