Repository navigation
[Vulkan] Allow assert-only SymBool chains in the partitioner - #23545
Merged
Merged
Conversation
Summary: ## Ulterior Motive Restore the "Test WebGPU Native (Dawn)" workflow on `main`, which has failed on every commit since "[Vulkan] Keep unrepresentable and symbolic scalars out of the delegate (#23247)". ## Rationale **What**: The Vulkan partitioner no longer applies its SymBool/SymFloat rejection to assert ops (`_assert_scalar`, `sym_constrain_range_for_size`) or to nodes whose only users are assert ops. **Why**: #23247 rejects every node that produces or consumes a SymBool. Models that guard an `.item()` value with `torch._check` produce `u <= N` / `u >= 0` comparisons feeding `_assert_scalar`. Rejecting them cascades through the new "symbolic scalar has an unsupported consumer" check to the `_local_scalar_dense` producer, splitting the graph into multiple delegates. `test_rope_hf.py` then fails with `AssertionError: expected one delegate, got 2`. Asserts are stripped by `RemoveAssertsTransform` during preprocessing, so these SymBool values never need a runtime representation. ## Details - Assert-only SymBool chains - **Before**: Comparisons and asserts rejected; `.item()` producer rejected as a consequence; graph split into multiple delegates. - **After**: Delegated, then removed during preprocessing; graph stays in one delegate. - SymBool/SymFloat values feeding non-assert ops - **Before**: Rejected. - **After**: Unchanged (still rejected). Test Plan: ## Observed Results | Validation | Device | Result | |---|---|---| | Vulkan partition count for `DynamicHfRope` (`test_rope_hf.py`), parent commit | macOS (Apple Silicon) host, partitioner only | 2 partitions; skip logs match the CI failure | | Vulkan partition count for `DynamicHfRope`, this commit | macOS (Apple Silicon) host, partitioner only | 1 partition, no skipped nodes | | `test_rope_hf.py` end-to-end ("Test WebGPU Native (Dawn)") | — | Not run: needs the Dawn/SwiftShader runtime; relying on CI | | `backends/vulkan/test/test_vulkan_dynamic.py` | — | Not run: needs the Vulkan runtime; relying on CI | | Lint | — | Not run: lint tooling not installed locally; relying on CI | ## Reproduction Guide ### Setup ```bash git clone https://github.com/pytorch/executorch.git cd executorch ./install_executorch.sh ``` ### Count Vulkan partitions for the dynamic HF RoPE model ```bash cat > /tmp/count_partitions.py <<'PY' import torch from executorch.exir import to_edge from executorch.backends.vulkan.partitioner.vulkan_partitioner import VulkanPartitioner from executorch.backends.webgpu.test.ops import test_rope_hf as t inputs = t._dynamic_inputs() with torch._dynamo.config.patch(capture_scalar_outputs=True): ep = torch.export.export(t.DynamicHfRope().eval(), inputs) res = VulkanPartitioner().partition(to_edge(ep).exported_program()) tags = { n.meta["delegation_tag"] for n in res.tagged_exported_program.graph.nodes if n.meta.get("delegation_tag") } print("partitions:", len(tags)) PY python /tmp/count_partitions.py 2>&1 | grep -E "Vulkan Partitioner|partitions:" ``` Run once on this commit (expect `partitions: 1`) and once on its parent (`git checkout HEAD~1`, expect `partitions: 2` plus "symbolic float or bool values are not supported" skip logs). Reviewers: This PR was authored with Claude Code.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23545
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 115aac7 with merge base 9cced90 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
shoumikhin
approved these changes
Oct 7, 2026
SS-JIA
added a commit
that referenced
this pull request
Oct 8, 2026
Summary: ## Ulterior Motive Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests. ## Rationale **What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader. **Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made that partitioner fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). `DynamicShape.SwiGluReusedGraph` and `DynamicShape.SwiGluCommutativeAndOwnership` failed at `load_forward()`. ## Details - Programs containing SwiGLU - **Before**: Load fails with "unsupported op: et_vk.swiglu.default". - **After**: Lowered to one `silu_mul_fused` dispatch, the same kernel the runtime's load-time SwiGLU fusion already emits. - `add_silu_mul_fused_dispatch` - **Before**: Registered its resize hook on a single tensor (the shared q4 linear input). - **After**: Takes a list of trigger tensors. The load-time fusion passes its shared input; the new op passes both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks. The op requires fp32 operands with identical `gate`, `up`, and output shapes, which is what the flat `silu_mul_fused` kernel supports. Test Plan: ## Observed Results ### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader) | Run | Result | |---|---| | `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) | | This branch via `workflow_dispatch` (run 37718579736) | Passed | Per-binary results for run 37718579736: | Test binary | Passed | Skipped | Failed | |---|---|---|---| | webgpu_native_test | 29 | 3 | 0 | | webgpu_update_cache_test | 3 | 0 | 0 | | webgpu_dispatch_order_test | 9 | 0 | 0 | | webgpu_index_test | 4 | 0 | 0 | | webgpu_dynamic_shape_test | 40 | 4 | 0 | | webgpu_scratch_buffer_test | 8 | 0 | 0 | | webgpu_dispatch_2d_test | 13 | 0 | 0 | | webgpu_compute_dispatch_test | 20 | 0 | 0 | | webgpu_execution_options_test | 12 | 0 | 0 | | webgpu_output_suppression_test | 3 | 0 | 0 | | webgpu_op_test_util_test | 8 | 0 | 0 | | webgpu_op_test (op-test framework) | 391 | 0 | 0 | The 4 skipped dynamic-shape tests are the same ones skipped on `main`. ### Local (macOS, Apple Silicon, Dawn on Metal) | Validation | Result | |---|---| | SwiGLU fixtures, parent commit | All SwiGLU programs fail to load (reproduces CI) | | SwiGLU fixtures, this commit | All load and match goldens except `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) | | `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QuantizedLinearSweep` | Fail on Metal on both parent and this commit; pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 linear path, and passes in CI | | Performance | Not run: the op emits the same single `silu_mul_fused` dispatch as the existing load-time fusion | ## Reproduction Guide ### Run the CI workflow on this branch ```bash gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1 ``` ### Run locally on Linux (mirrors the CI job) ```bash git clone --recursive https://github.com/pytorch/executorch.git cd executorch git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD ./install_executorch.sh source .ci/scripts/setup-webgpu-linux-deps.sh bash backends/webgpu/scripts/test_webgpu_native_ci.sh ``` ### Run only the SwiGLU tests after the script has built and exported fixtures ```bash cmake-out-webgpu-dawn-ci/backends/webgpu/webgpu_dynamic_shape_test /tmp/dynamic_shape \ --gtest_filter='*SwiGlu*' ``` Reviewers: This PR was authored with Claude Code.
SS-JIA
added a commit
that referenced
this pull request
Oct 8, 2026
Summary: ## Ulterior Motive Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests. ## Rationale **What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader. **Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made that partitioner fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). `DynamicShape.SwiGluReusedGraph` and `DynamicShape.SwiGluCommutativeAndOwnership` failed at `load_forward()`. ## Details - Programs containing SwiGLU - **Before**: Load fails with "unsupported op: et_vk.swiglu.default". - **After**: Lowered to one `silu_mul_fused` dispatch, the same kernel the runtime's load-time SwiGLU fusion already emits. - `add_silu_mul_fused_dispatch` - **Before**: Registered its resize hook on a single tensor (the shared q4 linear input). - **After**: Takes a list of trigger tensors. The load-time fusion passes its shared input; the new op passes both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks. The op requires fp32 operands with identical `gate`, `up`, and output shapes, which is what the flat `silu_mul_fused` kernel supports. Test Plan: ## Observed Results ### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader) | Run | Result | |---|---| | `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) | | This branch via `workflow_dispatch` (run 37718579736) | Passed | Per-binary results for run 37718579736: | Test binary | Passed | Skipped | Failed | |---|---|---|---| | webgpu_native_test | 29 | 3 | 0 | | webgpu_update_cache_test | 3 | 0 | 0 | | webgpu_dispatch_order_test | 9 | 0 | 0 | | webgpu_index_test | 4 | 0 | 0 | | webgpu_dynamic_shape_test | 40 | 4 | 0 | | webgpu_scratch_buffer_test | 8 | 0 | 0 | | webgpu_dispatch_2d_test | 13 | 0 | 0 | | webgpu_compute_dispatch_test | 20 | 0 | 0 | | webgpu_execution_options_test | 12 | 0 | 0 | | webgpu_output_suppression_test | 3 | 0 | 0 | | webgpu_op_test_util_test | 8 | 0 | 0 | | webgpu_op_test (op-test framework) | 391 | 0 | 0 | The 4 skipped dynamic-shape tests are the same ones skipped on `main`. ### Local (macOS, Apple Silicon, Dawn on Metal) | Validation | Result | |---|---| | SwiGLU fixtures, parent commit | All SwiGLU programs fail to load (reproduces CI) | | SwiGLU fixtures, this commit | All load and match goldens except `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) | | `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QuantizedLinearSweep` | Fail on Metal on both parent and this commit; pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 linear path, and passes in CI | | Performance | Not run: the op emits the same single `silu_mul_fused` dispatch as the existing load-time fusion | ## Reproduction Guide ### Run the CI workflow on this branch ```bash gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1 ``` ### Run locally on Linux (mirrors the CI job) ```bash git clone --recursive https://github.com/pytorch/executorch.git cd executorch git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD ./install_executorch.sh source .ci/scripts/setup-webgpu-linux-deps.sh bash backends/webgpu/scripts/test_webgpu_native_ci.sh ``` ### Run only the SwiGLU tests after the script has built and exported fixtures ```bash cmake-out-webgpu-dawn-ci/backends/webgpu/webgpu_dynamic_shape_test /tmp/dynamic_shape \ --gtest_filter='*SwiGlu*' ``` Reviewers: This PR was authored with Claude Code.
SS-JIA
added a commit
that referenced
this pull request
Oct 8, 2026
Summary: ## Ulterior Motive Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests. ## Rationale **What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader, and delete the runtime SwiGLU graph fusion it replaces. **Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made Vulkan preprocessing fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). With fusion now done at export time, the runtime pattern matcher can no longer match: everything it accepts (fp32, q4 gate/up producers, gate used only by the pattern, equal shapes) is a strict subset of what the export-time pattern fuses. Instrumenting it on the current fixtures confirmed zero matches across all 89 graph builds in `webgpu_dynamic_shape_test`. ## Details - Programs containing SwiGLU - **Before**: Load fails with "unsupported op: et_vk.swiglu.default". - **After**: Lowered to one `silu_mul_fused` dispatch, with the resize hook registered on both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks. - Runtime SwiGLU graph fusion (`passes/SwiGLU.{h,cpp}`) - **Before**: Matched q4 gate/up + `sigmoid` + 2x `mul` at load time, emitted `silu_mul_fused`, and placed `gate` in a scratch buffer. - **After**: Deleted. The dispatch code moved into `ops/mul/SwiGLU.cpp`. - QKV-BK64 vs. SwiGLU precedence - **Before**: The runtime SwiGLU pass claimed the gate/up linears, and `retain_unclaimed_qkv_fusions` dropped any QKV-BK64 group overlapping them. - **After**: Removed. Since #22979 the elementwise fusion no longer owns the linears, so both fusions apply: q/gate/up share one `linear_q4gsw_bk64_qkv` dispatch followed by `silu_mul_fused`. QKV candidates are grouped by input tensor and cannot overlap each other, so `retain_unclaimed_qkv_fusions` had no other effect. - Profiling-only tests (`SwiGluFusionProfile`, `SwiGluQkvOverlapProfile`) - **Before**: Encoded the runtime matcher's rules: no fusion when `gate` has an extra consumer, is a graph output, or when `up` comes from a separate input, and no QKV-BK64 dispatch in the overlap case. These tests only build with `EXECUTORCH_BUILD_WEBGPU_PROFILING`, so CI does not run them and they had gone stale since #22979. - **After**: Expect fusion in those three cases (`gate` stays materialized, so fusing is valid) and one `linear_q4gsw_bk64_qkv` dispatch in the overlap case. Cases where `sigmoid` or `silu` has another user still expect no fusion. Test Plan: ## Observed Results ### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader) | Run | Result | |---|---| | `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) | | Op only, before removing the runtime fusion (run 37718579736) | Passed (all 11 native test binaries; `webgpu_dynamic_shape_test` 40 passed / 4 skipped, same skips as `main`; op-test framework 391 passed) | | This commit, rebased on "Vulkan: narrow the int4 linear output tile on Mali" | Pending | ### Local (macOS, Apple Silicon, Dawn on Metal, `EXECUTORCH_BUILD_WEBGPU_PROFILING=ON`, `WEBGPU_TIMESTAMP_QUERY=1`) | Validation | Result | |---|---| | Runtime SwiGLU matcher hit count on current fixtures (temporary instrumentation, before deletion) | 0 matches in 89 graph builds | | `SwiGluReusedGraph`, `SwiGluFusionProfile` | Pass | | `SwiGluQkvOverlapProfile` dispatch counts (1 `linear_q4gsw_bk64_qkv`, 1 `silu_mul_fused`, 0 `mul`) | Pass | | `SwiGluCommutativeAndOwnership`, `SwiGluQkvOverlapProfile` numerics | Fail only on `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) | | `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QkvBk64LiveRoutesProfile`, `QuantizedLinearBk64QkvProfileSoleWriter` | Fail on Metal with and without this change; BK64 tests pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 path and passed in CI run 37718579736 | | Performance | Not run. New exports lose only the `gate` scratch-buffer reuse; programs exported before #22979 lose the load-time fusion and run `sigmoid`/`mul`/`mul` as three dispatches | ## Reproduction Guide ### Run the CI workflow on this branch ```bash gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1 ``` ### Run locally on Linux (mirrors the CI job) ```bash git clone --recursive https://github.com/pytorch/executorch.git cd executorch git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD ./install_executorch.sh source .ci/scripts/setup-webgpu-linux-deps.sh bash backends/webgpu/scripts/test_webgpu_native_ci.sh ``` ### Run the profiling-only SwiGLU tests After the script above has exported fixtures to `/tmp/dynamic_shape`: ```bash cmake -DPYTHON_EXECUTABLE=python \ -DEXECUTORCH_BUILD_WEBGPU=ON -DEXECUTORCH_BUILD_WEBGPU_TEST=ON \ -DEXECUTORCH_BUILD_WEBGPU_PROFILING=ON -DEXECUTORCH_BUILD_TESTS=ON \ -DDawn_DIR="${Dawn_DIR}" \ -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \ -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON -DCMAKE_BUILD_TYPE=Release \ -B cmake-out-webgpu-prof . cmake --build cmake-out-webgpu-prof --target webgpu_dynamic_shape_test WEBGPU_TIMESTAMP_QUERY=1 cmake-out-webgpu-prof/backends/webgpu/webgpu_dynamic_shape_test \ /tmp/dynamic_shape --gtest_filter='*SwiGlu*' ``` Reviewers: This PR was authored with Claude Code.
SS-JIA
added a commit
that referenced
this pull request
Oct 8, 2026
Summary: ## Ulterior Motive Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests. ## Rationale **What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader, and delete the runtime SwiGLU graph fusion it replaces. **Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made Vulkan preprocessing fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). With fusion now done at export time, the runtime pattern matcher can no longer match: everything it accepts (fp32, q4 gate/up producers, gate used only by the pattern, equal shapes) is a strict subset of what the export-time pattern fuses. Instrumenting it on the current fixtures confirmed zero matches across all 89 graph builds in `webgpu_dynamic_shape_test`. ## Details - Programs containing SwiGLU - **Before**: Load fails with "unsupported op: et_vk.swiglu.default". - **After**: Lowered to one `silu_mul_fused` dispatch, with the resize hook registered on both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks. - Runtime SwiGLU graph fusion (`passes/SwiGLU.{h,cpp}`) - **Before**: Matched q4 gate/up + `sigmoid` + 2x `mul` at load time, emitted `silu_mul_fused`, and placed `gate` in a scratch buffer. - **After**: Deleted. The dispatch code moved into `ops/mul/SwiGLU.cpp`. - QKV-BK64 vs. SwiGLU precedence - **Before**: The runtime SwiGLU pass claimed the gate/up linears, and `retain_unclaimed_qkv_fusions` dropped any QKV-BK64 group overlapping them. - **After**: Removed. Since #22979 the elementwise fusion no longer owns the linears, so both fusions apply: q/gate/up share one `linear_q4gsw_bk64_qkv` dispatch followed by `silu_mul_fused`. QKV candidates are grouped by input tensor and cannot overlap each other, so `retain_unclaimed_qkv_fusions` had no other effect. - Profiling-only tests (`SwiGluFusionProfile`, `SwiGluQkvOverlapProfile`) - **Before**: Encoded the runtime matcher's rules: no fusion when `gate` has an extra consumer, is a graph output, or when `up` comes from a separate input, and no QKV-BK64 dispatch in the overlap case. These tests only build with `EXECUTORCH_BUILD_WEBGPU_PROFILING`, so CI does not run them and they had gone stale since #22979. - **After**: Expect fusion in those three cases (`gate` stays materialized, so fusing is valid) and one `linear_q4gsw_bk64_qkv` dispatch in the overlap case. Cases where `sigmoid` or `silu` has another user still expect no fusion. Test Plan: ## Observed Results ### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader) | Run | Result | |---|---| | `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) | | Op only, before removing the runtime fusion (run 37718579736) | Passed (all 11 native test binaries; `webgpu_dynamic_shape_test` 40 passed / 4 skipped, same skips as `main`; op-test framework 391 passed) | | This commit, rebased on "Vulkan: narrow the int4 linear output tile on Mali" (run 37820744165) | Passed | Per-binary results for run 37820744165: | Test binary | Passed | Skipped | Failed | |---|---|---|---| | webgpu_native_test | 29 | 3 | 0 | | webgpu_update_cache_test | 3 | 0 | 0 | | webgpu_dispatch_order_test | 9 | 0 | 0 | | webgpu_index_test | 4 | 0 | 0 | | webgpu_dynamic_shape_test | 40 | 4 | 0 | | webgpu_scratch_buffer_test | 8 | 0 | 0 | | webgpu_dispatch_2d_test | 13 | 0 | 0 | | webgpu_compute_dispatch_test | 20 | 0 | 0 | | webgpu_execution_options_test | 12 | 0 | 0 | | webgpu_output_suppression_test | 3 | 0 | 0 | | webgpu_op_test_util_test | 8 | 0 | 0 | | webgpu_op_test (op-test framework) | 391 | 0 | 0 | The 4 skipped dynamic-shape tests are the same ones skipped on `main`. The profiling-only SwiGLU tests are not built in CI; see the local results below. ### Local (macOS, Apple Silicon, Dawn on Metal, `EXECUTORCH_BUILD_WEBGPU_PROFILING=ON`, `WEBGPU_TIMESTAMP_QUERY=1`) | Validation | Result | |---|---| | Runtime SwiGLU matcher hit count on current fixtures (temporary instrumentation, before deletion) | 0 matches in 89 graph builds | | `SwiGluReusedGraph`, `SwiGluFusionProfile` | Pass | | `SwiGluQkvOverlapProfile` dispatch counts (1 `linear_q4gsw_bk64_qkv`, 1 `silu_mul_fused`, 0 `mul`) | Pass | | `SwiGluCommutativeAndOwnership`, `SwiGluQkvOverlapProfile` numerics | Fail only on `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) | | `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QkvBk64LiveRoutesProfile`, `QuantizedLinearBk64QkvProfileSoleWriter` | Fail on Metal with and without this change; BK64 tests pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 path and passed in CI run 37718579736 | | Performance | Not run. New exports lose only the `gate` scratch-buffer reuse; programs exported before #22979 lose the load-time fusion and run `sigmoid`/`mul`/`mul` as three dispatches | ## Reproduction Guide ### Run the CI workflow on this branch ```bash gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1 ``` ### Run locally on Linux (mirrors the CI job) ```bash git clone --recursive https://github.com/pytorch/executorch.git cd executorch git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD ./install_executorch.sh source .ci/scripts/setup-webgpu-linux-deps.sh bash backends/webgpu/scripts/test_webgpu_native_ci.sh ``` ### Run the profiling-only SwiGLU tests After the script above has exported fixtures to `/tmp/dynamic_shape`: ```bash cmake -DPYTHON_EXECUTABLE=python \ -DEXECUTORCH_BUILD_WEBGPU=ON -DEXECUTORCH_BUILD_WEBGPU_TEST=ON \ -DEXECUTORCH_BUILD_WEBGPU_PROFILING=ON -DEXECUTORCH_BUILD_TESTS=ON \ -DDawn_DIR="${Dawn_DIR}" \ -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \ -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON -DCMAKE_BUILD_TYPE=Release \ -B cmake-out-webgpu-prof . cmake --build cmake-out-webgpu-prof --target webgpu_dynamic_shape_test WEBGPU_TIMESTAMP_QUERY=1 cmake-out-webgpu-prof/backends/webgpu/webgpu_dynamic_shape_test \ /tmp/dynamic_shape --gtest_filter='*SwiGlu*' ``` Reviewers: This PR was authored with Claude Code.
SS-JIA
added a commit
that referenced
this pull request
Oct 8, 2026
Summary: ## Ulterior Motive Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests. ## Rationale **What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader, and delete the runtime SwiGLU graph fusion it replaces. **Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made Vulkan preprocessing fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). With fusion now done at export time, the runtime pattern matcher can no longer match: everything it accepts (fp32, q4 gate/up producers, gate used only by the pattern, equal shapes) is a strict subset of what the export-time pattern fuses. Instrumenting it on the current fixtures confirmed zero matches across all 89 graph builds in `webgpu_dynamic_shape_test`. ## Details - Programs containing SwiGLU - **Before**: Load fails with "unsupported op: et_vk.swiglu.default". - **After**: Lowered to one `silu_mul_fused` dispatch, with the resize hook registered on both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks. - Runtime SwiGLU graph fusion (`passes/SwiGLU.{h,cpp}`) - **Before**: Matched q4 gate/up + `sigmoid` + 2x `mul` at load time, emitted `silu_mul_fused`, and placed `gate` in a scratch buffer. - **After**: Deleted. The dispatch code moved into `ops/mul/SwiGLU.cpp`. - QKV-BK64 vs. SwiGLU precedence - **Before**: The runtime SwiGLU pass claimed the gate/up linears, and `retain_unclaimed_qkv_fusions` dropped any QKV-BK64 group overlapping them. - **After**: Removed. Since #22979 the elementwise fusion no longer owns the linears, so both fusions apply: q/gate/up share one `linear_q4gsw_bk64_qkv` dispatch followed by `silu_mul_fused`. QKV candidates are grouped by input tensor and cannot overlap each other, so `retain_unclaimed_qkv_fusions` had no other effect. - Profiling-only tests (`SwiGluFusionProfile`, `SwiGluQkvOverlapProfile`) - **Before**: Encoded the runtime matcher's rules: no fusion when `gate` has an extra consumer, is a graph output, or when `up` comes from a separate input, and no QKV-BK64 dispatch in the overlap case. These tests only build with `EXECUTORCH_BUILD_WEBGPU_PROFILING`, so CI does not run them and they had gone stale since #22979. - **After**: Expect fusion in those three cases (`gate` stays materialized, so fusing is valid) and one `linear_q4gsw_bk64_qkv` dispatch in the overlap case. Cases where `sigmoid` or `silu` has another user still expect no fusion. Test Plan: ## Observed Results ### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader) | Run | Result | |---|---| | `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) | | Op only, before removing the runtime fusion (run 37718579736) | Passed (all 11 native test binaries; `webgpu_dynamic_shape_test` 40 passed / 4 skipped, same skips as `main`; op-test framework 391 passed) | | This commit, rebased on "Vulkan: narrow the int4 linear output tile on Mali" (run 37820744165) | Passed | Per-binary results for run 37820744165: | Test binary | Passed | Skipped | Failed | |---|---|---|---| | webgpu_native_test | 29 | 3 | 0 | | webgpu_update_cache_test | 3 | 0 | 0 | | webgpu_dispatch_order_test | 9 | 0 | 0 | | webgpu_index_test | 4 | 0 | 0 | | webgpu_dynamic_shape_test | 40 | 4 | 0 | | webgpu_scratch_buffer_test | 8 | 0 | 0 | | webgpu_dispatch_2d_test | 13 | 0 | 0 | | webgpu_compute_dispatch_test | 20 | 0 | 0 | | webgpu_execution_options_test | 12 | 0 | 0 | | webgpu_output_suppression_test | 3 | 0 | 0 | | webgpu_op_test_util_test | 8 | 0 | 0 | | webgpu_op_test (op-test framework) | 391 | 0 | 0 | The 4 skipped dynamic-shape tests are the same ones skipped on `main`. The profiling-only SwiGLU tests are not built in CI; see the local results below. ### Local (macOS, Apple Silicon, Dawn on Metal, `EXECUTORCH_BUILD_WEBGPU_PROFILING=ON`, `WEBGPU_TIMESTAMP_QUERY=1`) | Validation | Result | |---|---| | Runtime SwiGLU matcher hit count on current fixtures (temporary instrumentation, before deletion) | 0 matches in 89 graph builds | | `SwiGluReusedGraph`, `SwiGluFusionProfile` | Pass | | `SwiGluQkvOverlapProfile` dispatch counts (1 `linear_q4gsw_bk64_qkv`, 1 `silu_mul_fused`, 0 `mul`) | Pass | | `SwiGluCommutativeAndOwnership`, `SwiGluQkvOverlapProfile` numerics | Fail only on `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) | | `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QkvBk64LiveRoutesProfile`, `QuantizedLinearBk64QkvProfileSoleWriter` | Fail on Metal with and without this change; BK64 tests pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 path and passed in CI run 37718579736 | | Performance | Not run. New exports lose only the `gate` scratch-buffer reuse; programs exported before #22979 lose the load-time fusion and run `sigmoid`/`mul`/`mul` as three dispatches | ## Reproduction Guide ### Run the CI workflow on this branch ```bash gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1 ``` ### Run locally on Linux (mirrors the CI job) ```bash git clone --recursive https://github.com/pytorch/executorch.git cd executorch git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD ./install_executorch.sh source .ci/scripts/setup-webgpu-linux-deps.sh bash backends/webgpu/scripts/test_webgpu_native_ci.sh ``` ### Run the profiling-only SwiGLU tests After the script above has exported fixtures to `/tmp/dynamic_shape`: ```bash cmake -DPYTHON_EXECUTABLE=python \ -DEXECUTORCH_BUILD_WEBGPU=ON -DEXECUTORCH_BUILD_WEBGPU_TEST=ON \ -DEXECUTORCH_BUILD_WEBGPU_PROFILING=ON -DEXECUTORCH_BUILD_TESTS=ON \ -DDawn_DIR="${Dawn_DIR}" \ -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \ -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \ -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON -DCMAKE_BUILD_TYPE=Release \ -B cmake-out-webgpu-prof . cmake --build cmake-out-webgpu-prof --target webgpu_dynamic_shape_test WEBGPU_TIMESTAMP_QUERY=1 cmake-out-webgpu-prof/backends/webgpu/webgpu_dynamic_shape_test \ /tmp/dynamic_shape --gtest_filter='*SwiGlu*' ``` Reviewers: This PR was authored with Claude Code.
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Ulterior Motive
Restore the "Test WebGPU Native (Dawn)" workflow on
main, which has failed on every commit since "[Vulkan] Keep unrepresentable and symbolic scalars out of the delegate (#23247)".Rationale
What: The Vulkan partitioner no longer applies its SymBool/SymFloat rejection to assert ops (
_assert_scalar,sym_constrain_range_for_size) or to nodes whose only users are assert ops.Why: #23247 rejects every node that produces or consumes a SymBool. Models that guard an
.item()value withtorch._checkproduceu <= N/u >= 0comparisons feeding_assert_scalar. Rejecting them cascades through the new "symbolic scalar has an unsupported consumer" check to the_local_scalar_denseproducer, splitting the graph into multiple delegates.test_rope_hf.pythen fails withAssertionError: expected one delegate, got 2. Asserts are stripped byRemoveAssertsTransformduring preprocessing, so these SymBool values never need a runtime representation.Details
.item()producer rejected as a consequence; graph split into multiple delegates.Test Plan:
Observed Results
DynamicHfRope(test_rope_hf.py), parent commitDynamicHfRope, this committest_rope_hf.pyend-to-end ("Test WebGPU Native (Dawn)")backends/vulkan/test/test_vulkan_dynamic.pyReproduction Guide
Setup
git clone https://github.com/pytorch/executorch.git cd executorch ./install_executorch.shCount Vulkan partitions for the dynamic HF RoPE model
Run once on this commit (expect
partitions: 1) and once on its parent (git checkout HEAD~1, expectpartitions: 2plus "symbolic float or bool values are not supported" skip logs).Reviewers:
This PR was authored with Claude Code.
cc @manuelcandales @digantdesai @cbilgin