Skip to content

[Vulkan] Allow assert-only SymBool chains in the partitioner - #23545

Merged
SS-JIA merged 1 commit into
mainfrom
vulkan-assert-symbool
Oct 7, 2026
Merged

SS-JIA merged 1 commit into
mainfrom
vulkan-assert-symbool

Conversation

@SS-JIA

@SS-JIA SS-JIA commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary:

Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on main, which has failed on every commit since "[Vulkan] Keep unrepresentable and symbolic scalars out of the delegate (#23247)".

Rationale

What: The Vulkan partitioner no longer applies its SymBool/SymFloat rejection to assert ops (_assert_scalar, sym_constrain_range_for_size) or to nodes whose only users are assert ops.

Why: #23247 rejects every node that produces or consumes a SymBool. Models that guard an .item() value with torch._check produce u <= N / u >= 0 comparisons feeding _assert_scalar. Rejecting them cascades through the new "symbolic scalar has an unsupported consumer" check to the _local_scalar_dense producer, splitting the graph into multiple delegates. test_rope_hf.py then fails with AssertionError: expected one delegate, got 2. Asserts are stripped by RemoveAssertsTransform during preprocessing, so these SymBool values never need a runtime representation.

Details

  • Assert-only SymBool chains
    • Before: Comparisons and asserts rejected; .item() producer rejected as a consequence; graph split into multiple delegates.
    • After: Delegated, then removed during preprocessing; graph stays in one delegate.
  • SymBool/SymFloat values feeding non-assert ops
    • Before: Rejected.
    • After: Unchanged (still rejected).

Test Plan:

Observed Results

Validation Device Result
Vulkan partition count for DynamicHfRope (test_rope_hf.py), parent commit macOS (Apple Silicon) host, partitioner only 2 partitions; skip logs match the CI failure
Vulkan partition count for DynamicHfRope, this commit macOS (Apple Silicon) host, partitioner only 1 partition, no skipped nodes
test_rope_hf.py end-to-end ("Test WebGPU Native (Dawn)") — Not run: needs the Dawn/SwiftShader runtime; relying on CI
backends/vulkan/test/test_vulkan_dynamic.py — Not run: needs the Vulkan runtime; relying on CI
Lint — Not run: lint tooling not installed locally; relying on CI

Reproduction Guide

Setup

git clone https://github.com/pytorch/executorch.git
cd executorch
./install_executorch.sh

Count Vulkan partitions for the dynamic HF RoPE model

cat > /tmp/count_partitions.py <<'PY'
import torch
from executorch.exir import to_edge
from executorch.backends.vulkan.partitioner.vulkan_partitioner import VulkanPartitioner
from executorch.backends.webgpu.test.ops import test_rope_hf as t

inputs = t._dynamic_inputs()
with torch._dynamo.config.patch(capture_scalar_outputs=True):
    ep = torch.export.export(t.DynamicHfRope().eval(), inputs)
res = VulkanPartitioner().partition(to_edge(ep).exported_program())
tags = {
    n.meta["delegation_tag"]
    for n in res.tagged_exported_program.graph.nodes
    if n.meta.get("delegation_tag")
}
print("partitions:", len(tags))
PY
python /tmp/count_partitions.py 2>&1 | grep -E "Vulkan Partitioner|partitions:"

Run once on this commit (expect partitions: 1) and once on its parent (git checkout HEAD~1, expect partitions: 2 plus "symbolic float or bool values are not supported" skip logs).

Reviewers:

This PR was authored with Claude Code.

cc @manuelcandales @digantdesai @cbilgin

Summary:

## Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on `main`, which has failed on every commit since "[Vulkan] Keep unrepresentable and symbolic scalars out of the delegate (#23247)".

## Rationale

**What**: The Vulkan partitioner no longer applies its SymBool/SymFloat rejection to assert ops (`_assert_scalar`, `sym_constrain_range_for_size`) or to nodes whose only users are assert ops.

**Why**: #23247 rejects every node that produces or consumes a SymBool. Models that guard an `.item()` value with `torch._check` produce `u <= N` / `u >= 0` comparisons feeding `_assert_scalar`. Rejecting them cascades through the new "symbolic scalar has an unsupported consumer" check to the `_local_scalar_dense` producer, splitting the graph into multiple delegates. `test_rope_hf.py` then fails with `AssertionError: expected one delegate, got 2`. Asserts are stripped by `RemoveAssertsTransform` during preprocessing, so these SymBool values never need a runtime representation.

## Details

- Assert-only SymBool chains
  - **Before**: Comparisons and asserts rejected; `.item()` producer rejected as a consequence; graph split into multiple delegates.
  - **After**: Delegated, then removed during preprocessing; graph stays in one delegate.
- SymBool/SymFloat values feeding non-assert ops
  - **Before**: Rejected.
  - **After**: Unchanged (still rejected).

Test Plan:

## Observed Results

| Validation | Device | Result |
|---|---|---|
| Vulkan partition count for `DynamicHfRope` (`test_rope_hf.py`), parent commit | macOS (Apple Silicon) host, partitioner only | 2 partitions; skip logs match the CI failure |
| Vulkan partition count for `DynamicHfRope`, this commit | macOS (Apple Silicon) host, partitioner only | 1 partition, no skipped nodes |
| `test_rope_hf.py` end-to-end ("Test WebGPU Native (Dawn)") | — | Not run: needs the Dawn/SwiftShader runtime; relying on CI |
| `backends/vulkan/test/test_vulkan_dynamic.py` | — | Not run: needs the Vulkan runtime; relying on CI |
| Lint | — | Not run: lint tooling not installed locally; relying on CI |

## Reproduction Guide

### Setup

```bash
git clone https://github.com/pytorch/executorch.git
cd executorch
./install_executorch.sh
```

### Count Vulkan partitions for the dynamic HF RoPE model

```bash
cat > /tmp/count_partitions.py <<'PY'
import torch
from executorch.exir import to_edge
from executorch.backends.vulkan.partitioner.vulkan_partitioner import VulkanPartitioner
from executorch.backends.webgpu.test.ops import test_rope_hf as t

inputs = t._dynamic_inputs()
with torch._dynamo.config.patch(capture_scalar_outputs=True):
    ep = torch.export.export(t.DynamicHfRope().eval(), inputs)
res = VulkanPartitioner().partition(to_edge(ep).exported_program())
tags = {
    n.meta["delegation_tag"]
    for n in res.tagged_exported_program.graph.nodes
    if n.meta.get("delegation_tag")
}
print("partitions:", len(tags))
PY
python /tmp/count_partitions.py 2>&1 | grep -E "Vulkan Partitioner|partitions:"
```

Run once on this commit (expect `partitions: 1`) and once on its parent (`git checkout HEAD~1`, expect `partitions: 2` plus "symbolic float or bool values are not supported" skip logs).

Reviewers:

This PR was authored with Claude Code.
@pytorch-bot

pytorch-bot Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23545

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 115aac7 with merge base 9cced90 (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot Bot added the module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/ label Oct 7, 2026
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@SS-JIA
SS-JIA merged commit 86896d4 into main Oct 7, 2026
@SS-JIA
SS-JIA deleted the vulkan-assert-symbool branch October 7, 2026 22:10
SS-JIA added a commit that referenced this pull request Oct 8, 2026
Summary:

## Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests.

## Rationale

**What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader.

**Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made that partitioner fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). `DynamicShape.SwiGluReusedGraph` and `DynamicShape.SwiGluCommutativeAndOwnership` failed at `load_forward()`.

## Details

- Programs containing SwiGLU
  - **Before**: Load fails with "unsupported op: et_vk.swiglu.default".
  - **After**: Lowered to one `silu_mul_fused` dispatch, the same kernel the runtime's load-time SwiGLU fusion already emits.
- `add_silu_mul_fused_dispatch`
  - **Before**: Registered its resize hook on a single tensor (the shared q4 linear input).
  - **After**: Takes a list of trigger tensors. The load-time fusion passes its shared input; the new op passes both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks.

The op requires fp32 operands with identical `gate`, `up`, and output shapes, which is what the flat `silu_mul_fused` kernel supports.

Test Plan:

## Observed Results

### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader)

| Run | Result |
|---|---|
| `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) |
| This branch via `workflow_dispatch` (run 37718579736) | Passed |

Per-binary results for run 37718579736:

| Test binary | Passed | Skipped | Failed |
|---|---|---|---|
| webgpu_native_test | 29 | 3 | 0 |
| webgpu_update_cache_test | 3 | 0 | 0 |
| webgpu_dispatch_order_test | 9 | 0 | 0 |
| webgpu_index_test | 4 | 0 | 0 |
| webgpu_dynamic_shape_test | 40 | 4 | 0 |
| webgpu_scratch_buffer_test | 8 | 0 | 0 |
| webgpu_dispatch_2d_test | 13 | 0 | 0 |
| webgpu_compute_dispatch_test | 20 | 0 | 0 |
| webgpu_execution_options_test | 12 | 0 | 0 |
| webgpu_output_suppression_test | 3 | 0 | 0 |
| webgpu_op_test_util_test | 8 | 0 | 0 |
| webgpu_op_test (op-test framework) | 391 | 0 | 0 |

The 4 skipped dynamic-shape tests are the same ones skipped on `main`.

### Local (macOS, Apple Silicon, Dawn on Metal)

| Validation | Result |
|---|---|
| SwiGLU fixtures, parent commit | All SwiGLU programs fail to load (reproduces CI) |
| SwiGLU fixtures, this commit | All load and match goldens except `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) |
| `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QuantizedLinearSweep` | Fail on Metal on both parent and this commit; pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 linear path, and passes in CI |
| Performance | Not run: the op emits the same single `silu_mul_fused` dispatch as the existing load-time fusion |

## Reproduction Guide

### Run the CI workflow on this branch

```bash
gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op
gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1
```

### Run locally on Linux (mirrors the CI job)

```bash
git clone --recursive https://github.com/pytorch/executorch.git
cd executorch
git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD
./install_executorch.sh
source .ci/scripts/setup-webgpu-linux-deps.sh
bash backends/webgpu/scripts/test_webgpu_native_ci.sh
```

### Run only the SwiGLU tests after the script has built and exported fixtures

```bash
cmake-out-webgpu-dawn-ci/backends/webgpu/webgpu_dynamic_shape_test /tmp/dynamic_shape \
  --gtest_filter='*SwiGlu*'
```

Reviewers:

This PR was authored with Claude Code.
SS-JIA added a commit that referenced this pull request Oct 8, 2026
Summary:

## Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests.

## Rationale

**What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader.

**Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made that partitioner fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). `DynamicShape.SwiGluReusedGraph` and `DynamicShape.SwiGluCommutativeAndOwnership` failed at `load_forward()`.

## Details

- Programs containing SwiGLU
  - **Before**: Load fails with "unsupported op: et_vk.swiglu.default".
  - **After**: Lowered to one `silu_mul_fused` dispatch, the same kernel the runtime's load-time SwiGLU fusion already emits.
- `add_silu_mul_fused_dispatch`
  - **Before**: Registered its resize hook on a single tensor (the shared q4 linear input).
  - **After**: Takes a list of trigger tensors. The load-time fusion passes its shared input; the new op passes both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks.

The op requires fp32 operands with identical `gate`, `up`, and output shapes, which is what the flat `silu_mul_fused` kernel supports.

Test Plan:

## Observed Results

### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader)

| Run | Result |
|---|---|
| `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) |
| This branch via `workflow_dispatch` (run 37718579736) | Passed |

Per-binary results for run 37718579736:

| Test binary | Passed | Skipped | Failed |
|---|---|---|---|
| webgpu_native_test | 29 | 3 | 0 |
| webgpu_update_cache_test | 3 | 0 | 0 |
| webgpu_dispatch_order_test | 9 | 0 | 0 |
| webgpu_index_test | 4 | 0 | 0 |
| webgpu_dynamic_shape_test | 40 | 4 | 0 |
| webgpu_scratch_buffer_test | 8 | 0 | 0 |
| webgpu_dispatch_2d_test | 13 | 0 | 0 |
| webgpu_compute_dispatch_test | 20 | 0 | 0 |
| webgpu_execution_options_test | 12 | 0 | 0 |
| webgpu_output_suppression_test | 3 | 0 | 0 |
| webgpu_op_test_util_test | 8 | 0 | 0 |
| webgpu_op_test (op-test framework) | 391 | 0 | 0 |

The 4 skipped dynamic-shape tests are the same ones skipped on `main`.

### Local (macOS, Apple Silicon, Dawn on Metal)

| Validation | Result |
|---|---|
| SwiGLU fixtures, parent commit | All SwiGLU programs fail to load (reproduces CI) |
| SwiGLU fixtures, this commit | All load and match goldens except `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) |
| `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QuantizedLinearSweep` | Fail on Metal on both parent and this commit; pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 linear path, and passes in CI |
| Performance | Not run: the op emits the same single `silu_mul_fused` dispatch as the existing load-time fusion |

## Reproduction Guide

### Run the CI workflow on this branch

```bash
gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op
gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1
```

### Run locally on Linux (mirrors the CI job)

```bash
git clone --recursive https://github.com/pytorch/executorch.git
cd executorch
git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD
./install_executorch.sh
source .ci/scripts/setup-webgpu-linux-deps.sh
bash backends/webgpu/scripts/test_webgpu_native_ci.sh
```

### Run only the SwiGLU tests after the script has built and exported fixtures

```bash
cmake-out-webgpu-dawn-ci/backends/webgpu/webgpu_dynamic_shape_test /tmp/dynamic_shape \
  --gtest_filter='*SwiGlu*'
```

Reviewers:

This PR was authored with Claude Code.
SS-JIA added a commit that referenced this pull request Oct 8, 2026
Summary:

## Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests.

## Rationale

**What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader, and delete the runtime SwiGLU graph fusion it replaces.

**Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made Vulkan preprocessing fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). With fusion now done at export time, the runtime pattern matcher can no longer match: everything it accepts (fp32, q4 gate/up producers, gate used only by the pattern, equal shapes) is a strict subset of what the export-time pattern fuses. Instrumenting it on the current fixtures confirmed zero matches across all 89 graph builds in `webgpu_dynamic_shape_test`.

## Details

- Programs containing SwiGLU
  - **Before**: Load fails with "unsupported op: et_vk.swiglu.default".
  - **After**: Lowered to one `silu_mul_fused` dispatch, with the resize hook registered on both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks.
- Runtime SwiGLU graph fusion (`passes/SwiGLU.{h,cpp}`)
  - **Before**: Matched q4 gate/up + `sigmoid` + 2x `mul` at load time, emitted `silu_mul_fused`, and placed `gate` in a scratch buffer.
  - **After**: Deleted. The dispatch code moved into `ops/mul/SwiGLU.cpp`.
- QKV-BK64 vs. SwiGLU precedence
  - **Before**: The runtime SwiGLU pass claimed the gate/up linears, and `retain_unclaimed_qkv_fusions` dropped any QKV-BK64 group overlapping them.
  - **After**: Removed. Since #22979 the elementwise fusion no longer owns the linears, so both fusions apply: q/gate/up share one `linear_q4gsw_bk64_qkv` dispatch followed by `silu_mul_fused`. QKV candidates are grouped by input tensor and cannot overlap each other, so `retain_unclaimed_qkv_fusions` had no other effect.
- Profiling-only tests (`SwiGluFusionProfile`, `SwiGluQkvOverlapProfile`)
  - **Before**: Encoded the runtime matcher's rules: no fusion when `gate` has an extra consumer, is a graph output, or when `up` comes from a separate input, and no QKV-BK64 dispatch in the overlap case. These tests only build with `EXECUTORCH_BUILD_WEBGPU_PROFILING`, so CI does not run them and they had gone stale since #22979.
  - **After**: Expect fusion in those three cases (`gate` stays materialized, so fusing is valid) and one `linear_q4gsw_bk64_qkv` dispatch in the overlap case. Cases where `sigmoid` or `silu` has another user still expect no fusion.

Test Plan:

## Observed Results

### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader)

| Run | Result |
|---|---|
| `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) |
| Op only, before removing the runtime fusion (run 37718579736) | Passed (all 11 native test binaries; `webgpu_dynamic_shape_test` 40 passed / 4 skipped, same skips as `main`; op-test framework 391 passed) |
| This commit, rebased on "Vulkan: narrow the int4 linear output tile on Mali" | Pending |

### Local (macOS, Apple Silicon, Dawn on Metal, `EXECUTORCH_BUILD_WEBGPU_PROFILING=ON`, `WEBGPU_TIMESTAMP_QUERY=1`)

| Validation | Result |
|---|---|
| Runtime SwiGLU matcher hit count on current fixtures (temporary instrumentation, before deletion) | 0 matches in 89 graph builds |
| `SwiGluReusedGraph`, `SwiGluFusionProfile` | Pass |
| `SwiGluQkvOverlapProfile` dispatch counts (1 `linear_q4gsw_bk64_qkv`, 1 `silu_mul_fused`, 0 `mul`) | Pass |
| `SwiGluCommutativeAndOwnership`, `SwiGluQkvOverlapProfile` numerics | Fail only on `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) |
| `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QkvBk64LiveRoutesProfile`, `QuantizedLinearBk64QkvProfileSoleWriter` | Fail on Metal with and without this change; BK64 tests pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 path and passed in CI run 37718579736 |
| Performance | Not run. New exports lose only the `gate` scratch-buffer reuse; programs exported before #22979 lose the load-time fusion and run `sigmoid`/`mul`/`mul` as three dispatches |

## Reproduction Guide

### Run the CI workflow on this branch

```bash
gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op
gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1
```

### Run locally on Linux (mirrors the CI job)

```bash
git clone --recursive https://github.com/pytorch/executorch.git
cd executorch
git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD
./install_executorch.sh
source .ci/scripts/setup-webgpu-linux-deps.sh
bash backends/webgpu/scripts/test_webgpu_native_ci.sh
```

### Run the profiling-only SwiGLU tests

After the script above has exported fixtures to `/tmp/dynamic_shape`:

```bash
cmake -DPYTHON_EXECUTABLE=python \
  -DEXECUTORCH_BUILD_WEBGPU=ON -DEXECUTORCH_BUILD_WEBGPU_TEST=ON \
  -DEXECUTORCH_BUILD_WEBGPU_PROFILING=ON -DEXECUTORCH_BUILD_TESTS=ON \
  -DDawn_DIR="${Dawn_DIR}" \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \
  -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON -DCMAKE_BUILD_TYPE=Release \
  -B cmake-out-webgpu-prof .
cmake --build cmake-out-webgpu-prof --target webgpu_dynamic_shape_test
WEBGPU_TIMESTAMP_QUERY=1 cmake-out-webgpu-prof/backends/webgpu/webgpu_dynamic_shape_test \
  /tmp/dynamic_shape --gtest_filter='*SwiGlu*'
```

Reviewers:

This PR was authored with Claude Code.
SS-JIA added a commit that referenced this pull request Oct 8, 2026
Summary:

## Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests.

## Rationale

**What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader, and delete the runtime SwiGLU graph fusion it replaces.

**Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made Vulkan preprocessing fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). With fusion now done at export time, the runtime pattern matcher can no longer match: everything it accepts (fp32, q4 gate/up producers, gate used only by the pattern, equal shapes) is a strict subset of what the export-time pattern fuses. Instrumenting it on the current fixtures confirmed zero matches across all 89 graph builds in `webgpu_dynamic_shape_test`.

## Details

- Programs containing SwiGLU
  - **Before**: Load fails with "unsupported op: et_vk.swiglu.default".
  - **After**: Lowered to one `silu_mul_fused` dispatch, with the resize hook registered on both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks.
- Runtime SwiGLU graph fusion (`passes/SwiGLU.{h,cpp}`)
  - **Before**: Matched q4 gate/up + `sigmoid` + 2x `mul` at load time, emitted `silu_mul_fused`, and placed `gate` in a scratch buffer.
  - **After**: Deleted. The dispatch code moved into `ops/mul/SwiGLU.cpp`.
- QKV-BK64 vs. SwiGLU precedence
  - **Before**: The runtime SwiGLU pass claimed the gate/up linears, and `retain_unclaimed_qkv_fusions` dropped any QKV-BK64 group overlapping them.
  - **After**: Removed. Since #22979 the elementwise fusion no longer owns the linears, so both fusions apply: q/gate/up share one `linear_q4gsw_bk64_qkv` dispatch followed by `silu_mul_fused`. QKV candidates are grouped by input tensor and cannot overlap each other, so `retain_unclaimed_qkv_fusions` had no other effect.
- Profiling-only tests (`SwiGluFusionProfile`, `SwiGluQkvOverlapProfile`)
  - **Before**: Encoded the runtime matcher's rules: no fusion when `gate` has an extra consumer, is a graph output, or when `up` comes from a separate input, and no QKV-BK64 dispatch in the overlap case. These tests only build with `EXECUTORCH_BUILD_WEBGPU_PROFILING`, so CI does not run them and they had gone stale since #22979.
  - **After**: Expect fusion in those three cases (`gate` stays materialized, so fusing is valid) and one `linear_q4gsw_bk64_qkv` dispatch in the overlap case. Cases where `sigmoid` or `silu` has another user still expect no fusion.

Test Plan:

## Observed Results

### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader)

| Run | Result |
|---|---|
| `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) |
| Op only, before removing the runtime fusion (run 37718579736) | Passed (all 11 native test binaries; `webgpu_dynamic_shape_test` 40 passed / 4 skipped, same skips as `main`; op-test framework 391 passed) |
| This commit, rebased on "Vulkan: narrow the int4 linear output tile on Mali" (run 37820744165) | Passed |

Per-binary results for run 37820744165:

| Test binary | Passed | Skipped | Failed |
|---|---|---|---|
| webgpu_native_test | 29 | 3 | 0 |
| webgpu_update_cache_test | 3 | 0 | 0 |
| webgpu_dispatch_order_test | 9 | 0 | 0 |
| webgpu_index_test | 4 | 0 | 0 |
| webgpu_dynamic_shape_test | 40 | 4 | 0 |
| webgpu_scratch_buffer_test | 8 | 0 | 0 |
| webgpu_dispatch_2d_test | 13 | 0 | 0 |
| webgpu_compute_dispatch_test | 20 | 0 | 0 |
| webgpu_execution_options_test | 12 | 0 | 0 |
| webgpu_output_suppression_test | 3 | 0 | 0 |
| webgpu_op_test_util_test | 8 | 0 | 0 |
| webgpu_op_test (op-test framework) | 391 | 0 | 0 |

The 4 skipped dynamic-shape tests are the same ones skipped on `main`. The profiling-only SwiGLU tests are not built in CI; see the local results below.

### Local (macOS, Apple Silicon, Dawn on Metal, `EXECUTORCH_BUILD_WEBGPU_PROFILING=ON`, `WEBGPU_TIMESTAMP_QUERY=1`)

| Validation | Result |
|---|---|
| Runtime SwiGLU matcher hit count on current fixtures (temporary instrumentation, before deletion) | 0 matches in 89 graph builds |
| `SwiGluReusedGraph`, `SwiGluFusionProfile` | Pass |
| `SwiGluQkvOverlapProfile` dispatch counts (1 `linear_q4gsw_bk64_qkv`, 1 `silu_mul_fused`, 0 `mul`) | Pass |
| `SwiGluCommutativeAndOwnership`, `SwiGluQkvOverlapProfile` numerics | Fail only on `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) |
| `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QkvBk64LiveRoutesProfile`, `QuantizedLinearBk64QkvProfileSoleWriter` | Fail on Metal with and without this change; BK64 tests pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 path and passed in CI run 37718579736 |
| Performance | Not run. New exports lose only the `gate` scratch-buffer reuse; programs exported before #22979 lose the load-time fusion and run `sigmoid`/`mul`/`mul` as three dispatches |

## Reproduction Guide

### Run the CI workflow on this branch

```bash
gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op
gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1
```

### Run locally on Linux (mirrors the CI job)

```bash
git clone --recursive https://github.com/pytorch/executorch.git
cd executorch
git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD
./install_executorch.sh
source .ci/scripts/setup-webgpu-linux-deps.sh
bash backends/webgpu/scripts/test_webgpu_native_ci.sh
```

### Run the profiling-only SwiGLU tests

After the script above has exported fixtures to `/tmp/dynamic_shape`:

```bash
cmake -DPYTHON_EXECUTABLE=python \
  -DEXECUTORCH_BUILD_WEBGPU=ON -DEXECUTORCH_BUILD_WEBGPU_TEST=ON \
  -DEXECUTORCH_BUILD_WEBGPU_PROFILING=ON -DEXECUTORCH_BUILD_TESTS=ON \
  -DDawn_DIR="${Dawn_DIR}" \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \
  -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON -DCMAKE_BUILD_TYPE=Release \
  -B cmake-out-webgpu-prof .
cmake --build cmake-out-webgpu-prof --target webgpu_dynamic_shape_test
WEBGPU_TIMESTAMP_QUERY=1 cmake-out-webgpu-prof/backends/webgpu/webgpu_dynamic_shape_test \
  /tmp/dynamic_shape --gtest_filter='*SwiGlu*'
```

Reviewers:

This PR was authored with Claude Code.
SS-JIA added a commit that referenced this pull request Oct 8, 2026
Summary:

## Ulterior Motive

Restore the "Test WebGPU Native (Dawn)" workflow on `main`. After "[Vulkan] Allow assert-only SymBool chains in the partitioner (#23545)" fixed the RoPE export, the job advanced to a second failure in the SwiGLU dynamic-shape tests.

## Rationale

**What**: Register `et_vk.swiglu.default` in the WebGPU runtime, backed by the existing `silu_mul_fused` shader, and delete the runtime SwiGLU graph fusion it replaces.

**Why**: The WebGPU backend reuses the Vulkan partitioner. "[Vulkan] Fuse SwiGLU elementwise operations (#22979)" made Vulkan preprocessing fuse `sigmoid`/`mul`/`mul` into `et_vk.swiglu`, which the WebGPU runtime did not implement, so loading any program containing a SwiGLU failed with `DelegateInvalidCompatibility` (`0x30`). With fusion now done at export time, the runtime pattern matcher can no longer match: everything it accepts (fp32, q4 gate/up producers, gate used only by the pattern, equal shapes) is a strict subset of what the export-time pattern fuses. Instrumenting it on the current fixtures confirmed zero matches across all 89 graph builds in `webgpu_dynamic_shape_test`.

## Details

- Programs containing SwiGLU
  - **Before**: Load fails with "unsupported op: et_vk.swiglu.default".
  - **After**: Lowered to one `silu_mul_fused` dispatch, with the resize hook registered on both `gate` and `up`, matching how two-input ops such as `mul` register resize hooks.
- Runtime SwiGLU graph fusion (`passes/SwiGLU.{h,cpp}`)
  - **Before**: Matched q4 gate/up + `sigmoid` + 2x `mul` at load time, emitted `silu_mul_fused`, and placed `gate` in a scratch buffer.
  - **After**: Deleted. The dispatch code moved into `ops/mul/SwiGLU.cpp`.
- QKV-BK64 vs. SwiGLU precedence
  - **Before**: The runtime SwiGLU pass claimed the gate/up linears, and `retain_unclaimed_qkv_fusions` dropped any QKV-BK64 group overlapping them.
  - **After**: Removed. Since #22979 the elementwise fusion no longer owns the linears, so both fusions apply: q/gate/up share one `linear_q4gsw_bk64_qkv` dispatch followed by `silu_mul_fused`. QKV candidates are grouped by input tensor and cannot overlap each other, so `retain_unclaimed_qkv_fusions` had no other effect.
- Profiling-only tests (`SwiGluFusionProfile`, `SwiGluQkvOverlapProfile`)
  - **Before**: Encoded the runtime matcher's rules: no fusion when `gate` has an extra consumer, is a graph output, or when `up` comes from a separate input, and no QKV-BK64 dispatch in the overlap case. These tests only build with `EXECUTORCH_BUILD_WEBGPU_PROFILING`, so CI does not run them and they had gone stale since #22979.
  - **After**: Expect fusion in those three cases (`gate` stays materialized, so fusing is valid) and one `linear_q4gsw_bk64_qkv` dispatch in the overlap case. Cases where `sigmoid` or `silu` has another user still expect no fusion.

Test Plan:

## Observed Results

### CI ("Test WebGPU Native (Dawn)", Linux, Dawn + SwiftShader)

| Run | Result |
|---|---|
| `main` at "Give device-tagged method outputs device memory in the Python bindings (#23547)" (run 37704744609) | Failed: `DynamicShape.SwiGluReusedGraph`, `DynamicShape.SwiGluCommutativeAndOwnership` (`load_forward()` returned `0x30`) |
| Op only, before removing the runtime fusion (run 37718579736) | Passed (all 11 native test binaries; `webgpu_dynamic_shape_test` 40 passed / 4 skipped, same skips as `main`; op-test framework 391 passed) |
| This commit, rebased on "Vulkan: narrow the int4 linear output tile on Mali" (run 37820744165) | Passed |

Per-binary results for run 37820744165:

| Test binary | Passed | Skipped | Failed |
|---|---|---|---|
| webgpu_native_test | 29 | 3 | 0 |
| webgpu_update_cache_test | 3 | 0 | 0 |
| webgpu_dispatch_order_test | 9 | 0 | 0 |
| webgpu_index_test | 4 | 0 | 0 |
| webgpu_dynamic_shape_test | 40 | 4 | 0 |
| webgpu_scratch_buffer_test | 8 | 0 | 0 |
| webgpu_dispatch_2d_test | 13 | 0 | 0 |
| webgpu_compute_dispatch_test | 20 | 0 | 0 |
| webgpu_execution_options_test | 12 | 0 | 0 |
| webgpu_output_suppression_test | 3 | 0 | 0 |
| webgpu_op_test_util_test | 8 | 0 | 0 |
| webgpu_op_test (op-test framework) | 391 | 0 | 0 |

The 4 skipped dynamic-shape tests are the same ones skipped on `main`. The profiling-only SwiGLU tests are not built in CI; see the local results below.

### Local (macOS, Apple Silicon, Dawn on Metal, `EXECUTORCH_BUILD_WEBGPU_PROFILING=ON`, `WEBGPU_TIMESTAMP_QUERY=1`)

| Validation | Result |
|---|---|
| Runtime SwiGLU matcher hit count on current fixtures (temporary instrumentation, before deletion) | 0 matches in 89 graph builds |
| `SwiGluReusedGraph`, `SwiGluFusionProfile` | Pass |
| `SwiGluQkvOverlapProfile` dispatch counts (1 `linear_q4gsw_bk64_qkv`, 1 `silu_mul_fused`, 0 `mul`) | Pass |
| `SwiGluCommutativeAndOwnership`, `SwiGluQkvOverlapProfile` numerics | Fail only on `dyn_swiglu_qkv_overlap` output 1 (max err 0.025 vs 0.01) |
| `QuantizedLinearBk64QkvReusedGraphAndFallbacks`, `QkvBk64LiveRoutesProfile`, `QuantizedLinearBk64QkvProfileSoleWriter` | Fail on Metal with and without this change; BK64 tests pass in CI. `dyn_swiglu_qkv_overlap` uses the same BK64 path and passed in CI run 37718579736 |
| Performance | Not run. New exports lose only the `gate` scratch-buffer reuse; programs exported before #22979 lose the load-time fusion and run `sigmoid`/`mul`/`mul` as three dispatches |

## Reproduction Guide

### Run the CI workflow on this branch

```bash
gh workflow run test-webgpu-native.yml --repo pytorch/executorch --ref webgpu-swiglu-op
gh run list --repo pytorch/executorch --workflow test-webgpu-native.yml --branch webgpu-swiglu-op --limit 1
```

### Run locally on Linux (mirrors the CI job)

```bash
git clone --recursive https://github.com/pytorch/executorch.git
cd executorch
git fetch origin webgpu-swiglu-op && git checkout FETCH_HEAD
./install_executorch.sh
source .ci/scripts/setup-webgpu-linux-deps.sh
bash backends/webgpu/scripts/test_webgpu_native_ci.sh
```

### Run the profiling-only SwiGLU tests

After the script above has exported fixtures to `/tmp/dynamic_shape`:

```bash
cmake -DPYTHON_EXECUTABLE=python \
  -DEXECUTORCH_BUILD_WEBGPU=ON -DEXECUTORCH_BUILD_WEBGPU_TEST=ON \
  -DEXECUTORCH_BUILD_WEBGPU_PROFILING=ON -DEXECUTORCH_BUILD_TESTS=ON \
  -DDawn_DIR="${Dawn_DIR}" \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON -DEXECUTORCH_BUILD_EXTENSION_FLAT_TENSOR=ON \
  -DEXECUTORCH_BUILD_EXTENSION_NAMED_DATA_MAP=ON -DCMAKE_BUILD_TYPE=Release \
  -B cmake-out-webgpu-prof .
cmake --build cmake-out-webgpu-prof --target webgpu_dynamic_shape_test
WEBGPU_TIMESTAMP_QUERY=1 cmake-out-webgpu-prof/backends/webgpu/webgpu_dynamic_shape_test \
  /tmp/dynamic_shape --gtest_filter='*SwiGlu*'
```

Reviewers:

This PR was authored with Claude Code.

This branch was previously deployed

1 inactive deployment
cadence — 115aac72 Deployed Oct 7, 2026 by SS-JIA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: vulkan Issues related to the Vulkan delegate and code under backends/vulkan/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants