Skip to content

[RELEASE ONLY CHANGES] Finalize ExecuTorch 1.5 dependencies - #22724

Merged
JacobSzwejbka merged 13 commits into
release/1.5from
release-only/1.5-finalize-versions
Sep 14, 2026
Merged

[RELEASE ONLY CHANGES] Finalize ExecuTorch 1.5 dependencies#22724
JacobSzwejbka merged 13 commits into
release/1.5from
release-only/1.5-finalize-versions

Conversation

@JacobSzwejbka

@JacobSzwejbka JacobSzwejbka commented Sep 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • pin the tokenizers submodule to v1.5.0-rc2 while keeping the wheel dependency at the latest published version, pytorch-tokenizers 1.4.1
  • pin torchao to v0.18.0 and update the submodule to its release tag
  • move TorchCodec, QNN, and ROCm release installs to stable packages and production indexes
  • restore the Qwen CUDA export matmul mode that TorchAO 0.18 no longer sets globally
  • update source, SwiftPM, and Raspberry Pi documentation for release/1.5
  • accommodate the current Transformers Qwen3-VL annotations in mypy

Validation

  • python3 -m py_compile setup.py examples/models/qwen3_5_moe/export.py backends/arm/test/models/Qwen3_VL/test_qwen3_vl_model.py
  • Black 24.4.2 check for the changed Python files
  • git diff --check
  • verified pytorch-tokenizers 1.4.1 publishes wheels for Python 3.10-3.14 on Windows, Linux x86-64/aarch64, and macOS arm64
  • verified the tokenizers and torchao gitlinks resolve exactly to v1.5.0-rc2 and v0.18.0
  • verified torch 2.14.0, torchao 0.18.0, and TorchCodec 0.16.0 resolve from their configured indexes

Release status

This stays draft while pytorch/test-infra#8780 blocks persistence of the tokenizers RC2 wheel artifacts. Raise the package dependency to pytorch-tokenizers>=1.5.0 after the final tokenizer wheels are promoted, then verify that the final tag points to the RC2 commit or update the submodule gitlink.

This PR was authored with Codex assistance.

@JacobSzwejbka
Finalize ExecuTorch 1.5 release dependencies
39bb903 
Pin the release dependency set and update the release documentation. Authored with Codex assistance.
@pytorch-bot

pytorch-bot Bot commented Sep 11, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22724

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Cancelled Job, 52 Pending

As of commit d220ca0 with merge base 5cdcc02 (image):

NEW FAILURES - The following jobs have failed:

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2026
@JacobSzwejbka
JacobSzwejbka deployed to upload-benchmark-results September 11, 2026 08:56 — with GitHub Actions Active
@JacobSzwejbka
Fix release dependency CI
4386774 
Keep wheel installation resolvable until the tokenizers 1.5 package is published, adapt Qwen CUDA export to TorchAO 0.18 matmul settings, and accommodate current Transformers annotations.\n\nAuthored with Codex assistance.
@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Sep 11, 2026
@JacobSzwejbka
JacobSzwejbka deployed to upload-benchmark-results September 11, 2026 19:15 — with GitHub Actions Active
@JacobSzwejbka
Trace Qwen CUDA launch failure
c29ed28 
Enable synchronous CUDA launches only for the Qwen3.5 end-to-end export so CI identifies the kernel that causes the PyTorch 2.14 illegal memory access. Revert the ineffective TF32 precision workaround.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Limit Qwen CUDA autotuning to GEMMs
769ae45 
Keep Triton GEMM selection enabled without expanding pointwise and reduction autotuning under PyTorch 2.14. Add an explicit CUDA backend compile spec for the narrower setting.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Remove Qwen CUDA diagnostic mode
608cb73 
Drop the temporary launch-blocking setting now that the PyTorch 2.14 CUDA graph capture behavior is identified.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Fix CUDA compile option lint
a5ebb6d 
Move compile-spec parsing into a small helper so the CUDA option builder stays below the cyclomatic complexity limit.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Drop unrelated Qwen CUDA workaround
68d732c 
The same Qwen3.5 illegal-memory-access failure is present on main after the PyTorch 2.14 update, so keep the release-only dependency PR from changing shared CUDA backend behavior.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Scope Qwen CUDA autotuning to GEMMs
3e23959 
Keep Triton GEMM selection without enabling PyTorch 2.14's broader pointwise and reduction autotuning paths during the low-memory Qwen export.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Disable Qwen CUDA autotuning
93c57c7 
Avoid launching Inductor autotuning kernels against Qwen's released low-memory KV-cache storage under PyTorch 2.14. Keep the workaround local to the model exporter.\n\nAuthored with Codex assistance.
@JacobSzwejbka
JacobSzwejbka deployed to upload-benchmark-results September 12, 2026 00:01 — with GitHub Actions Active
@JacobSzwejbka
Keep Qwen KV storage resident during CUDA export
83c0812 
Avoid the PyTorch 2.14 compile path that launches kernels against released Qwen KV-cache storage while retaining the required Triton GEMM lowering.\n\nAuthored with Codex assistance.
@JacobSzwejbka
Eagerly autotune Qwen CUDA kernels
710103a 
Keep Qwen low-memory export enabled while forcing compile-time Triton autotuning so AOTI embeds kernels instead of retaining lazy references to temporary files.\n\nAuthored with Codex assistance.
@JacobSzwejbka
JacobSzwejbka deployed to upload-benchmark-results September 12, 2026 04:12 — with GitHub Actions Active
@JacobSzwejbka
Require pytorch-tokenizers 1.5
d220ca0 
Restore the intended release dependency now that the 1.5.0 wheels are published on PyPI.\n\nAuthored with Codex assistance.
@JacobSzwejbka
JacobSzwejbka merged commit f7140a4 into release/1.5 Sep 14, 2026
860 of 869 checks passed
@JacobSzwejbka
JacobSzwejbka deleted the release-only/1.5-finalize-versions branch September 14, 2026 19:09
@JacobSzwejbka
JacobSzwejbka deployed to upload-benchmark-results September 14, 2026 19:42 — with GitHub Actions Active
This was referenced Oct 27, 2026
Merged
Merged
JacobSzwejbka added a commit that referenced this pull request Oct 30, 2026
@JacobSzwejbka
[RELEASE ONLY CHANGES] Prepare ExecuTorch 1.6 release dependencies (#…
72afc21 
…24199)

Release-only preparation for ExecuTorch 1.6.0 and PyTorch 2.15,
following #21563 and #22724. Relates to #23608.

Review the installer and `torch_pin.py` first: they select PyTorch
2.15.0, torchvision 0.29.2, torchaudio 2.11.1, and production PyTorch
indexes. The full wheel requires `torch==2.15.0`, `torchao>=0.18.0`, and
unpinned `pytorch-tokenizers`. The minimal wheel excludes torch and
TorchAO. `pyproject.toml` documents these dynamically generated
requirements. When `install_executorch.py --use-pt-pinned-commit` builds
a CI wheel, its metadata instead pins the already installed source build
so pip cannot replace the torch used to link its extensions. Ordinary
release wheel metadata retains the exact stable torch pin.

The PyTorch source pin is the final `v2.15.0` tag,
`31a77fc9c0a7c6fff15a1ff58c743f53b1a9d457`, including the Inductor
prefetch revert. The other dependency edits align QNN, TorchCodec
0.17.0, and the Windows smoke check with the release packages.

CI follow-up: the branch now includes the current `release/1.6`
cherry-picks, including the device placement tests that the release
workflow already invokes. CUDA CI uses the published 13.2 and 13.4
channels and their verified available container images. Every CUDA
install step constrains torch and its domain libraries to the selected
local wheel variant; requesting an unpublished 12.6/13.0 variant fails
before pip and directs source-build users to `--use-pt-pinned-commit`.
The existing separate nightly TorchAO indexes and aarch64 CPU fallback
are retained with the shared minimum requirement.

ROCm CI uses the published torch 2.15.0+rocm7.14 wheel and the available
ROCm 7.14 builder image. That channel has only Python-only TorchAO
wheels, so both ROCm jobs build the existing pinned TorchAO source with
`USE_CPP=1 USE_CPU_KERNELS=1` against the installed torch/HIP libraries.
The CPU kernel option supplies the extension sources on ROCm; without it
the pinned setup produces a Python-only wheel. The installer adds
`rocm[devel]` at the already installed ROCm SDK version and uses `python
-m rocm_sdk path --root` for the build and runtime paths, including the
HIP linker library. The ROCm jobs use conda-forge GCC 16.2 with matching
libstdc++/libgcc and a glibc 2.28 sysroot matching the builder image,
avoiding the observed Red Hat GCC-toolset nonshared-library link failure
when mixed with conda runtime libraries. The shared installer explicitly
loads the resulting extension and fails on a missing or unloadable
library instead of accepting TorchAO's logged-and-suppressed import
errors.

FLA is updated from 0.4.2 to 0.6.0 in CUDA CI and the Qwen 3.5
requirements. The wrapper is updated for the new kernel arguments,
flattened launch grids and base-2 gate scaling. Numerical test
tolerances are unchanged; an additional GPU regression covers multiple
batches, value tiles and partial chunks. The cumulative-sum launch now
passes `USE_GRAPH=False` explicitly: PyTorch 2.15 export does not supply
the Triton default, causing the observed missing-argument failures in
the export tests and Qwen export. Windows CUDA wheel builds select MSVC
through `CUDAHOSTCXX`, which CMake reads after PyTorch overrides
`CMAKE_CUDA_HOST_COMPILER` with clang-cl. This avoids the observed
CUDA/MSVC math-header collision without changing the C++ compiler. The
earlier CMake argument was ineffective because PyTorch overwrote it. The
Linux Windows-export image and workflow now also use CUDA 13.2, with the
verified available 13.2.2 Windows cross-compilation toolkit. Vulkan GPU
jobs explicitly disable the CUDA delegate, and the installer honors that
CMake option by choosing CPU torch even when nvcc is present. The Vulkan
tests still execute against the real NVIDIA Vulkan driver. Upload and
aggregate jobs retain their existing build dependencies.

## Before this can merge

Platform CI must validate installation and runtime compatibility with
the final production packages. The local checks below do not execute
CUDA or ROCm kernels or the Windows compiler.

pytorch-tokenizers 1.6.0 is published on PyPI:
https://pypi.org/project/pytorch-tokenizers/1.6.0/. All 20 published
wheel hashes match the release manifest from `v1.6.0-rc2`, commit
`f54baeab00d5b664bfc4a977b884ff679fd3f15a`; a downloaded TestPyPI wheel
passed all 17 Python tests. The existing tokenizer gitlink is retained;
final source-tag verification and tokenizer source finalization still
need a release-only follow-up.

## Validation

All 16 installer dependency tests, 6 CUDA workflow tests, and the ROCm
dependency sequencing/failure-propagation test pass locally. Checked all
222 literal FLA 0.6.0 autotune keys against their kernel argument names.
Checked the wrapper's six launch signatures, grid dimensions and gate
scaling against the published FLA source for ordinary, batched, and
partial-chunk inputs. Changed Python files pass Black; shell scripts and
workflow shell blocks pass syntax checks; YAML parses and `git diff
--check` passes. Production torch CUDA 13.2/13.4 and ROCm 7.14 wheel
availability and the corresponding CI container manifests are verified.
The CUDA 13.2 Ubuntu development packages and Windows 13.2.2 installer
are verified available. The pinned TorchAO source selection is checked
with and without the CPU kernel option. The CUDA 13.2 and 13.4
build/runtime probe jobs passed on commit `8846440476`. TorchAO compiled
and loaded on ROCm after the SDK path fix at commit `d7c54f49d4`; that
run then exposed the mixed C++ toolchain link failure addressed by the
latest compiler change. The compiler migration also explicitly pins
`sysroot_linux-64=2.28`: conda otherwise selected 2.12 headers, which
lack the `timespec_get` declaration used by the newer C++ headers.
Verified the host-compiler selection block from CI's CMake 3.31.2 with
an MSVC-style path containing spaces: `CUDAHOSTCXX` replaces PyTorch's
clang-cl selection. This local check does not run the Windows compiler.
The ROCm Voxtral runtime job passed on `b376004507`. CUDA system tests
passed on `8846440476`; kernel tests passed 225 cases and failed two
export cases on the omitted `USE_GRAPH` argument now addressed. The
stricter local launch-contract check fails without that argument and
passes with it. Full kernel, ROCm AOTI and Windows runtime results
remain pending in CI.

Authored with Codex assistance.
JacobSzwejbka added a commit that referenced this pull request Oct 30, 2026
@JacobSzwejbka
[RELEASE ONLY CHANGES] Update documentation for ExecuTorch 1.6 (#24198)
42effcd 
Release-only changes for `release/1.6`, following the documentation
updates for releases 1.3–1.5 (#19791, #21401, #21563, #22724). Relates
to #23608.

The release documentation now clones `release/1.6` and uses ExecuTorch
1.6.0 in SwiftPM, Android AAR, and Raspberry Pi examples. The C++
package instructions describe the prebuilt libraries shipped with 1.6
and install the exact release with PyTorch 2.15 from the production CPU
index. Nightly installation remains available as an explicit choice.

The nightly version guidance sits beside the installation command,
allowing this PR and #24211 to merge cleanly in either order. Git
merge-tree checks verified that both orders produce the same tree.

Validation: reviewed the diff and checked that the touched installation
guides contain no stale clone branches, 1.4/1.5 SwiftPM examples, or
nightly-only C++ install commands. `git diff --check` passes. The 1.6
package and mobile artifact examples target publication with the
release; the final artifacts are not published yet.

Authored with Codex assistance.

This branch was successfully deployed

1 active and 1 inactive deployments
upload-benchmark-results — d220ca05 Deployed Sep 14, 2026 by JacobSzwejbka via upload-benchmark-results #2708
cadence — d220ca05 Deployed Sep 14, 2026 by JacobSzwejbka via hifi-op-test / hifi4 #27759
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Gasoonjia Gasoonjia Gasoonjia approved these changes
@larryliu0820 larryliu0820 Awaiting requested review from larryliu0820 larryliu0820 is a code owner
@mergennachin mergennachin Awaiting requested review from mergennachin mergennachin is a code owner
@digantdesai digantdesai Awaiting requested review from digantdesai digantdesai is a code owner

Assignees

No one assigned

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: arm Issues related to arm backend module: rocm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@JacobSzwejbka @Gasoonjia