Skip to content

[2.15] Bump the ExecuTorch delegate to 0.2.1, bring #4927 and #4928, and fix the #4921 test - #4909

Open
shoumikhin wants to merge 9 commits into
pytorch:release/2.15from
shoumikhin:backport-2.15-4833
Open

shoumikhin wants to merge 9 commits into
pytorch:release/2.15from
shoumikhin:backport-2.15-4833

Conversation

@shoumikhin

@shoumikhin shoumikhin commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What is wrong today

The 2.15 branch has the ExecuTorch changes from main, but five problems are left:

#4927 fixed the second and third on main.

What this change does

  1. Bumps the delegate to 0.2.1, so pip sees it as newer and replaces the old one.
  2. Adds a test that the guide's release install stays one command.
  3. Brings docs(executorch): rewrite the ExecuTorch deployment docs and fix the archive-only delegate build #4927's docs here, with this branch's versions. The other pages link to the guide's install commands:
    • Releases: pip install "torch_tensorrt[executorch]"
    • Nightly: pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132
  4. Backports docs(executorch): rewrite the ExecuTorch deployment docs and fix the archive-only delegate build #4927's CMake fix: the source-built extension_cuda gets ExecuTorch's c10 headers and, when no executorch_core target exists, the prebuilt core library.
  5. Says fold_shared_kv_attention has no effect under torch.compile on PyTorch 2.15, and skips that test below 2.16. With 2.15, torch.compile splits each matmul into expand, reshape and bmm before the pass runs. The 2.16 nightly keeps matmul.
  6. Cherry-picks fix(dynamo): decompose attention TensorRT cannot build, and report why a build failed #4928 as it is.

What the release command still needs

  1. ExecuTorch 1.6.0 CUDA wheels on the test index for CUDA 13.2 and 13.4, Linux and Windows.
  2. This branch moves its ExecuTorch pin to 1.6.0, and the next release candidate is built from it. The guide's Python runtime options and clone_outputs also need 1.6.0.
  3. ExecuTorch 1.6.0's Linux PyPI wheels are its CUDA 13.2 build.
  4. ExecuTorch, the delegate and torch-tensorrt are published in that order.

Do not promote the first 2.15.0 release candidate. Until the nightly index carries dllist and TensorRT, the nightly command also needs --extra-index-url https://pypi.org/simple.

What was tested

  • The pin and docs tests pass with CPU-only PyTorch 2.15. The new CMake test fails without the fix.
  • Linux aarch64, CUDA 13.2: the archive-only build fails before the CMake fix, fails to link with half of it, and builds after it. A program linked as the README shows has no undefined symbols.
  • Linux x86-64, A100: the fold_shared_kv_attention tests had one failure with PyTorch 2.15 and none with the nightly; now 2.15 skips it. fix(dynamo): decompose attention TensorRT cannot build, and report why a build failed #4928's tests pass with 2.15.
  • The changed guide pages build with Sphinx, with no new warnings.

@meta-cla meta-cla Bot added the cla signed label Oct 21, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: api [C++] Issues re: C++ API labels Oct 21, 2026
@github-actions
github-actions Bot requested a review from narendasan October 21, 2026 17:59
@shoumikhin shoumikhin changed the title [2.15] Backport #4833: allow CUDA graph replay on shared activation scratch [2.15] Backport ExecuTorch fixes: #4833, #4839, #4821, #4881, #4882 Oct 22, 2026
@github-actions github-actions Bot added documentation Improvements or additions to documentation component: build system Issues re: Build system component: api [Python] Issues re: Python API labels Oct 22, 2026
@shoumikhin
shoumikhin force-pushed the backport-2.15-4833 branch 2 times, most recently from 74dcb90 to fa1d8a3 Compare October 23, 2026 13:34
@shoumikhin shoumikhin changed the title [2.15] Backport ExecuTorch fixes: #4833, #4839, #4821, #4881, #4882 [2.15] Backport ExecuTorch fixes: #4833, #4839, #4821, #4881, #4882, #4919 Oct 23, 2026
@shoumikhin
shoumikhin force-pushed the backport-2.15-4833 branch 2 times, most recently from 566d13c to 9bac35f Compare October 25, 2026 05:30
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 26, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes, and a nightly command with two indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 26, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes, and a nightly command with two indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
@shoumikhin shoumikhin changed the title [2.15] Backport ExecuTorch fixes: #4833, #4839, #4821, #4881, #4882, #4919 [2.15] Backport #4919 and bring the ExecuTorch delegate version and guide up to date Oct 26, 2026
@lanluo-nvidia
lanluo-nvidia self-requested a review October 26, 2026 20:24
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 26, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes, and a nightly command with two indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
shoumikhin added a commit to pytorch/executorch that referenced this pull request Oct 27, 2026
This is a draft because the page describes the intended end state of
Torch-TensorRT 2.15. Some links go to sections of the Torch-TensorRT
guide that exist only after pytorch/TensorRT#4927 merges and its docs
site is rebuilt. And the examples rely on 2.15 changes from
pytorch/TensorRT#4909, such as `torch_tensorrt.save` keeping the
compiled graph by default.

## What is wrong today

The TensorRT backend page has drifted away from Torch-TensorRT, the
project that builds and ships the TensorRT delegate.

- It points readers at a nightly build from early October.
- It says CUDA graph replay never runs for engines on the shared
activation scratch. Newer builds run it.
- It repeats most of the Torch-TensorRT guide, so every delegate change
is made twice.
- The backends table lists TensorRT as Linux only.

The CUDA backend page and the getting started page also disagree with
the TensorRT page and with the code: on outputs kept on the GPU, on what
happens when a planned input gets GPU memory, on CUDA versions, GPUs and
Python versions, and on Windows.

## What this change does

The TensorRT page keeps the usual backend sections, says less in each,
and shows more code:

- Features, target requirements, and development requirements with the
one-line install command.
- Export, including sending the operators TensorRT cannot compile to the
CUDA backend, and keeping inputs and outputs on the GPU.
- Runtime integration from Python, and a complete C++ project with the
commands to build and run it.
- A short list of performance options that link to the Torch-TensorRT
guide.

It describes Torch-TensorRT 2.15 and newer, and marks the two features
that need a newer build.

A second commit fixes the CUDA backend page, the getting started page
and one docstring on the points above. To keep data on the GPU, all
pages now leave inputs and outputs unplanned, which copies the least.

## What was tested

On Linux x86-64 with an H100 GPU, an earlier version of the TensorRT
page was run end to end: every Python block in order with nightly
Torch-TensorRT, and the C++ example built and run against the installed
packages.

With nightly packages, the GPU input and output recipe was run from
Python for both backends, and the CUDA page's C++ lines were built and
run. A profiler showed one GPU copy fewer per Python call than with
planned outputs.

The other fixes were checked only by reading them against the sources.
Windows was not tested.
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 27, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes, and a nightly command with two indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
@shoumikhin shoumikhin changed the title [2.15] Backport #4919 and bring the ExecuTorch delegate version and guide up to date [2.15] Bump the ExecuTorch delegate to 0.2.1 and fix the ExecuTorch guide Oct 27, 2026
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 27, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132 --extra-index-url https://pypi.org/simple

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index. The nightly command keeps PyPI as a second index,
because TensorRT's wheels and a few other dependencies are only there, the same as the README.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 27, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes, and a nightly command with two indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
shoumikhin added a commit to shoumikhin/TensorRT that referenced this pull request Oct 27, 2026
The install section showed a long command for 2.15 that named the delegate package and four
indexes, and a nightly command with two indexes. Each channel now has one command:

    pip install "torch_tensorrt[executorch]"
    pip install --pre "torch_tensorrt[executorch]" --index-url https://download.pytorch.org/whl/nightly/cu132

A release from PyPI is built for CUDA 13.2, the CUDA version of PyTorch's own PyPI wheel, and its
executorch extra brings ExecuTorch's CUDA 13.2 build and the delegate. A nightly's extra brings
the matching nightlies from the nightly index.

On Windows, PyPI has only the CPU builds of PyTorch and ExecuTorch, so the release section says to
add the PyTorch CUDA index there.

The pages also said that 2.15 re-exports in save() by default, never replays an engine on the
shared activation scratch, has no Windows delegate, and cannot pass runtime options from Python.
The 2.15 backport (pytorch#4909) brings the save() and replay fixes and the Windows delegate, and the
ExecuTorch release it pins has the Python option API, so those notes go.

The install command test let a docs command that names the delegate skip --pre. Only the old
2.15 command needed that, so the exception and its test cases go. The case that refuses a
nightly command without --pre now uses the guide's --index-url form.
Erik-Lundell pushed a commit to Erik-Lundell/executorch that referenced this pull request Oct 28, 2026
This is a draft because the page describes the intended end state of
Torch-TensorRT 2.15. Some links go to sections of the Torch-TensorRT
guide that exist only after pytorch/TensorRT#4927 merges and its docs
site is rebuilt. And the examples rely on 2.15 changes from
pytorch/TensorRT#4909, such as `torch_tensorrt.save` keeping the
compiled graph by default.

## What is wrong today

The TensorRT backend page has drifted away from Torch-TensorRT, the
project that builds and ships the TensorRT delegate.

- It points readers at a nightly build from early October.
- It says CUDA graph replay never runs for engines on the shared
activation scratch. Newer builds run it.
- It repeats most of the Torch-TensorRT guide, so every delegate change
is made twice.
- The backends table lists TensorRT as Linux only.

The CUDA backend page and the getting started page also disagree with
the TensorRT page and with the code: on outputs kept on the GPU, on what
happens when a planned input gets GPU memory, on CUDA versions, GPUs and
Python versions, and on Windows.

## What this change does

The TensorRT page keeps the usual backend sections, says less in each,
and shows more code:

- Features, target requirements, and development requirements with the
one-line install command.
- Export, including sending the operators TensorRT cannot compile to the
CUDA backend, and keeping inputs and outputs on the GPU.
- Runtime integration from Python, and a complete C++ project with the
commands to build and run it.
- A short list of performance options that link to the Torch-TensorRT
guide.

It describes Torch-TensorRT 2.15 and newer, and marks the two features
that need a newer build.

A second commit fixes the CUDA backend page, the getting started page
and one docstring on the points above. To keep data on the GPU, all
pages now leave inputs and outputs unplanned, which copies the least.

## What was tested

On Linux x86-64 with an H100 GPU, an earlier version of the TensorRT
page was run end to end: every Python block in order with nightly
Torch-TensorRT, and the C++ example built and run against the installed
packages.

With nightly packages, the GPU input and output recipe was run from
Python for both backends, and the CUDA page's C++ lines were built and
run. A profiler showed one GPU copy fewer per Python call than with
planned outputs.

The other fixes were checked only by reading them against the sources.
Windows was not tested.
@shoumikhin shoumikhin changed the title [2.15] Bump the ExecuTorch delegate to 0.2.1 and fix the ExecuTorch guide [2.15] Bump the ExecuTorch delegate to 0.2.1, and bring the ExecuTorch docs and build fix from #4927 Oct 28, 2026
JacobSzwejbka pushed a commit to pytorch/executorch that referenced this pull request Oct 28, 2026
This is a draft because the page describes the intended end state of
Torch-TensorRT 2.15. Some links go to sections of the Torch-TensorRT
guide that exist only after pytorch/TensorRT#4927 merges and its docs
site is rebuilt. And the examples rely on 2.15 changes from
pytorch/TensorRT#4909, such as `torch_tensorrt.save` keeping the
compiled graph by default.

## What is wrong today

The TensorRT backend page has drifted away from Torch-TensorRT, the
project that builds and ships the TensorRT delegate.

- It points readers at a nightly build from early October.
- It says CUDA graph replay never runs for engines on the shared
activation scratch. Newer builds run it.
- It repeats most of the Torch-TensorRT guide, so every delegate change
is made twice.
- The backends table lists TensorRT as Linux only.

The CUDA backend page and the getting started page also disagree with
the TensorRT page and with the code: on outputs kept on the GPU, on what
happens when a planned input gets GPU memory, on CUDA versions, GPUs and
Python versions, and on Windows.

## What this change does

The TensorRT page keeps the usual backend sections, says less in each,
and shows more code:

- Features, target requirements, and development requirements with the
one-line install command.
- Export, including sending the operators TensorRT cannot compile to the
CUDA backend, and keeping inputs and outputs on the GPU.
- Runtime integration from Python, and a complete C++ project with the
commands to build and run it.
- A short list of performance options that link to the Torch-TensorRT
guide.

It describes Torch-TensorRT 2.15 and newer, and marks the two features
that need a newer build.

A second commit fixes the CUDA backend page, the getting started page
and one docstring on the points above. To keep data on the GPU, all
pages now leave inputs and outputs unplanned, which copies the least.

## What was tested

On Linux x86-64 with an H100 GPU, an earlier version of the TensorRT
page was run end to end: every Python block in order with nightly
Torch-TensorRT, and the C++ example built and run against the installed
packages.

With nightly packages, the GPU input and output recipe was run from
Python for both backends, and the CUDA page's C++ lines were built and
run. A profiler showed one GPU copy fewer per Python call than with
planned outputs.

The other fixes were checked only by reading them against the sources.
Windows was not tested.

Co-authored-by: Anthony Shoumikhin <anthony@shoumikh.in>
Since the pytorch#4821 backport, every .pte saved from this branch stores its
engines as named data (blob tag TR03). The 0.2.0 delegate published with
the first 2.15.0 release candidate predates that and refuses these files
with only "failed to parse TensorRT blob". A delegate rebuilt from this
branch at the same version would look identical to pip, so an
environment that already has the old one would keep it.

Bump the delegate to 0.2.1, so pip sees it as newer than 0.2.0 and
replaces the old delegate wherever it installs one. The README's build
recipe uses the new version.
The guide on this branch installs a release with one command:

    pip install "torch_tensorrt[executorch]"

Once this branch pins an ExecuTorch release, its PyPI wheel's executorch
extra brings ExecuTorch's CUDA 13.2 build and the delegate built for
CUDA 13.2, so naming either in the command would only let them drift
apart. A new test checks that the guide's release install is exactly
this command.

The install command test also gains a case that refuses a command which
names the nightly index without --pre.
The saving section still printed a nightly install command and said the
integration supports Linux. On this branch that command takes main's
nightly in a fresh environment, and with 2.15 installed it adds ExecuTorch
but no delegate, because a release's extra leaves the delegate out. It
also disagreed with the guide linked just above it.

Replace the platform sentence and the command with a pointer to the
guide's Installation section.
The 2.15 guide and the pages it links disagreed with each other and with the code:

- The saving section said each delegate of a coalesced program runs on its own CUDA stream, and
  that a zero-copy TensorRT delegate returns before its engine finishes. Both delegates use the
  calling thread's default stream, and the TensorRT delegate waits for its work. The section
  now says so, and names the one case that needs a sync: the CUDA backend running last.
- Two pages described an overwritten coalesced weight file differently. Both now say it can give
  wrong answers or fail to load.
- The comparison table said a .pt2 runs on Windows. AOTInductor packaging is Linux only.
- The guide promised dynamic shapes in a section that has none, and its example list missed two
  examples.
- A coalesced program's data file is passed when loading from Python, the JetPack build joins
  the builds without a delegate, and ATen, KV and SM are spelled out.
- The saving page's format list no longer calls ExecuTorch output Linux-only.
- The guide said a host tensor passed to a device-resident program only costs a copy, and that
  Python cannot run a program with unplanned outputs. ExecuTorch's Python runtime refuses the host
  tensor and allocates device memory for unplanned outputs, so the guide and the runtime README
  now say so.
- Two save() error messages said the integration supports only Linux. It supports Windows x86-64
  too.
On this branch the guide was picked after the change that stops save()
from retracing, so its examples still pass retrace=False and
arg_inputs, and the text presents them as two extra arguments save
needs. For the ExecuTorch format save() now keeps the compiled graph as
it is and needs no example inputs. The examples now use one extra
argument, output_format="executorch", as on main.

The CUDA graph paragraph also said that engines on the shared
activation scratch never replay. With pytorch#4833 they do, each with its own
buffers for its inputs and outputs, and they record again when the
pool moves to a larger buffer. The paragraph now says so.
…and core

The delegate can build ExecuTorch's shared extension_cuda library from a checkout when nothing
else provides it. Built standalone, as the delegate README's archive-only recipe does, that build
failed twice. First it stopped compiling on c10/util/safe_numerics.h, because the include list it
was given held only the checkout's parent folder, not ExecuTorch's own copy of c10. With that
fixed, it stopped linking with "cannot find -lexecutorch_core": extension_cuda links
executorch_core, and only an ExecuTorch build in the same project defines that target.

The source-build branch now adds the c10 folder to the include list, as ExecuTorch's own build
does, and, when no executorch_core target exists, defines one from the prebuilt core library. It
is the same libexecutorch_core.a the delegate already links further down, and
EXECUTORCH_CORE_LIBRARY, given on the command line or by a parent project, still overrides it. A project that adds ExecuTorch with
add_subdirectory already has the target, so nothing changes there.

Tested on this branch, Linux aarch64 with CUDA 13.2, following the archive-only
recipe with no prebuilt extension_cuda: the build fails on the missing c10 header
before this change, fails to find executorch_core with only the header half of it,
and builds after it. A new unit test checks the branch keeps both pieces, and fails
without this change.

Backported from pytorch#4927 on main, where it landed with the ExecuTorch docs rewrite.
… main

The ExecuTorch guide, the .pte saving section, the three ExecuTorch
READMEs, the example scripts and a few docstrings had drifted from the
code and from each other. pytorch#4927 rewrote them on main. The delegate and
export code on this branch are the same as on main, so the same text
applies here, with these changes for this branch:

- The runtime wheel README builds version 0.2.1 of the delegate against
  this branch's ExecuTorch pin, 1.6.0.dev20261009, instead of main's
  0.3.0 and 1.6.0.dev20261021.
- The reference runner README checks out the ExecuTorch commit that this
  branch pins in MODULE.bazel.
- The test that the guide's release install is one command stays.

The guide's Python examples for load and process options and for
clone_outputs use ExecuTorch 1.6.0's Python runtime. This branch's
current development pin predates them; the release moves the pin to
1.6.0, which has them.

What the rewrite changes:

- One Installation section, linked from the other pages, with one
  command per channel. Releases: pip install "torch_tensorrt[executorch]".
  Nightly: pip install --pre "torch_tensorrt[executorch]" --index-url
  https://download.pytorch.org/whl/nightly/cu132.
- A complete C++ project, with the commands that build and run it.
- Sections on runtime options, weight streaming, engines in a .ptd file
  and optimization profiles.
- The device-resident recipe leaves GPU outputs unplanned, and says
  what ExecuTorch's Python runtime does with host inputs.
- save(), export() and TensorRTPartitioner docstrings now match the
  code: a target_device spec only labels a delegate's inputs and
  outputs, a process option set to False also overrides
  use_cuda_graphs, and generate_etrecord=True fails for a program with
  TensorRT engines.
- The example scripts point at the guide instead of printing their own
  install commands.
…e on PyTorch 2.15

pytorch#4921 came to this branch with a test that compiles a multi-query attention
model with torch.compile and expects the new pass to fold its two matmuls. It
fails here in every dynamo-lowering CI job:

    AssertionError: 'Folded 2 matmul(s)' not found in '... found nothing to
    fold in this graph ...'

The code is the same as on main, where the test passes. The difference is
PyTorch. With 2.15, the graph that torch.compile hands to Torch-TensorRT has
each matmul already split into expand, reshape and bmm, so there is no matmul
left to fold. With the 2.16 nightly the same graph still has aten.matmul.
torch.export keeps aten.matmul on both, so the pass works there and its other
tests pass.

This branch requires PyTorch 2.15, so here the option never has an effect
under torch.compile. The docs, docstrings and the pass's log message now say
that instead of naming only use_distributed_mode_trace, and the test is
skipped below PyTorch 2.16.

Reproduced on Linux x86-64 with an A100 and TensorRT 11.4, from this branch's
Python sources: the test file gives 1 failed, 21 passed with torch
2.15.0+cu130, and 22 passed with torch 2.16.0.dev20261028+cu130. After this
change: 21 passed, 1 skipped with 2.15, and 22 passed with the nightly.
@shoumikhin shoumikhin changed the title [2.15] Bump the ExecuTorch delegate to 0.2.1, and bring the ExecuTorch docs and build fix from #4927 [2.15] Bump the ExecuTorch delegate to 0.2.1, bring the ExecuTorch docs and build fix from #4927, and fix the #4921 test Oct 29, 2026
@github-actions github-actions Bot added component: lowering Issues re: The lowering / preprocessing passes component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Oct 29, 2026
@shoumikhin shoumikhin changed the title [2.15] Bump the ExecuTorch delegate to 0.2.1, bring the ExecuTorch docs and build fix from #4927, and fix the #4921 test [2.15] Bump the ExecuTorch delegate to 0.2.1, bring #4927 and #4928, and fix the #4921 test Oct 29, 2026
@github-actions github-actions Bot added the component: conversion Issues re: Conversion stage label Oct 29, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed component: api [C++] Issues re: C++ API component: api [Python] Issues re: Python API component: build system Issues re: Build system component: conversion Issues re: Conversion stage component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes component: tests Issues re: Tests documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant