Skip to content

Let Elements unpack type-level DSL results with an IntTuple domain - #5101

Open
nitishagar wants to merge 1 commit into
facebook:mainfrom
nitishagar:fix/elements-dsl-broadcast-4977
Open

nitishagar wants to merge 1 commit into
facebook:mainfrom
nitishagar:fix/elements-dsl-broadcast-4977

Conversation

@nitishagar

Copy link
Copy Markdown
Contributor

Fixes #4977

Problem

Elements[...] could not unpack the result of a type-level DSL function:

def f[B1: IntTuple, B2: IntTuple, Dim: IntVar](
    x: Tensor[B1], y: Tensor[[*Elements[B2], Dim]]
) -> Tensor[[*Elements[broadcast(B1, B2)], Dim]]: ...
# ERROR: Function call cannot be used in annotations
# ERROR: `Elements[...]` requires an `IntTuple` or integer tuple, got `Unknown`

As stroxler noted in the issue, broadcast should be able to expand through Elements — a @type_shape_dsl_function declares its result domain, and an IntTuple-domain call should be legal wherever an IntTuple carrier is.

Change

Two gaps, fixed in Elements' argument handling (mirroring how the Tensor shape path already treats DSL calls):

  1. The Elements[...] argument was parsed under a fixed TypeExpression-rooted context whose rules refuse function calls. It now inherits the ambient annotation context through the type-argument parent, so a DSL call parses wherever the surrounding annotation already allows one.
  2. The carrier validator had no type-level-DSL arm. It now accepts a Type::TypeLevelDslCall whose result_domain() is IntTuple.

Because the allowance is inherited rather than granted, the design intent that DSL calls are return-annotation-only is preserved: Elements[broadcast(...)] stays rejected inside parameter annotations, aliases, and globals (pinned by a new test), and calls with a non-IntTuple result domain keep failing the existing requires an IntTuple check (also newly pinned). Symbolic-rank and malformed-argument rejections are untouched.

Scope note for reviewers: at a call site, a return type carried by a DSL call reveals Tensor[IntTuple] — the same result the non-Elements DSL-call path produces today (def g[B1, B2](x: Tensor[B1], y: Tensor[B2]) -> Tensor[broadcast(B1, B2)] behaves identically). Concrete materialization of broadcast shapes at call sites is a pre-existing, separate behavior of the DSL-call evaluation path; this change makes the Elements path consistent with it rather than extending it.

Test plan

  • New test: the issue's signature with Elements[broadcast(B1, B2)] in the return annotation → no errors (written failing-first; failed with exactly the issue's two invalid-annotation errors before the fix)
  • New tests pinning the boundary: rejection under param/alias/global roots; non-IntTuple-domain DSL results still rejected with the existing message
  • cargo test -p pyrefly --lib test::shape_dsl: 355 passed, 0 failed; full --skip test::lsp suite: 7741 passed, 0 failed (the two missing-source LSP interaction failures also fail on unmodified main)
  • Formatting and linting clean

@meta-cla meta-cla Bot added the cla signed label Oct 5, 2026
@github-actions github-actions Bot added the size/s label Oct 5, 2026
@meta-codesync

meta-codesync Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

This pull request has been imported. If you are a Meta employee, you can view this in D123347321. (Because this pull request was imported automatically, there will not be any future comments.)

@github-actions

This comment has been minimized.

@github-actions
github-actions Bot requested a review from kinto0 October 5, 2026 09:32
@kinto0 kinto0 assigned stroxler and unassigned kinto0 Oct 8, 2026
@kinto0
kinto0 requested a review from stroxler October 8, 2026 00:29
@kinto0

kinto0 commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@stroxler gets back tomorrow and can take a look after that. thanks for your patience!!

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

@stroxler

stroxler commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Thanks! This looks good to me

@stroxler stroxler left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay this looks good to me, but can you rebase?

It looks like there are some merge conflicts, and I think an error message relevant to the test changed. I've been landing a lot of minor improvements lately, but I just wrapped up that work so I think things should be more stable now.

`Elements[broadcast(B1, B2)]` in a return annotation was rejected twice
over: the Elements argument was parsed under a fixed type-expression
context whose rules refuse function calls outright, and its validator
knew nothing of type-level DSL calls (issue facebook#4977). A `@type_shape_dsl_function`
such as `broadcast` has a declared `IntTuple` result domain, so its call
should be legal wherever an `IntTuple` carrier is — exactly how the
Tensor shape path already treats it.

Elements' argument is now parsed with the ambient annotation context
inherited through the type-argument parent (mirroring the Tensor shape
argument path), and the carrier validator accepts a type-level DSL call
whose result domain is `IntTuple`. Because the allowance is inherited
rather than granted, DSL calls inside Elements stay rejected in
parameter annotations, aliases, and globals, and calls with a
non-`IntTuple` result domain keep failing the existing `requires an
IntTuple` check; symbolic-rank and malformed-argument rejections are
untouched.
@nitishagar
nitishagar force-pushed the fix/elements-dsl-broadcast-4977 branch from 8e49dec to c8a34d6 Compare October 10, 2026 11:12
@nitishagar

Copy link
Copy Markdown
Contributor Author

@stroxler Thanks for the review — I've rebased onto latest main and fixed the test affected by the changed error message.

@github-actions github-actions Bot added size/s and removed size/s labels Oct 10, 2026
@github-actions

Copy link
Copy Markdown

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[tensor-shapes] Combination of Elements and DSL functions

3 participants