[gokay-ai]

Fixes #48924

OPT was filling an all-ones attention_mask for learned position ids, then passing that same tensor into create_causal_mask. Under torch.compile, SDPA then gets an explicit mask instead of is_causal=True and falls back to slower kernels.

The ones mask is only used for position ids now. create_causal_mask still sees attention_mask is None.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline and the Pull Request checks?
• Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes according to the guidelines?
• Did you write any new necessary tests?

Who can review?

Text models / attention: @ArthurZucker @Cyrilvallez @vasqu

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: opt

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 35335750928
Result: success | Grafana metrics are not available yet.

[Cyrilvallez]

create_causal_mask will create the same mask for an all 1s mask and a None mask so I'm not sure what's really the point of this PR?

[gokay-ai]

Good question — in eager mode you’re right that an all-ones 2D mask and None end up equivalent for create_causal_mask / SDPA.

The difference shows up under torch.compile / tracing. In _ignore_causal_mask_sdpa, a concrete padding tensor takes the is_tracing(padding_mask) path and forces a material attn mask (return False), so SDPA can’t use is_causal=True. Passing attention_mask=None through to create_causal_mask avoids that and keeps the FlashAttention / oneDNN path (that’s the regression in #48924).

This PR only fabricates the all-ones tensor for OPT’s learned position ids; the causal-mask call keeps None when the caller didn’t pass a mask.

[gokay-ai]

Correction on my earlier note: under torch.compile, is_tracing() is true from is_torchdynamo_compiling() even when the padding mask arg is None, so _ignore_causal_mask_sdpa declines the is_causal skip for both None and an all-ones mask. You’re right that this PR doesn’t change the SDPA path in current masking_utils.

Closing this — thanks for the catch. If #48924 still shows a real compile regression, the fix likely belongs in the shared mask/SDPA skip logic rather than OPT alone.