Repository navigation
🚨 [ROCm] gpt-oss: route FA3 to aiter-flash-attn, generate ROCm fixtures - #46837
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
vasqu
left a comment
vasqu
left a comment
There was a problem hiding this comment.
Have a few comments, especially hesitant on the tests side because we change a few things that would break the original cuda fixtures so gotta be a bit careful
71f135c to
cae1c1f
Compare
Noticed significanlty different outputs across eager vs fa
|
This PR allows multiple different names for the same argument (especially for sinks): #45153. |
# Conflicts: # src/transformers/models/gpt_oss/modeling_gpt_oss.py # src/transformers/models/gpt_oss/modular_gpt_oss.py # src/transformers/models/openai_privacy_filter/modeling_openai_privacy_filter.py # tests/models/gpt_oss/test_modeling_gpt_oss.py # tests/test_modeling_common.py
|
run-slow: gpt_oss, openai_privacy_filter |
vasqu
left a comment
vasqu
left a comment
There was a problem hiding this comment.
Looks overall good just smaller comments
|
run-slow: gpt_oss, openai_privacy_filter |
AMD CIThis comment contains models: ["models/gpt_oss", "models/openai_privacy_filter"] |
Nvidia CIThis comment contains models: ["models/gpt_oss", "models/openai_privacy_filter"] |
vasqu
left a comment
vasqu
left a comment
There was a problem hiding this comment.
Last comment but then good to merge 🫡 a hw dependent fallback would be a nice follow up but not here then
Abdennacer-Badaoui
changed the title
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gpt_oss, openai_privacy_filter |
CI recapDashboard: View test results in Grafana |
On ROCm, gpt-oss now routes the FA3-style attention to
kernels-community/aiter-flash-attn(_compatible_flash_implementations) and generates separate ROCm fixtures (non-distributed +tp_size=2). Configs that depend onkernels-community/megablocksare skipped because it doesn't ship a ROCm build for torch 2.10 (the version the AMD CI runs today) we have the ones for 2.11 and 2.12, and configs that hitkernels-community/sonic-moeare skipped since that kernel has no ROCm build yet.Fixes around 140 failing tests.