Repository navigation
End-to-end test of Gemma 3 + FA2 construction - #45760
Closed
jamesbraza wants to merge 2 commits into
Closed
jamesbraza wants to merge 2 commits into
jamesbraza wants to merge 2 commits into
Conversation
Before huggingface#45589, `flash_attention_forward` called `s_aux.to(query.dtype)` unconditionally and raised `AttributeError` on models that do not use attention sinks (see huggingface#45588). The fix guards the cast behind `s_aux is not None`, but no test exercises the sinkless branch through the real dispatcher. Add `test_flash_attention_2_forward_on_sinkless_model` to `Gemma3TextModelTest`, which loads the tiny-random sinkless Gemma3 fixture with `attn_implementation="flash_attention_2"` and runs a forward pass. Guarded with `@slow`, `@require_torch_gpu`, and `@require_flash_attn` so it only runs under `RUN_SLOW=1` with FA installed.
Drop the issue/PR-specific narrative from the docstring in favor of a summary describing what the test actually does — an end-to-end integration check of model construction and a forward pass through the flash-attention backend. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 of 4 tasks
Contributor
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gemma3 |
Member
|
Hey! This test is just running on forward on fa2, which is already tested in other integration tests! So there is no need to add it again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This small PR adds an end-to-end regression test for the fix in #45589 for #45588:
Cherry-picked from @truffle-dev's fork; verified on compute cluster with H100.
This PR adds test coverage for the fix in #45589 for #45588.
@ArthurZucker @Cyrilvallez