Repository navigation
Expose use_associative_scan in ZambaConfig - #48470
lorenzozanee wants to merge 1 commit into
Conversation
|
[For maintainers] Suggested jobs to run (before merge) run-slow: zamba |
CI recapDashboard: View test results in Grafana |
|
cc @vasqu here as well, but I don't think we want to re-add this kind of stuff do we? |
|
I swear I have seen another one of those PR and it failed on the test not being properly done. I already flagged that reviews on this might get delayed as its low prio Tbh, it's ok to add and I was probably to lazy at the time of the refactor. It's a free win on perf using the same tricks as in mamba1 @lorenzozanee if you are still up for it I'd review but would like a response first before investing time here |
|
Thanks @lorenzozanee! |
What does this PR do?
Fixes #48080
ZambaMambaMixerhard-codeduse_associative_scan=False, soZambaConfigcould not control whethertorch.compileused the associative-scan path.This adds
use_associative_scantoZambaConfig(defaultTrue, matchingMambaConfig) and forwards it from the mixer instead of the hard-coded value. The associative-scan branch only runs whiletorch.compileis tracing (is_torchdynamo_compiling()), so eager execution is unchanged. A regression test intests/models/zamba/test_modeling_zamba.pypins that the configured value reachesmamba_selective_scan.Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@ArthurZucker
@Cyrilvallez
@albertvillanova
Code Agent Policy