Skip to content

Expose use_associative_scan in ZambaConfig - #48470

Closed
lorenzozanee wants to merge 1 commit into
huggingface:mainfrom
lorenzozanee:fix/zamba-use-associative-scan
Closed

lorenzozanee wants to merge 1 commit into
huggingface:mainfrom
lorenzozanee:fix/zamba-use-associative-scan

Conversation

@lorenzozanee

@lorenzozanee lorenzozanee commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

CPU CI GPU run-slow

What does this PR do?

Fixes #48080

ZambaMambaMixer hard-coded use_associative_scan=False, so ZambaConfig could not control whether torch.compile used the associative-scan path.

This adds use_associative_scan to ZambaConfig (default True, matching MambaConfig) and forwards it from the mixer instead of the hard-coded value. The associative-scan branch only runs while torch.compile is tracing (is_torchdynamo_compiling()), so eager execution is unchanged. A regression test in tests/models/zamba/test_modeling_zamba.py pins that the configured value reaches mamba_selective_scan.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@ArthurZucker
@Cyrilvallez
@albertvillanova

Code Agent Policy

  • (First-time contributors only): I confirm that this PR description and code is not written by an LLM or code agent

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: zamba

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

CI recap

Dashboard: View test results in Grafana
Latest run: 33576824417
Result: success | Grafana metrics are not available yet.

@Cyrilvallez

Copy link
Copy Markdown
Member

cc @vasqu here as well, but I don't think we want to re-add this kind of stuff do we?

@vasqu

vasqu commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

I swear I have seen another one of those PR and it failed on the test not being properly done. I already flagged that reviews on this might get delayed as its low prio

Tbh, it's ok to add and I was probably to lazy at the time of the refactor. It's a free win on perf using the same tricks as in mamba1

@lorenzozanee if you are still up for it I'd review but would like a response first before investing time here

@ArthurZucker

Copy link
Copy Markdown
Collaborator

Thanks @lorenzozanee! use_associative_scan was added to ZambaConfig in #48331, so closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zamba-7B-v1 runs slow during torch.compile due to use_associative_scan=False

4 participants