Repository navigation
Allow opt-out of linear-attention refusal and make disable_in_eval configurable in sequence parallelism (#4476) - #4522
Open
Rajan-vish wants to merge 1 commit into
Conversation
…in_eval configurable (huggingface#4476)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4476
This PR addresses two limitations on DeepSpeed Ulysses sequence parallelism:
_refuse_recurrent_layers_under_sequence_parallelismnow checks model capability flags (supports_sequence_parallelorsupports_context_parallel) andDeepSpeedSequenceParallelConfig.sp_allow_recurrent_layers. SP-aware models (or architectures with custom recurrent state exchange wrappers) can now opt out of the refusal, while keeping the safe refusal default for stock models.disable_in_eval:Added
sp_disable_in_evaltoDeepSpeedSequenceParallelConfig(defaulting toTrueto retain current behavior, configurable via config orPARALLELISM_CONFIG_SP_DISABLE_IN_EVAL), allowing long-context evaluation to keep sequence parallelism enabled when desired.Added tests covering configuration defaults/overrides and model capability bypasses.