Repository navigation
Conversation
guarin
force-pushed
the
auto-generate-model-inits
branch
from
August 4, 2026 09:49
30c493f to
c07cbe4
Compare
…ritance (huggingface#48443) * add features * add tests * style
* docs * feedback
* docs * feedback * offload
…rch path (huggingface#48185) * Warn once when a hub-kernel function falls back to its reference PyTorch path * Name the pip distribution for causal_conv1d too * Keep the logger out of traced graphs, drop the distribution map * Trim the comments * Move the distribution name next to its only use * Say how much slower the reference path is * Reword the comment to say what the code does * Update src/transformers/integrations/hub_kernels.py Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * Move the distribution alias next to the internal path mapping --------- Co-authored-by: q <q@h> Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
* start where i was * fix expected conv shapes * much cleaner * style * continue * fix * fix device placement
* fix * also on main model * use buffer as everywhere else * norm as well * set buffer to zeros in init as well * last small fix
…d generate (huggingface#48430) [AfMoE] Standardize past_key_values argument naming across forward and generate (huggingface#48310)
remove deprec
…_mismatch` (tensor values differ (2)) (huggingface#48496) * [serge] Fix 2 integration tests for model `fsmt` failing with `output_mismatch` (tensor values differ (2)) * use get_model across tests --------- Co-authored-by: serge[bot] <serge[bot]@users.noreply.github.com> Co-authored-by: Tarek Ziade <tarek@ziade.org>
* better error msg * add back comment --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>
* fix * this way
Signed-off-by: kaixuanliu <kaixuan.liu@intel.com>
* fix: resolve nested optional kernel imports * test: clarify nested kernel fallback coverage
…r_vision_model` (huggingface#48482) * Fix sliding mask layer_idx in Gemma3/Gemma4 create_masks_for_vision_model `create_masks_for_vision_model` pads `block_sequence_ids` using the KV sizes of cache layer 0, but then calls `create_causal_mask` without `layer_idx`, which defaults to the first `full_attention` layer. On Gemma3/Gemma4 checkpoints layer 0 is a `sliding_attention` layer, so with a hybrid cache the two lengths differ: * DynamicCache: the sliding mask is built with the full layer's kv_length, so it no longer matches the sliding KV cache. * StaticCache: the full layer's kv_length is the padded `max_cache_len`, so `blockwise_overlay` indexes `block_sequence_ids` out of bounds. This raises an IndexError on CPU and triggers a device-side assert (crashing the process) on CUDA and XPU. Pin both the padding and the mask itself to the same `sliding_attention` layer. Also add a regression test covering a decode step with both a dynamic and a static hybrid cache. The existing `test_attention_mask_composition` only checks prefill with `past_key_values=None`, and the tester config places `full_attention` at layer 0, so neither exercised this path. * Cover the prefill path in the vision sliding mask regression test The crash actually happens on the first (prefill) forward: Gemma3's `prepare_inputs_for_generation` clears `token_type_ids` after the first step, so decode never reaches `create_masks_for_vision_model`. With an empty StaticCache the sliding layer reports `kv_length == q_len` while the full layer reports the padded `max_cache_len`, which is exactly the divergence that made `blockwise_overlay` read out of bounds. Rename the test accordingly and parameterize it over prefill/static, decode/static and decode/dynamic. * Cover the vision sliding mask regression on Gemma4 as well The Gemma4 checkpoints that reach create_masks_for_vision_model (use_bidirectional_attention="vision": the 31B and 26B-A4B variants) use layer_types starting with sliding_attention and sliding_window=1024, so they hit exactly the same layer_idx mismatch as Gemma3. Mirror the gemma3 test on Gemma4Vision2TextModelTest. Verified: 3 of the 4 parameterizations fail before the fix and all pass after. * Test the vision sliding mask regression through the model instead of the mask utility
* delete * fix repo * . * add video processor docs * seems like this is expected * style * fix repo * i dont understand * ah this was not needed * add docstring back where needed * self reveiw * modular has no way to stop copying docstring... * comments * unpack kwargs * fix tests * oke, at least these are same, let's keep digging * . * second . * and like this * draft * maybe * commit before i break it * i think this is the last one from varity of ropes, now to update all models * vision-only rope qwen * run fix-repo * this will take soo long to fix every model * sooooo long * ! * one by one, hopefully i fix it all by eow * fix the rest * and left-out sam * fix repo most * and now fix repo for special omni models * a bit more of fix repo * new model was merged, ci wont run * suppress TRF, it comes from `main` * wording * . * gemma4, standardize 2D position IDs tmrw * fix repo * testers were working on pure luck, love it * fix repo * failing tests * a bit more * i suppose revert, why i changed it * fix style * lost type annotations * fix Sam copied incorrectly * qwen3 vl fix * move sam positions to buffer * modular convert * what? why CI never complained?! * forgot this * add the decorators where missing * ssome comments * more * axial rope type register separately * forgot * this is way too long * i am making bad decisions, revert * return dtype is fp32 in most models * ooops revert * bit more, rest comes tmrw * i broke smth and i dont know what * need to fix repo again * skip the test for vision rope for now, will add later more * example * Revert "i am making bad decisions, revert" This reverts commit 852a0c8. * axial as type * fix repo * fix repo again * missing * gemma4 * fix * comments * new mrope with 2 sections only, no temporal! * fix repo
* noisy comment checker * tweaks * sort by size * ignore top of files * restrict to class/functions * added progress bar, cache and cut off date * tweaks * fix the 7 occurences and added to make style
* . * draft it * fix some * maybe * . * chat template in qwen style * draft sth for video sampling tests * fix * . * fix last * move around * draft * . * renamed those * fix repo deleted one line in all files * fix * the rest * merge this * typo * when will this end? * fix? * save coz now i will break stuff * rename it all, except for xx_token_id * missed these renames * fix last * revert * comments * less small private helpers * more comments * last failing tests, and i stop adding more stuff * typo * forgot about this one * why i cannot get green CI * move around to skip more tests * few more * fix * new model * here it is, opened PR on the hub * forgot modular...
`nn.DataParallel` cannot replicate or gather the forward patterns and output types modern Transformers models use, so this common test measures PyTorch's legacy DP path rather than model correctness. DataParallel is a Trainer feature, not a per-model contract, and it keeps its coverage there (tests/trainer/test_trainer.py). This removes the common test and all 66 overrides; no `src/` behavior changes.
…uggingface#48490) * Pass kwargs to the Nemotron-H Mamba2 mixer NemotronHBlock forwards **kwargs to the attention mixer but dropped them for the linear-attention branch, so kernel kwargs such as seq_idx never reached the mixer. The mixer already splats them into the fused conv1d+scan, the conv fallback and the chunk scan, and Bamba, GraniteMoEHybrid and Zamba2 all pass them through. Co-authored-by: Cursor <cursoragent@cursor.com> * Pass kwargs to the Mamba2 mixer in falcon_h1 and mamba2 too Same gap as nemotron_h: the block accepts **kwargs but drops them at the mixer, so kernel kwargs such as seq_idx never reach the kernels. In these two the model also did not pass kwargs down to the block. Also move the nemotron_h test into the tester and drop the seq_idx pop, which is not needed since the torch fallback kernels take **kwargs. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: kaixuanliu <kaixuan.liu@intel.com>
* fix failed test cases for glm5_next Signed-off-by: kaixuanliu <kaixuan.liu@intel.com> * update comment Signed-off-by: kaixuanliu <kaixuan.liu@intel.com> * update Signed-off-by: kaixuanliu <kaixuan.liu@intel.com> * update Signed-off-by: kaixuanliu <kaixuan.liu@intel.com> --------- Signed-off-by: kaixuanliu <kaixuan.liu@intel.com>
…huggingface#48521) * Point test fixtures at hf-internal-testing copies we already host For 13 of those assets the identical bytes are already stored in hf-internal-testing, so no copy is needed - only the URL changes: - 12 COCO val2017 images -> hf-internal-testing/fixtures-coco - bcn_weather.mp3 -> hf-internal-testing/dummy-audio-samples * Mirror the last two COCO test images into hf-internal-testing * more copies * More mirroring * Mirror audiobox.mp4 out of eustlb/dummy-video-dataset * Load test images through load_test_image instead of fetching them directly * Route chat-template urls and multi-line image fetches through the cache helpers * Drop a dead commented-out image load in the llava test * remove useless if * make style * Wrap the remaining pre-cached urls, keyed on the pre-fetch list
Shorten noisy OpenVINO SDPA comment; don't claim a patch scan when the diff is unresolved
* feat: adapt the load_adapter method for dtensor TP * style: ruff * feat: fail if peft version is not compatible with dtensor TP * feat: remove TP info when relevant * fix: add _tp_size attribute * chore: update the min peft version * fix: final adjustment
…le slowdown (huggingface#48331) * fix(zamba): wire use_associative_scan through ZambaConfig * Update tests/models/zamba/test_modeling_zamba.py avoid the del model before merge Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
A job with no `permissions:` block inherits whatever the repository hands out. Each block added here grants what that job's own steps need and nothing more. Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com> Co-authored-by: Pauline Bailly-Masson <155966238+paulinebm@users.noreply.github.com>
…e#48975) * Skip mask creation under torch.compile when there is no padding mask `_ignore_causal_mask_sdpa` and `_ignore_bidirectional_mask_sdpa` bailed out on `is_tracing(padding_mask)`, which is always True under `torch.compile`. As a result the 4D mask was materialized even without a padding mask, so sdpa could never dispatch to its flash/oneDNN kernels via `is_causal`. Whether a padding mask is passed at all is a static property that dynamo guards on. Only the checks reading the mask values are a data-dependent control flow, so the tracing guard is moved down to those, and `torch.export` keeps bailing out early through `is_torchdynamo_exporting()`. OPT additionally overwrote `attention_mask` with a dense all-ones mask to infer its learned positional embeddings, which then leaked into `create_causal_mask` and defeated the skip. The dense mask is now kept in a local variable. Fixes huggingface#48924 Fixes huggingface#48925 * Gate the compile skip test on torch>=2.14 and return bools from the skip helpers `is_torchdynamo_exporting()` maps to `torch.compiler.is_exporting()`, which dynamo hard-coded to `True` in its `tracing_state_functions` table, i.e. it also reported exporting under plain `torch.compile`. That was only fixed by pytorch#176499, released in torch 2.14, so on older versions the export guard still triggers under compile and the mask keeps being materialized (conservative and correct, just without the speedup). Since setup.py allows torch>=2.5, gate the new compile test with `@require_torch_greater_or_equal("2.14")` so it does not fail on older versions, and document the dependency next to the guard. Note that this cannot be worked around from user code: on 2.7.1 every tracing-state signal has identical values under `torch.compile` and strict `torch.export`, and reading `torch.compiler._is_exporting_flag` directly is unsafe because it is `False` during strict export and would bake the skip into the exported program. Also return a Python `bool` via `bool(fast_all(padding_mask))` from `_ignore_bidirectional_mask_sdpa` / `_can_skip_bidirectional_mask_xpu` instead of a 0-dim tensor from `padding_mask.all()`, matching their `-> bool` annotation and the `fast_all` helper already used by the causal path. * Reduce the diff to a single shared tracing guard Address review feedback: instead of reordering the skip helpers, keep their original structure and only replace the tracing guard. The new helper makes the export check explicit and removes the duplicated `is_tracing` calls. Verified to be decision-identical to the previous version across the full grid of padding masks, query lengths, cache offsets and local attention sizes, in eager, `torch.compile` and `torch.export`. * Move the OPT fix to a separate PR The `modeling_opt.py` change is independent of the mask skipping logic and is better reviewed on its own, so this PR is now limited to `masking_utils.py`. * Simplify the tracing guard docstring * Inline the tracing guard and keep the original comments Drops the helper introduced earlier: the condition is now inlined at the three call sites, so masking_utils.py is down to +7/-3 with every original comment restored. Also rephrases the torch 2.14 note, which was misleading: dynamo replaces calls to torch.compiler.is_exporting() by a constant True while tracing, it is not the function itself that is hard-coded. * Address review nits: single NOTE block, comment in docstring, no loop
chore: load gguf served model with chat cli
A tag is mutable: whoever controls the action's repository can move `v4` to different code, and the next run picks it up with no diff to review. Pinning to the commit SHA freezes the code that runs today. Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com> Co-authored-by: Pauline Bailly-Masson <155966238+paulinebm@users.noreply.github.com>
…ggingface#49037) * Only seed numpy in BigBird block-sparse attention during training `bigbird_block_sparse_attention` calls `np.random.seed(seed)` on every forward. The random block adjacency list it seeds is only randomized when training: in eval mode both `_bigbird_block_rand_mask` and `_bigbird_block_rand_mask_with_head` return a constant all-zero list before touching `np.random`. So during inference the call is a pure side effect that resets the global numpy RNG, and it is also a hard graph break under `torch.compile` (`np.random.seed` is a C extension that dynamo cannot trace), which makes `fullgraph=True` fail outright. Guarding it with `self.training` leaves the training path bit-for-bit identical while making eval side-effect free and fully compilable. * Propagate the BigBird numpy-seed fix to BigBirdPegasus * Shorten the numpy-seed comment to one line
…ot grant (huggingface#49055) Drop permissions reusable-workflow callers cannot grant huggingface#49046 added `pull-requests: read` to get-pr-info.yml and `actions: read` to slack-report.yml. Both are reusable workflows, capped by their callers, which only grant `contents: read`, so every calling run now fails at startup. Both jobs ran fine with `contents: read` before huggingface#49046. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s.yml (huggingface#49057) Drop pull-requests: read from check_failed_tests.yml check_failed_tests.yml is called by transformers-ci daily-ci_reusable.yml, whose callers only grant `contents: read`. huggingface#49046 added `pull-requests: read` to check_new_failures, so push/scheduled/past CI runs still fail at startup. The job ran with `contents: read` before. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
) Follow-up to huggingface#48702, which only mapped causal_conv1d/RMSNormGated/KDA on XPU. The mamba-ssm scan and selective-state layers were left CUDA-only, so on XPU they fell back to the reference PyTorch implementation of mamba2_chunk_scan, which materializes a large fp32 intermediate and OOMs on Bamba-9B. kernels-community/mamba-ssm v3 ships XPU builds that export these layers, so map them the same way as CUDA.
* Add workflow for building XPU CI Docker images * Modify DockerHub login method in CI workflow Updated DockerHub login to use OIDC connection ID and changed username. * Move everyting into build-docker-images workflow * Take review comments into account * Runs on group --------- Co-authored-by: Pauline Bailly-Masson <155966238+paulinebm@users.noreply.github.com> Co-authored-by: Yih-Dar <2521628+ydshieh@users.noreply.github.com>
* Add MPS maintainer * Add MPS device to bug report template
* add nemotron3 diarization * add repro gist * Apply suggestion from @vasqu Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * Apply suggestion from @vasqu Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * flags + decorators fix * address review comments + simplify API * remove conversion mapping * use revision * Apply suggestion from @vasqu Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com> * use memorymixin * unnecessary get_text_config * test updates * doc update * test udpate * use type validators * modular * BaseModelOutput on audio model * add comment * no need for _get_feat_extract_output_lengths * use automodel * rtol=0 explained * tmp commit * clearer Nemotron3DiarizationSpeakerCache init * Nemotron3DiarizationEncoderLayer -> Nemotron3DiarizationAudioLayer * position_ids directly * pass streaming config directly to cache * pass head config directly * add licence * doc update * clearer offline_sizes * not necessary * remove test_missing_keys * date update * update model id * fix ci (make) --------- Co-authored-by: Anton Vlasjuk <73884904+vasqu@users.noreply.github.com>
* Fix assistant masks for processors * Get rid of the weird test refactor * Add padding support and test batching too * Code simplification
* cache/commit * feedback
…face#49059) fix(ci): harden workflow files flagged on huggingface#49057 Co-authored-by: hf-security-analysis[bot] <265538906+hf-security-analysis[bot]@users.noreply.github.com>
…ace#48885) * Honor `config.output_router_logits` in the MoE VLM wrappers Four `ForConditionalGeneration` wrappers read the flag straight off `kwargs` instead of resolving it against the config, so setting `config.output_router_logits = True` did nothing and only a forward argument turned the auxiliary loss on: Qwen3-VL-MoE, GLM4V-MoE, Qwen3.5-MoE and Qwen4-Exp. Every text-only MoE honors both, Qwen3-MoE being the reference. Resolve the argument against the text config before the backbone call, as `Qwen3MoeForCausalLM` does, and pass it down explicitly. Two modular files cover all four, since Qwen3.5-MoE and Qwen4-Exp inherit the wrapper from Qwen3-VL-MoE. `Qwen3VLMoeTextConfig` had `output_router_logits` deleted in huggingface#41250 because nothing read it, and `Glm4vMoeTextConfig` never had it, inheriting from the auxiliary-loss-free `Glm4MoeConfig`. Both need it now, so restore one and add the other. * Wire the auxiliary loss through the MiniMax-M3-VL wrapper Same gap as the other four, one step further along. `MiniMaxM3VLTextForCausalLM` implements the auxiliary loss in full, while `MiniMaxM3SparseForConditionalGeneration` has no `output_router_logits` argument and no auxiliary-loss block, and neither of its output types carries `aux_loss`. Add the argument and resolve it against the text config like the others, carry `router_logits` through the backbone output instead of dropping it when the output is rebuilt, and add `aux_loss` and `router_logits` to the output types. * Test config-driven router logits in the common mixin instead of per model * Resolve output_router_logits against the config in JetMoe, use the decoder text config in the common test
* Add Nemotron Omni model * Declare patch_size, img_context_token_id, video_context_token_id, and related token fields explicitly in NemotronH_Nano_Omni_Reasoning_V3_Config * Use language model from models/nemotron_h * Remove processing script * Move audio model into same model file * Change RMSNorm to fp32 * Fix rms import * Update Radio config and model * Add Nemotron Omni in AutoConfig * Remove Nano in class * Add doc and tests * Fix config check * Fix for consistency pipeline * Change config init instead of add the allow list * Fix consistency * Isolate pixel shuffle * Add prefix for some module * Update SoundConfig and SoundEncoder SoundProjector * Isolate visual feature extract * Change MLP and SoundEncoder * Make video_patch_projection as args in Radio instead of manually patching it in omni model * Change multimodal path * Update to use parent's generate path * Update doc * Reuse MLP for soundprojection * Remove unused variable * Fix consistency check * Remove NemotronH_Omni_Reasoning_V3SoundEncoder wrap * Remove vision_model in VisionProjector forward args * Add integration test * Update for consistency * Align get_image_features get_video_features with tests * Remove config config-derived parameters from checkpoint * Remove legacy config * Remove helper * Fix for code quality test * Add final layernorm for multi token prediction * Update doc date * conditional video patch in radio * Route NemotronH-Omni sub-configs through CONFIG_MAPPING/AutoConfig * Keep parameter for campability * Update config file * Update processor * Update nemotron_h_omni modeling file * Update radio * Update tests * Add nemotron_h changes * last nits to merge! --------- Co-authored-by: Arthur <arthur.zucker@gmail.com>
Remove file and set context correctly
* fix * better
…uggingface#48839) * [serge] Fix 1 integration test for model `moshi` failing with `OOM` (other (1)) * Use MemoryCleanupMixin for Moshi integration tests; restore expected values --------- Co-authored-by: serge[bot] <serge[bot]@users.noreply.github.com> Co-authored-by: Tarek Ziade <tarek@ziade.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Auto generates all
<model>/__init__.pyfilesFollow up from huggingface#47656
This PR doesn't add the auto-generate header to the files yet to focus the diff on the actual changes. Will enable it once the PR is approved.
Similarly, the license header from the original
__init__.pyis kept to keep the diff small. Given that all__init__.pyfiles are now generated it might make sense to copy it from somewhere else. However there is no deterministic file that is present in all model directories (except__init__.py). The generation code in this PR falls back to the next file it finds in the directory if no__init__.pyis present.Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.