Reject path traversal in the `transformers_weights` config field by LinZiyuu · Pull Request #46890 · huggingface/transformers

LinZiyuu · 2026-06-25T15:20:58Z

from_pretrained resolves which checkpoint file to load via _get_resolved_checkpoint_files. When a model's config.json carries a transformers_weights field, that untrusted value is joined onto the model directory and opened after only a suffix check (it must end .safetensors / .safetensors.index.json, or equal adapter_model.bin) — with no .. / absolute-path / containment validation (modeling_utils.py:597). Because os.path.join drops the prefix on an absolute component and .. walks out of the directory, a malicious local model directory makes AutoModel.from_pretrained(...) open a .safetensors-format file outside the model directory (path traversal, CWE-22), with no trust_remote_code.

The escape base is pretrained_model_name_or_path — the directory the caller chose to load, not a config field — so the file is expected to live inside it; the same containment mistake was fixed on the write side for chat templates (#46191) and Bark voice presets (#46237), and this is the read side in the core loader. The read is constrained to .safetensors-format files (no arbitrary-text disclosure) and weights_only=True means no code execution, but a crafted local model can still read an out-of-directory checkpoint and acts as a file-existence oracle — in multi-tenant model hosting this can surface another tenant's weights.

The fix validates that the resolved transformers_weights path stays inside the model directory (rejecting .. and absolute paths), leaving the sibling branches (which use hard-coded constants) untouched. A legitimate in-repo filename still loads.

AI assistance was used while drafting this change; I reviewed every line and ran the tests below.

What does this PR do?

Adds a containment check so a transformers_weights value in config.json cannot make from_pretrained open a .safetensors file outside the model directory (path traversal, CWE-22, no trust_remote_code), matching the existing write-side traversal fixes (#46191, #46237).

Tests run (torch==2.12.1):

pytest tests/utils/test_modeling_utils.py -k test_transformers_weights_config_field_rejects_path_traversal
# 1 passed (an injected ../ or absolute transformers_weights is rejected;
#           a legitimate in-repo value still loads)
ruff check / ruff format --check    # clean

Reported via huntr: https://huntr.com/bounties/bdd08581-3537-4ca1-bd8c-de724529617a

Fixes # (security hardening)

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline, Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?

Who can review?

model loading (from pretrained): @Cyrilvallez

Rocketknight1 · 2026-06-29T12:51:56Z

Hey! In #46237 I think we used a different approach so it doesn't break when loading from snapshot dirs, right?

LinZiyuu · 2026-06-29T14:07:59Z

Good catch — you're right, thanks. I'd used os.path.realpath, which follows symlinks, so it would reject loads from a populated HF cache: the snapshot files are symlinks into the sibling blobs/ dir and resolve outside the snapshot dir. Confirmed that locally.

Switched to the same lexical approach as #46237. os.path.abspath + os.path.commonpath (no realpath/resolve, so symlinks aren't followed). It still blocks .. and absolute paths but leaves snapshot loads intact. Added a regression test that loads a symlinked snapshot-style weights file so it can't regress.

Rocketknight1 · 2026-06-30T12:09:03Z

Looks good! One last point: Can we get rid of the big Claude comments? They can just be removed entirely, or reduced to a single line that explains we're catching path traversal bugs.

Also this touches core code so cc @Cyrilvallez @vasqu if you're okay with it!

`transformers_weights` in config.json names the checkpoint file to load. During `from_pretrained` it was joined onto the model directory and opened after only a suffix check, with no containment validation, so a malicious local model directory could make `AutoModel.from_pretrained(...)` read a `.safetensors` file outside the model directory (path traversal, CWE-22), with no `trust_remote_code`. Validate that the resolved path stays inside the model directory using a purely lexical `os.path.abspath` + `os.path.commonpath` check (not realpath/Path.resolve, so it does not follow symlinks and does not break loading from HF-cache snapshot dirs whose files symlink into a sibling blobs/ directory) — same approach as huggingface#46237. A legitimate in-repo filename still loads. Regression tests: rejects `..`/absolute `transformers_weights`, and still loads a symlinked snapshot-style weights file.

LinZiyuu · 2026-06-30T12:51:04Z

Done — removed the comments. Let me know if anything else is needed!

HuggingFaceDocBuilderDev · 2026-07-01T11:17:05Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Cyrilvallez

Alright

github-actions · 2026-07-08T09:39:32Z

CI recap

Dashboard: View test results in Grafana
Latest run: 28930699035:2
Result: success | Jobs: 15 | Tests: 172,360 | Failures: 0 | Duration: 15h 34m

LinZiyuu · 2026-07-08T09:43:36Z

Thanks for the review, @Cyrilvallez! Could you merge it when you have a moment? I don't have write access. Appreciate it!

Rocketknight1 · 2026-07-08T12:13:25Z

Merging!

…gingface#46890) * Reject path traversal in the `transformers_weights` config field `transformers_weights` in config.json names the checkpoint file to load. During `from_pretrained` it was joined onto the model directory and opened after only a suffix check, with no containment validation, so a malicious local model directory could make `AutoModel.from_pretrained(...)` read a `.safetensors` file outside the model directory (path traversal, CWE-22), with no `trust_remote_code`. Validate that the resolved path stays inside the model directory using a purely lexical `os.path.abspath` + `os.path.commonpath` check (not realpath/Path.resolve, so it does not follow symlinks and does not break loading from HF-cache snapshot dirs whose files symlink into a sibling blobs/ directory) — same approach as huggingface#46237. A legitimate in-repo filename still loads. Regression tests: rejects `..`/absolute `transformers_weights`, and still loads a symlinked snapshot-style weights file. * clearer --------- Co-authored-by: Cyril Vallez <cyril.vallez@gmail.com>

…es when probing the expert layout (#15386) ## Summary `gpt_oss_checkpoint_has_fused_experts` decides between the fused and the split (pre-quantized) bitsandbytes expert layout by reading tensor names from the checkpoint. `_checkpoint_tensor_dtypes` only looked at the standard filenames (`model.safetensors`, its index, `pytorch_model.bin`, its index), but Transformers loads `config.transformers_weights` first whenever it is set, and never falls back to the standard names in that case: - transformers 4.57.6: `PreTrainedModel.from_pretrained` reads `getattr(config, "transformers_weights", None)` and `_get_resolved_checkpoint_files` uses it ahead of every standard name for local directories and Hub repos (`modeling_utils.py` lines 918-921 and 1010-1012, call site at 4869). - transformers main (5.x): same behaviour in `_get_resolved_checkpoint_files` (`modeling_utils.py` lines 556-590 and 636-638, call site at 4259), plus a path traversal check added in huggingface/transformers#46890. So a repo that has a split `model.safetensors` next to a fused `chosen.safetensors` with `"transformers_weights": "chosen.safetensors"` was probed as split, which routes the load onto the split-expert classes and skips the fused-layout float16 refusal in `loader.py`. The reverse (fused standard file, split explicit file) was probed as fused. ## Change - `_checkpoint_tensor_dtypes` takes `transformers_weights`. When it names a `*.safetensors` file or a `*.safetensors.index.json` index, only that file is read (no variant, no `use_safetensors`, no `.bin` fallback, no standard-name Hub metadata shortcut), matching Transformers. - A name that escapes the model directory returns "unknown", since newer Transformers rejects it. - A name that is not a safetensors file or index keeps the existing order (Transformers raises on it at load anyway). - `gpt_oss_checkpoint_has_fused_experts` forwards `config.transformers_weights`. Configs without the field behave exactly as before. ## Reachability None of the public gpt-oss configs I checked set the field (`openai/gpt-oss-20b`, `openai/gpt-oss-120b`, `unsloth/gpt-oss-20b-unsloth-bnb-4bit`, `unsloth/gpt-oss-20b-bnb-4bit`, `unsloth/gpt-oss-20b-BF16`), and `PretrainedConfig.save_pretrained` deletes it before writing. Only a hand-edited config can carry it, so this is a low-severity consistency fix. ## Tests New `tests/test_gpt_oss_transformers_weights_layout.py` (CPU, tiny generated safetensors): | Test | base (origin/main) | this branch | | --- | --- | --- | | `test_explicit_file_decides_the_layout[split_standard_fused_chosen]` | FAIL | PASS | | `test_explicit_file_decides_the_layout[fused_standard_split_chosen]` | FAIL | PASS | | `test_explicit_index_decides_the_layout` | FAIL | PASS | | `test_explicit_file_ignores_variant_and_use_safetensors` | FAIL | PASS | | `test_invalid_or_escaping_explicit_names_never_read_other_files` | FAIL | PASS | | `test_hub_probe_reads_the_explicit_file` | FAIL | PASS | The existing gpt-oss layout suites (`test_gpt_oss_bnb_checkpoint_layout.py`, `test_gpt_oss_bf16_load_in_4bit.py`, `test_gpt_oss_fused_4bit_checkpoint_roundtrip.py`) still pass. I also checked the probe against Transformers' own `_get_resolved_checkpoint_files` (5.17.0) on both layouts: the probe now agrees with the file Transformers actually loads. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

LinZiyuu force-pushed the fix-transformers-weights-path-traversal branch from 188e7bb to 070134b Compare June 29, 2026 13:58

LinZiyuu force-pushed the fix-transformers-weights-path-traversal branch from 070134b to 260862e Compare June 30, 2026 12:42

clearer

32f3ee0

Cyrilvallez approved these changes Jul 8, 2026

View reviewed changes

Rocketknight1 enabled auto-merge July 8, 2026 12:13

Rocketknight1 added this pull request to the merge queue Jul 8, 2026

github-merge-queue Bot removed this pull request from the merge queue due to no response for status checks Jul 8, 2026

Cyrilvallez merged commit bce8fd0 into huggingface:main Jul 8, 2026
106 checks passed

danielhanchen mentioned this pull request Oct 26, 2026

gpt-oss: read config.transformers_weights before the standard filenames when probing the expert layout unslothai/unsloth#15386

Merged

Navigation Menu

Repository navigation

Reject path traversal in the `transformers_weights` config field - #46890

Reject path traversal in the `transformers_weights` config field#46890
Cyrilvallez merged 2 commits into
huggingface:mainfrom
LinZiyuu:fix-transformers-weights-path-traversal

LinZiyuu commented Jun 25, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Rocketknight1 commented Jun 29, 2026

Uh oh!

LinZiyuu commented Jun 29, 2026

Uh oh!

Rocketknight1 commented Jun 30, 2026

Uh oh!

LinZiyuu commented Jun 30, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Jul 1, 2026

Uh oh!

Cyrilvallez left a comment

Uh oh!

github-actions Bot commented Jul 8, 2026

Uh oh!

LinZiyuu commented Jul 8, 2026

Uh oh!

Rocketknight1 commented Jul 8, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

LinZiyuu commented Jun 25, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Who can review?

Uh oh!

Rocketknight1 commented Jun 29, 2026

Uh oh!

LinZiyuu commented Jun 29, 2026

Uh oh!

Rocketknight1 commented Jun 30, 2026

Uh oh!

LinZiyuu commented Jun 30, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Jul 1, 2026

Uh oh!

Cyrilvallez left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 8, 2026

CI recap

Uh oh!

LinZiyuu commented Jul 8, 2026

Uh oh!

Rocketknight1 commented Jul 8, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

LinZiyuu commented Jun 25, 2026 •

edited by github-actions Bot

Loading