Skip to content

fix(qwen): remap legacy single-file VL encoder keys under transformers 5.x - #9333

Merged
lstein merged 2 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/qwen-vl-singlefile-key-remap
Jul 5, 2026
Merged

fix(qwen): remap legacy single-file VL encoder keys under transformers 5.x#9333
lstein merged 2 commits into
invoke-ai:mainfrom
Pfannkuchensack:fix/qwen-vl-singlefile-key-remap

Conversation

@Pfannkuchensack

Copy link
Copy Markdown
Collaborator

Summary

Qwen2_5_VLForConditionalGeneration._checkpoint_conversion_mapping is empty in transformers 5.x, so the if key_mapping: guard skipped remapping and ComfyUI single-file checkpoints (visual.*, model.*) failed to load: their keys were reported as unexpected while model.visual.* params stayed as meta tensors. Fall back to the legacy mapping when transformers no longer provides one. The negative lookahead keeps new-layout checkpoints untouched.

Related Issues / Discussions

N/A

QA Instructions

  1. Obtain a ComfyUI-style single-file Qwen2.5-VL encoder checkpoint (e.g. qwen_2.5_vl_7b_fp8_scaled.safetensors from the starter models) that uses the legacy key layout (visual.*, model.layers.*).
  2. Install it as a Qwen VL encoder and run a qwen_image_text_encoder invocation.
  3. Before this fix: load fails with RuntimeError: Failed to load all parameters from checkpoint. Meta tensors remain: ['model.visual.patch_embed.proj.weight', ...] plus a 728 unexpected keys warning.
  4. After this fix: the encoder loads cleanly with no unexpected-key warning and no meta tensors remaining; text encoding completes normally.

Verified against the real checkpoint header (1446 keys): 730 → model.language_model.*, 714 → model.visual.*, 1 → lm_head.weight, 1 → scaled_fp8 (fp8 metadata, stripped before remap). No weight keys left unmapped.

Merge Plan

Standard merge — no DB/schema changes, no migrations.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

…s 5.x

Qwen2_5_VLForConditionalGeneration._checkpoint_conversion_mapping is empty
in transformers 5.x, so the `if key_mapping:` guard skipped remapping and
ComfyUI single-file checkpoints (`visual.*`, `model.*`) failed to load:
their keys were reported as unexpected while `model.visual.*` params stayed
as meta tensors. Fall back to the legacy mapping when transformers no longer
provides one. The negative lookahead keeps new-layout checkpoints untouched.
@github-actions github-actions Bot added python PRs that change python files backend PRs that change backend files labels Jul 5, 2026

@lstein lstein left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as advertised.

@lstein

lstein commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Supersedes #9334

@lstein
lstein merged commit 6ca505f into invoke-ai:main Jul 5, 2026
17 checks passed
@Pfannkuchensack
Pfannkuchensack deleted the fix/qwen-vl-singlefile-key-remap branch July 5, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend PRs that change backend files python PRs that change python files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants