feat(xla): support Qwen2-VL as the representative M-RoPE VLM

[inureyes]

Parent and dependencies

• Parent epic: epic: OpenXLA multimodal / VLM architecture support (Qwen2-VL, Gemma3n, Phi4MM, Molmo, and more) #566
• Blocked by test(xla): validate a LLaVA reference architecture end to end #862 and feat(xla): support multimodal 3D RoPE positions and decode deltas #864
• Existing MLX implementation: src/vision/qwen2_vl.rs, src/vision/encoders/qwen2_vl.rs, src/vision/processors/qwen2_vl.rs, src/multimodal/qwen_vl.rs, and src/models/qwen2_vl.rs

Problem

Qwen2-VL is the first representative family that combines packed visual sequences, temporal/spatial patch grids, 2D rotary embeddings inside the vision transformer, spatial token merging, and 3-axis M-RoPE in the language model. Treating it as a minor variant of the LLaVA path would miss sequence-boundary and position-state invariants.

Goal

Support a pinned Qwen2-VL checkpoint end to end on the XLA backend, including processor/grid metadata, IREE vision encoder and merger, M-RoPE language prefill/decode, CLI, and continuous-batch server execution.

Graph and preprocessing contract

• Reuse the canonical decoded-media/prepared-prefill seam and M-RoPE state from feat(xla): support multimodal 3D RoPE positions and decode deltas #864.
• Define a static-shape visual graph contract for flattened temporal patches, grid_thw, actual patch count, and packed-sequence boundaries. Choose and document padding/bucketing; do not recompile an unbounded graph for every image resolution.
• Preserve media boundaries in packed attention. Patches from different images/frames must not attend across a cu_seqlens segment unless the reference implementation does so.
• Emit projected visual tokens in the exact order expected by placeholder expansion and M-RoPE position construction after spatial_merge_size is applied.

Required implementation

• Reuse processor resize/rescale/normalize and smart-resolution rules from src/vision/processors/qwen2_vl.rs; share the implementation or add parity fixtures rather than copying constants.
• Emit the Qwen2-VL visual patch projection, 2D vision RoPE, packed full-attention blocks, normalization/MLP layers, and patch merger through StableHLO/IREE.
• Map all vision weights with explicit shape/layout checks, including combined QKV and merger tensors. Include patch/temporal/merge sizes, hidden/head dimensions, grid capacity, dtype, and vision depth in artifact compatibility identity.
• Generate language-model 3D positions and rope_delta with the existing multimodal/qwen_vl.rs rules, and route them through feat(xla): support multimodal 3D RoPE positions and decode deltas #864 for prefill and every decode step.
• Support image input first; support video only if the existing server request representation and the pinned checkpoint fixture can validate temporal grids without broadening this issue. Otherwise reject video explicitly and file/follow the dedicated scope.
• Integrate model-family detection, filtered host metadata/processor loading, XLA capabilities, CLI, and server admission without loading a duplicate full MLX decoder.
• Preserve text-only Qwen2 execution through the ordinary token-prefill path when no media is supplied.

Non-goals

• Qwen2.5-VL windowed vision attention.
• Qwen3-VL deepstack feature injection or Qwen3-VL-MoE.
• Arbitrary dynamic visual shapes or silent fallback to the MLX vision encoder.

Validation

• Pin one Qwen2-VL checkpoint/revision and deterministic image/prompt fixture.
• Compare processed pixels, grid_thw, packed boundaries, vision RoPE coordinates, selected vision-block outputs, merged visual tokens, language M-RoPE positions/delta, prefill logits/KV, and greedy tokens against the existing MLX path and an independent eager oracle.
• Cover one image, multiple images if supported, no-media text fallback, grid at a bucket boundary, over-capacity grid, malformed placeholders, and cross-image attention isolation.
• Run non-streaming CLI and mixed text/image continuous-batch server tests, including cancellation and slot reuse with distinct rope_delta values.
• Execute on at least one production-relevant IREE target and run test(xla): validate a LLaVA reference architecture end to end #862 plus all text-only XLA regression gates.

Acceptance criteria

• Qwen2-VL vision encoder and merger execute through IREE with explicit static-shape bucketing/padding.
• Packed visual sequence boundaries and post-merge token ordering match the reference.
• M-RoPE prefill coordinates and decode delta match the existing MLX implementation.
• Final deterministic output is token-exact on the pinned fixture.
• Image CLI and continuous-batch server paths pass without a duplicate full decoder.
• Unsupported grids/modalities fail explicitly; no MLX vision fallback occurs silently.
• Text-only Qwen2 remains token-exact.