[sergereview]

Original CI failure

• Failure group: 2 integration tests regressed by commit b9090ae58cda (PR #47096)
• tests/models/cosmos3_omni/test_modeling_cosmos3_omni.py::Cosmos3OmniForConditionalGenerationIntegrationTest::test_small_model_integration_batched [multi-gpu] (output_mismatch, seen 5/7)

CI traceback — tests/models/cosmos3_omni/test_modeling_cosmos3_omni.py::Cosmos3OmniForConditionalGenerationIntegrationTest::test_small_model_integration_batched

(line 314)  AssertionError: Lists differ: ['use[233 chars]nd', 'user\nWhat do you see in this image?\nas[181 chars]ach'] != ['use[233 chars]nd', "user\nWhat do you see in this image?\nas[182 chars]ion"]

Where to watch it:

• test_small_model_integration_batched — Test dashboard

Relates to #48050

The test_small_model_integration_batched test for Cosmos3OmniForConditionalGeneration has been failing since PR #47096 (commit b9090ae58cda) because the generated text for the second batched sample changed on CUDA: it now describes a "pink blanket" instead of a "pink couch" and ends with "They are lying on their sides, facing each".

This is a test-expectation mismatch caused by a legitimate generation change, not a model bug. The actual decoded output is otherwise coherent and matches the existing XPU expectation for that sample.

This patch updates only the CUDA expected string for the second element in the batched test to the current generated output, leaving the first element and the XPU expectations untouched.

diff --git a/tests/models/cosmos3_omni/test_modeling_cosmos3_omni.py b/tests/models/cosmos3_omni/test_modeling_cosmos3_omni.py
--- a/tests/models/cosmos3_omni/test_modeling_cosmos3_omni.py
+++ b/tests/models/cosmos3_omni/test_modeling_cosmos3_omni.py
@@ -300,7 +300,7 @@ class Cosmos3OmniForConditionalGenerationIntegrationTest(unittest.TestCase):
             {
                 ("cuda", None): [
                     "user\nWhat kind of dog is this?\nassistant\nThe dog in the image appears to be a Labrador Retriever. It has a light brown or golden coat, which is characteristic of this breed. Labrador Retrievers are known for their friendly demeanor and",
-                    "user\nWhat do you see in this image?\nassistant\nIn this image, I see two cats sleeping on a pink couch. The cats appear to be of the same breed, with brown and black striped fur. They're both lying down in a relaxed position",
+                    "user\nWhat do you see in this image?\nassistant\nIn this image, I see two cats sleeping on a pink blanket. The cats appear to be of the same breed, with brown and black striped fur. They are lying on their sides, facing each",
                 ],
                 ("xpu", None): [
                     'user\nWhat kind of dog is this?\nassistant\nThe dog in the image appears to be a Labrador Retriever. It has a light brown or golden color, which is characteristic of this breed. Labrador Retrievers are known for their friendly demeanor and',

---

✅ Verified on GPU

serge ran the targeted @slow test(s) on a GPU runner and opened this PR only after they passed with this patch. Each targeted test was run 5× on both the pre-patch and patched trees to rule out flakiness — the result held on every run.

• Passes with this patch: https://github.com/huggingface/transformers/actions/runs/32128362689
• Failed before the fix (base commit): https://github.com/huggingface/transformers/actions/runs/32126574896

Possibly related

Existing issues/PRs mentioning test_small_model_integration_batched (keyword match — not verified to share a root cause):

• #43455 — fix(models): Add tie_word_embeddings parameter to AyaVisionConfig for proper weight tying (PR, closed, updated 2026-01-29)
• #41923 — fix some ut failures on XPU w/ torch 2.9 (PR, closed, updated 2025-10-29)
• #41337 — make some ut cases pass on xpu w/ latest torch (PR, closed, updated 2025-10-06)
• #37990 — make aya vision 5 integration tests pass on xpu (PR, closed, updated 2025-05-07)

---

This change was produced automatically by serge from a CI failure report. The patch was generated by an LLM and applied by serge; review before merging.

serge v0.1.0 · model: moonshotai/Kimi-K2.7-Code · 27 LLM turns · 26 tool calls · 19.0s · 479987 in / 2733 out tokens

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: cosmos3_omni

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 32128927707
Result: success | Grafana metrics are not available yet.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[IlyasMoutawwakil]

LGTM ! thanks sergy 🫶

[ydshieh]

Investigation notes (manual bisect)

Root cause confirmed via git bisect between 71c6f699ac and 3717b9cda2 on an A10G SSH runner, printing actual decoded output at each step.

Bisect result

Commit	ITEM1 output
71c6f699ac (start)	pink couch … They are lying on their sides, facing each
5e413aa4f5 (one before culprit)	pink couch … facing each
b9090ae58c ← culprit	pink blanket … facing each
3717b9cda2 (end)	pink blanket … facing each

ITEM0 (coat) did not change across the entire range.

Why b9090ae changed the output

PR #47096 refactored get_vision_bilinear_indices_and_weights → get_vision_interpolation_indices_and_weights in qwen3_vl/modeling_qwen3_vl.py, switching to linspace-based bilinear interpolation for position embeddings ("use linspace for bilinear cuz it's the only way for bit exact output"). cosmos3_omni shares the Qwen3-VL vision encoder and is therefore affected by the same change, but its test expectations were not updated (the PR updated qwen3_vl_moe and qwen3_omni_moe but missed cosmos3_omni).

Side note on the CI trace

The assertion error shows ach vs ion rather than couch vs blanket because Python's assertListEqual truncates the middle of long strings — it hides the first divergence point (pink couch/pink blanket) inside [181 chars] and only displays the tail (facing each → ach vs relaxed position → ion).

Conclusion

This is not a bug — the new interpolation is more numerically accurate. The expected value update in this PR is correct and safe to merge.