Skip to content

[Issue]: Vectors collapse (non-discriminative) with the xclbins shipped for ≥0.9.45 #789

Open

Description

@fermosan

Problem Description

Summary
With FastFlowLM ≥0.9.45, embed-gemma:300m (--embed 1) returns deterministic but non-discriminative embeddings. Unrelated texts score about as high as similar ones, and the vectors are uncorrelated with
those from 0.9.42. Copying the four Embedding-Gemma-300M-NPU2 xclbins from 0.9.42 into a 1.0.8 install gives correct embeddings again. This points at the refreshed xclbins, not the host library or the
open-source embedding code. It looks like the same problem as #732 and #661.

Environment
- AMD Ryzen AI Max+ 395 (Strix Halo), NPU firmware 1.1.2.65, amdxdna 0.16, Linux 7.2.10 (Fedora 43)
- FastFlowLM from the Fedora COPR (alessandrolattao/fastflowlm). Kernels and libs come from flm-fetch-kernels, which fetches the matching release's kernel files.
- Model embed-gemma:300m, served with flm serve qwen3-it:4b --embed 1, queried through /v1/embeddings

Method
- Discrimination test: I embedded six texts and compared cosine similarity: two similar pairs (FreeIPA password reset, and Ceph/CloudStack storage full) and unrelated texts (a cake recipe, a Wazuh agent message).
- Fidelity test: I re-embedded 60 texts that already had vectors stored from an earlier run on 0.9.42 and compared them.

Results

│ FastFlowLM │ Similar pairs │ Unrelated pairs │ Cosine vs stored 0.9.42 vectors (60 texts) │
│ 0.9.42-5 │ 0.86 / 0.52 │ 0.09 / 0.08 / 0.18 │ mean 0.9998 (min 0.9996) │
│ 0.9.43-2 │ 0.86 / 0.52 │ 0.09 / 0.08 / 0.18 │ mean 0.9998 (min 0.9996) │
│ 0.9.45-2 │ 0.74 / 0.83 │ 0.79 / 0.75 / 0.72 │ mean −0.035 (min −0.129) │
│ 1.0.8-1 │ 0.75 / 0.83 │ 0.79 / 0.75 / 0.73 │ mean −0.033 (min −0.129) │

Things I ruled out
- Model weights: I re-pulled embed-gemma:300m, and the files are byte-identical (md5) to the July copy.
- Prompt format: neither a query prefix nor a document prefix fixed it, and passing dimensions: 768 made no difference. The same text embedded twice gave cosine 1.0000, so the output is deterministic, not random.
- Batching: one text at a time and a batch of six gave identical results.
- Stale libraries: I cleared /opt/fastflowlm/lib* and share/flm/xclbins and re-ran flm-fetch-kernels before each test.

Operating System

Fedora 43

CPU

AMD RYZEN AI MAX+ 395 w/ Radeon 8060S

GPU

AMD Radeon 8060S Graphics

ROCm Version

ROCm 10

Installation Method

Other (describe in Additional Information)

Installed ROCm Packages / Versions

Installed ROCm packages / versions
Paste output here

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of rocminfo --support

rocminfo --support output
Paste output here

Additional Information

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions