Repository navigation
/v1/embeddings endpoint always returns only 1 embedding regardless of batch size #373
Description
When sending multiple texts to /v1/embeddings, the response always contains only one embedding object (index: 0), regardless of how many inputs were provided.
Steps to reproduce:
flm serve qwen3:4b --embed 1
$body = '{"model":"embed-gemma:300m","input":["text1","text2","text3","text4","text5"]}'
Invoke-RestMethod -Uri "http://localhost:52625/v1/embeddings" -Method Post -ContentType "application/json" -Body $bodyExpected: 5 embedding objects in data array
Actual: 1 embedding object (index: 0 only)
Server logs:
Request received with full batch:
[⬇️ ] Incoming Request: POST
[LOG] Target: /v1/embeddings
[LOG] Body:
{
"input": ["text1", "text2", "text3", "text4", "text5"],
"model": "embed-gemma:300m"
}
[🟢 ] NPU Locked!
Embedding input: text1 ← only first element processed
[🔵 ] NPU Lock Released!
Same behavior with batch of 2:
[⬇️ ] Incoming Request: POST
[LOG] Target: /v1/embeddings
[LOG] Body:
{
"input": ["text1", "text2"],
"model": "embed-gemma:300m"
}
[🟢 ] NPU Locked!
Embedding input: text1 ← only first element processed
[🔵 ] NPU Lock Released!
Environment:
- FastFlowLM v0.9.34
- Windows 11
- AMD Ryzen AI 7 350 (Strix Point, XDNA2 NPU)
- Model: qwen3:4b with embed-gemma:300m
Activity
ZaneNi commentedon Feb 27, 2026
@Miclell Thank you for your feedback!
At the moment, we only process the first element of the input array, so the behavior you observed is expected.
We will address this limitation in the next release.
ZaneNi commentedon Mar 12, 2026
Please check the latest release. @Miclell