Whisper decode loop truncates sentences, ignores language and prompt, logs transcripts #793

flm serve --asr 1 loses about half its accuracy in the host-side decode loop, not on the NPU. Same NPU, same libs, same weights: English WER goes from 7.2% to 3.1% with the fixes below. Non-English is worse today (FLEURS de 30%, fr 41%, ar 66%) and comes back to 2-23%.

Five defects, all in src/common/whisper/modeling_whisper.cpp and src/server/:

1. Forced timestamp every 16 tokens. Whisper::generate (L185-224 at 419cf1b) forces the next token into the timestamp range after 16 text tokens. The model takes it as the end of the segment and often emits EOT. Sentences stop mid-word: "...has count <|26.20|>", "...von Sch", "...l'USOC, Com".
2. Language token never fed. The language is sampled from the <|startoftranscript|> logits (L149) but not fed back. The decoder sees <|startoftranscript|><|transcribe|>, a prompt Whisper was not trained on.
3. language and prompt ignored. /v1/audio/transcriptions reads only model and file (server.cpp L978-982). OpenAI clients that set language or prompt get neither, with no error.
4. <|30.00|> is not a timestamp. _build_time_map sums 0.02f while <= 30.00. Float error stops it at 29.98, so the map has 1500 entries, not 1501. There are also no timestamp rules: text can follow a closing timestamp, and a window can open at any time, not only within the first second.
5. Audio over 30 s: the text after the last timestamp of a window is emitted, then the next window starts at that timestamp and decodes it again.

Also: every transcript is printed to stdout ("Audio content: ..."). Under systemd that is the journal. For dictation that is a privacy problem.

Environment

• Ubuntu 26.10, kernel 7.3.0-6-generic, amdxdna 0.10 (in-tree)
• Framework 13, AMD Ryzen AI 7 350 (Krackan, XDNA2, 1022:17f0 rev 20), NPU firmware 1.1.2.64
• FastFlowLM main 419cf1b (1.0.8 + HRX 2026.10.13), built from source with cmake --preset linux-default -DFLM_USE_HRX=ON, g++ 15.3. Same behaviour in the 1.0.8 release tarball.
• Model FastFlowLM/Whisper-V3-Turbo-NPU2 @ 594eecd2, xclbins from src/xclbins/Whisper-V3-Turbo-NPU2
• flm validate: fw_ok, enough_cols, kernel_ok true; memlock_ok false (8 MiB). serve works at 8 MiB with HRX 2026.10.13 anyway.

Repro

FLM_MODEL_PATH=<dir with models/Whisper-V3-Turbo-NPU2> flm serve --asr 1 --port 52625
curl -s http://127.0.0.1:52625/v1/audio/transcriptions \
  -F model=whisper-v3:turbo -F file=@clip.wav -F language=de

Any 10-30 s English clip with a sentence over ~16 tokens shows (1). A German or Chinese clip with language set shows (2) and (3). An 80 s clip shows (5) when the model leaves text after the last timestamp of a window.

Evidence

82 English clips <= 30 s (LibriSpeech plus noisy variants), 10 clips of ~80 s made by concatenating them, 60 FLEURS clips (10 each of de, es, fr, ja, zh, ar). Each clip is one POST to flm serve. WER with Whisper-style normalisation; CER for ja and zh. Expected: close to faster-whisper large-v3-turbo greedy on CPU on the same clips, 2.41% English.

English errors S/D/I go from 49/82/7 to 43/13/4: the loss is truncation. Removing the forced timestamp alone gives 3.14. I reproduced the loop in Python against the same libwhisper_npu.so and got the same numbers (8.01% vs 2.98% on the 1.0.8 libs), so the NPU math is fine.

Fixes

I have fixes ready on a branch, one commit per defect, and will open a PR: fix(whisper): stop forcing a timestamp every 16 text tokens, feed the language token to the decoder, feat(server): honour language and prompt, apply Whisper's timestamp rules, <|30.00|> included, do not decode the tail of a full window twice, fix(server): do not log transcripts by default (only with FLM_LOG_RAW_OUTPUT).

One thing I tried and dropped: making <|30.00|> a timestamp without the timestamp rules. It took the ~80 s set from 3.1% to 21.2%: a window ended on <|30.00|>, greedy decoding carried on past it into a repetition loop, and the next window skipped its first seconds. The rules fix both.

Not addressed: no temperature fallback, no compression-ratio or no-speech gates, greedy only. The encoder output is also not deterministic run to run (1-2 of 82 clips change punctuation or case between passes), which is NPU-side.