Repository navigation
Parakeet TDT v3 (offline): multi-second span silently dropped from transcript #71
Description
I have a 70s window of a lecture recording where running Parakeet TDT 0.6b v3 inference silently drops a ~15s span of clearly-audible speech in the middle of the window. The same audio transcribes correctly when the window is shifted or shortened, and the failure is bit-identical across quantizations, so does not seem to be an audio or precision problem.
Observed via the Rust bindings (transcribe-cpp crate v0.1.1), CPU backend (x86-64, Zen 3), model handy-computer/parakeet-tdt-0.6b-v3-gguf.
Reproduction
https://jon.thesquareplanet.com/share/dropped-span-repro.wav is the audio snippet encoded as 16 kHz mono s16 WAV. When transcribed, the transcript is empty for the statement at 0:37–0:52: "There's a sort of a quote from one of the famous people behind the C programming language, Brian Kernigan, …").
To reproduce, set up a Cargo repository with dependencies on hound (for WAV decoding) + transcribe-cpp 0.1.1, then:
use transcribe_cpp::{init_backends_default, Model, RunOptions, TimestampKind};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut args = std::env::args().skip(1);
let usage = "usage: repro <model.gguf> <audio.wav>";
let model_path = args.next().expect(usage);
let wav_path = args.next().expect(usage);
// Decode the WAV to f32 mono in [-1, 1] (it is already 16 kHz mono s16).
let mut reader = hound::WavReader::open(&wav_path)?;
let spec = reader.spec();
assert_eq!(spec.sample_rate, 16000, "repro audio is 16 kHz");
assert_eq!(spec.channels, 1, "repro audio is mono");
assert_eq!(spec.sample_format, hound::SampleFormat::Int, "repro audio is s16");
let scale = (1i64 << (spec.bits_per_sample - 1)) as f32;
let samples = reader
.samples::<i32>()
.map(|s| s.map(|s| s as f32 / scale))
.collect::<Result<Vec<f32>, _>>()?;
eprintln!("audio: {:.1}s", samples.len() as f64 / 16000.0);
init_backends_default()?;
let model = Model::load(&model_path)?;
eprintln!("backend: {}", model.backend());
let mut session = model.session()?;
let transcript = session.run(
&samples,
&RunOptions {
timestamps: TimestampKind::Word,
..Default::default()
},
)?;
let mut max_gap = 0i64;
let mut gap_at = 0i64;
for pair in transcript.words.windows(2) {
let gap = pair[1].t0_ms - pair[0].t1_ms;
if gap > max_gap {
max_gap = gap;
gap_at = pair[0].t1_ms;
}
}
println!(
"{} words; largest inter-word gap: {}ms starting at t={}ms",
transcript.words.len(),
max_gap,
gap_at
);
if max_gap > 5_000 {
println!(
"BUG REPRODUCED: ~{}s of audible speech dropped after t={:.1}s \
(expected the \"There's a sort of a quote ...\" sentence there)",
max_gap / 1000,
gap_at as f64 / 1000.0
);
std::process::exit(1);
}
println!("no drop detected (transcript covers the full window)");
Ok(())
}RunOptions: defaults + timestamps = Word (drop reproduces with pure defaults as well). Word rows around the hole:
t0_ms t1_ms
35840 36400 "it." <-- speech before the hole
52000 52160 "And" <-- next emitted word; 15.6s hole
I also tried some window experiments with the same audio to see if the same gap appears:
| window | result |
|---|---|
| 0–70s | 15.6s hole (36.4→52.0) |
| 0–55s | same hole |
| 0–52s | no hole (quote fully transcribed) |
| 0–45s | no hole |
| 10–70s | no hole (!) |
| 20–50s | no hole |
| 34–64s | no hole |
Additional data points:
- Prepending 10s of digital silence to the 0–70s window makes it worse: the hole grows to 31.2s (36.4→67.6 in the original timeline).
- Q8_0 and F16 GGUFs produce bit-identical word rows (same 180 words, same hole), so this doesn't seem like a quantization artifact.
- CPU and Vulkan backends both reproduce the hole at the same position, so it's not a backend numerics issue.
spec_k_drafts = 0(speculative decode disabled): no change.- The int8 ONNX export of the same model (via onnxruntime and transcribe-rs) transcribes the full 195s window correctly, including the quote.
Activity
jonhoo commentedon Jul 8, 2026
I don't know how much of this is transcribe.cpp's fault, and how much of it really comes down to the underlying model though 😬
cjpais commentedon Jul 9, 2026
@jonhoo I will double check against the reference. I have been able to replicate things like this in the past with this codebase, but I also noticed the reference has the same issue. So this is possibly just an issue with the model itself. Certainly I will try to take a further look as well because we should try to make sure we have no bugs
hiSandog commentedon Jul 11, 2026
The reference comparison should capture token/timestamp outputs and encoder window boundaries for the failing 70s input, then repeat with shifted windows exactly as reported. If the reference also drops the span, the wrapper can still detect suspicious multi-second timestamp gaps and optionally retry an overlapping subwindow rather than silently returning incomplete text. A deterministic fixture should ensure any overlap recovery deduplicates repeated words at the splice.
feat!: migrate inference from ONNX Runtime to transcribe.cpp
feat!: transcribe locally with Parakeet instead of Gladia (#12)
felixabeck commentedon Aug 17, 2026
Second independent reproduction, on different audio and different hardware, plus a data point that I think narrows the cause considerably: shifting the input by a single 10 ms mel hop changes which span gets dropped.
Setup
transcribe-cpp0.1.3, called through the shippedlibtranscribe.so.0via ctypes, so this is the same code path the host app useshandy-computer/parakeet-tdt-0.6b-v3-gguf,parakeet-tdt-0.6b-v3-Q8_0.gguftranscribe_runSymptom
The last 5.6 s of clearly audible speech produce no text. The last emitted word ends at 59.92 s; speech continues to 65.70 s, after which the recording has 1.9 s of trailing silence. The dropped span is one complete sentence.
The surviving transcript ends on a grammatically complete sentence, because the drop happens to begin at a pause — which is what makes this class of failure invisible to someone reading the output.
Relevant negatives:
transcribe_was_truncated()is false,transcribe_runreturnsTRANSCRIBE_OK, and nothing is logged at any level. As far as I can tellarch/parakeetnever setsTRANSCRIBE_ERR_OUTPUT_TRUNCATEDat all, so for this family the failure is silent by construction and a caller has no signal to act on. That may be worth treating as a separate issue from the drop itself.max_audio_msis 0, T_enc ≈ 844 frames against the16*T_enc + 1024runaway guard, and hitting that guard would fail the run loudly instead.run_one_shot_inner), no windowing.Window experiments
Same shape as the table in the original report. The transcript consistently dies at the same absolute audio position (59.92 s) whenever enough leading context is present:
Decoded in isolation, the dropped sentence transcribes perfectly.
The new data point: one mel hop of leading silence changes the outcome
Prepending silence to the start of the identical buffer. Encoder frames are 80 ms here (
hop_length160 @ 16 kHz,subsampling_factor8), so the middle column is how far the shift moves content across the encoder frame grid:Two things fall out of this:
It is also worth noting that the failure does not so much disappear as move: at 10 ms the tail comes back but a span in the middle is dropped instead. So "tail truncation" and "hole in the middle" are the same defect — it only presents as truncation when the dropped span happens to reach the end of the input. Reports of the two may be worth triaging together.
Scale, and a cross-check against another family
I re-decoded 50 real dictations (1167 s total) through the same library with two models:
Measured by comparing decoded word/segment timestamps against an energy-based voice detector over the audio, so only gaps that actually contain speech are counted, not ordinary pauses. One recording in 50 was affected — rare, but silent and unbounded when it happens.
Happy to run further experiments on this sample if a specific hypothesis would help; the harness is set up and the failing case is stable. I have not attached the audio, since it is a personal recording, but I can produce whatever measurements from it are useful.
cjpais commentedon Aug 17, 2026
Wow I just remembered because of your comment I ran into this in transcribe-rs and I think I just ended up prepending silence which largely resolved this. Please send audio samples to contact@handy.computer and I'll see what I can do
I know I also can replicate this
felixabeck commentedon Aug 18, 2026
Audio sent to contact@handy.computer — the 67.5 s sample, raw as recorded, free to use as a public fixture.
One caveat on prepending silence, since it came up as the workaround. On this sample it does not resolve the drop, it relocates it. Re-verified today against transcribe-cpp 0.1.3:
At 160 ms the transcript ends "... has to investigate everything again. Don't you think so?" — it reaches the end of the audio, so any check based on "did the text run to the end" now passes, while "There should be a little bit more context than the mere question," has silently vanished from the middle. 10, 20, 80, 240, 320, 400, 480, 500, 560 and 640 ms all give the same 118 words. Only 40 ms was ever complete.
So prepending silence moves the hole out of the tail, where a reader notices it, into the middle, where nobody does — and removes the one signal a caller could have built a check on. Worth measuring per-model before it lands anywhere as a mitigation.
Independent of what the model turns out to be doing:
arch/parakeetappears never to setTRANSCRIBE_ERR_OUTPUT_TRUNCATEDat all, so this family is silent by construction and a caller has no signal to act on. That seems worth its own issue — #128 (cohere_asr) and the closed #89 (whisper) look like the same family.cjpais commentedon Aug 18, 2026
To be honest, I don't believe that the output is being truncated in a way the model can signal. I'm mostly asking for the audio because I want to compare it versus their reference inference implementation. If the reference also has the same drop, it's just a fundamental model limitation/issue
felixabeck commentedon Aug 18, 2026
Makes sense — the sample is with you now, so I'll leave the reference comparison to you.
One caveat in case it saves you a run: the drop is alignment sensitive, so a single clean decode of that file wouldn't clear the reference. Prepending 40 ms of silence already makes it decode completely here, while 160 ms moves the missing span into the middle.
No urgency from my side either way — I've moved my own setup to Whisper for this.
cjpais commentedon Aug 18, 2026
its always so bizarre interacting with people who heavily rely on ai to write for them. like a strange dream or something
felixabeck commentedon Aug 18, 2026
I understand, and I'm sorry for that. Since I don't understand the technical background myself, I have to rely on the analysis of AI. I just try to help and close a bug. Just ignore it if it's not useful
cjpais commentedon Aug 18, 2026
generally I think its just best to concur without any analysis, and just what I ask for. im more than capable myself to analyze and mostly it just adds noise