Small Linux tool for two recordings of the same movie release, e.g. one recorded with German audio and one with English audio. It aligns the audio of the second recording to the reference and muxes video + both audio tracks into a single MKV.
sudo apt install ffmpeg python3-numpy
./audiosync.py film-de.mkv film-en.mkv -o film.mkv
The first file is the reference. Its video and first audio track are retained. The first audio track from the second file is added as the second language track.
If the second file carries the better video, swap the two arguments — but then the language tags and the default flag must be passed explicitly, otherwise the tags and the default track will be wrong (track titles, however, follow the language tags automatically):
./audiosync.py film-en.mkv film-de.mkv -o film.mkv \
--ref-lang en --second-lang de \
--default-lang de
- Checks the video stream parameters.
- Decodes sample frames from both files and compares downscaled grayscale structural signatures at several positions. Recordings of the same release can start at slightly different times (different channel padding), so a constant video offset is first estimated and applied before comparing. The estimate is refined to sub-grid resolution (0.05 s), because the signature window is only a few frames wide and coarse seeking can otherwise push genuine matches below the similarity threshold.
- Extracts both first audio tracks as low-rate mono PCM solely for analysis.
- Determines the relative audio offset at several positions throughout the movie (default 5).
- Keeps the largest cluster of measurements that agree within 150 ms — an isolated outlier (quiet scene, a repeated music cue) whose score is still just above the threshold is dropped — uses the median of that cluster, and rejects the pair when the cluster itself still spreads by more than 150 ms. A genuine speed difference or a different cut shows up as a consistent trend across positions, not as an isolated point, so it still fails.
- Positive offset: trims the leading audio of the second recording with
atrimand resets its PTS withasetpts=PTS-STARTPTS. - Negative offset: delays the second recording's audio with
adelayand resets its PTS withasetpts=PTS-STARTPTS. - Muxes the reference video + both audio tracks + the subtitle tracks of both inputs into the final MKV.
positional:
reference reference recording, e.g. German
second second recording, e.g. English
-o, --output output file (required)
--ref-lang language tag of the reference track (default: de)
--second-lang language tag of the second track (default: en)
--ref-title title of the reference track (default: derived from
--ref-lang, e.g. en -> English)
--second-title title of the second track (default: derived from
--second-lang, e.g. de -> Deutsch)
--default-lang which audio track carries the `default` flag: a
language code (de, en, fr, ...) or the words
`ref` / `second` (default: de). Use it whenever the
roles are swapped so the German track stays the
default one. A value that matches neither track's
language aborts with an error instead of silently
flagging the other track.
--max-offset maximum search window in seconds for the audio
correlation (default: 180)
--samples number of audio analysis positions (default: 5)
--threshold minimum normalized correlation score for an audio
measurement to be accepted (default: 0.08)
--force continue despite a failed video identity check
--dry-run run all checks and analysis, but write no output
The reference video is copied with -c:v copy, so there is no video
re-encoding — and the output video is exactly the reference's video. The
quality of the result therefore depends entirely on which file you pass first.
Before merging, compare both sources:
ffprobe -v error -select_streams v:0 \
-show_entries stream=codec_name,width,height \
-show_entries format=duration,size -of json FILE
If the second file has the higher resolution/bitrate, swap the arguments and set
--ref-lang, --second-lang, --ref-title, --second-title and
--default-lang accordingly (see Usage above).
The subtitle tracks of both inputs are muxed into the output
(-map 0:s? -map 1:s? -c:s copy), reference subtitles first. Nothing is
re-encoded.
Known limitation: the tool does not set the subtitle disposition explicitly, so
the flags are whatever the muxer derives — in practice the tracks end up with
default=1, and a source-side forced flag is not reliably preserved. For
forced subtitles (foreign-language dialogue only) fix the flags afterwards:
mkvpropedit film_dual.mkv \
--edit track:s1 --set flag-forced=1 --set flag-default=0 \
--edit track:s2 --set flag-forced=0 --set flag-default=0
Always verify with ffprobe -show_streams and look at disposition.
The reference audio is stream-copied (-c:a:0 copy): it keeps its original
codec and bitrate bit-for-bit, with no generation loss. This matters because the
reference track is usually the one the library plays by default.
Only the second track is encoded, to AAC 160 kbit/s (-c:a:1 aac -b:a:1 160k). That is unavoidable — once an arbitrary audio filter (adelay/atrim)
is applied to fix the offset, the stream has to be rewritten.
160k is a measured floor, not a guess. Test signal: a lossless music excerpt, first encoded to MP3 128k (the capture stage in this collection), then the AAC transcode, comparing the error against the original:
| stage | total error | added by the AAC stage |
|---|---|---|
| MP3 128k (source) | -20.0 dB | — |
| + AAC 128k | -15.9 dB | -18.1 dB |
| + AAC 160k | -17.6 dB | -21.5 dB |
| + AAC 192k | -18.2 dB | -23.6 dB |
At 160k and 192k the added error sits below the error the source already carries, so it is masked in practice — and the two are only 0.6 dB apart. At 128k the add-on equals the source's own error and the total error energy grows ~2.6x. Hence 160k as the floor: above it the gain is negligible, below it the second lossy step stops hiding. (RMS error, not a hearing model — valid as a comparison between the three rates, not as an audibility proof.)
Result: two tracks of possibly different codecs (e.g. MP3 + AAC) in one MKV. That is fine for Matroska and for every player used here.
Fully lossless (both tracks copied, the offset applied as a container-level
track shift via mkvmerge --sync) is possible but would mean replacing the mux
step; millisecond precision there is far below the ~20 ms A/V tolerance, so it
is a real option if it is ever worth the rework.
./audiosync.py film-de.mkv film-en.mkv -o film.mkv --dry-run
Only use this if you have independently verified that the releases are the same:
./audiosync.py film-de.mkv film-en.mkv -o film.mkv --force
offset = second - reference.
+1.250s second recording's audio starts 1.25 seconds later
-> trim the first 1.25 seconds of the second track
-1.250s second recording's audio starts 1.25 seconds earlier
-> delay the second track by 1.25 seconds
The tool deliberately aborts when the offset changes substantially during the movie. That usually indicates different cuts, PAL/NTSC speed differences, extra/missing scenes, or a bad correlation.
AVI/Xvid sources break the MKV mux. An AVI whose video has no timestamps fails with "Can't write packet with unknown timestamp" / "Error muxing a packet" (exit 234). This is not a sync problem — remux first (stream copy, no re-encoding) and use the remux as input:
ffmpeg -fflags +genpts -i film-de.avi -c copy film-de.remux.mkv
Reading the failure output
RESULT: FAIL (n/4) / no position aligns
different transfer or cut — not mergeable
audio offset is unstable (>150 ms spread)
the video matches but the audio timing does not (different dub or mix)
CalledProcessError from ffmpeg inside mux()
container/timestamp issue, the analysis itself was fine (see AVI above)