Skip to content

Bound a w or W blank run at the end of the string buffer - #3589

Open
LouisDeconinck wants to merge 1 commit into
trailofbits:masterfrom
LouisDeconinck:fix/blank-skip-maxstring-3571
Open

LouisDeconinck wants to merge 1 commit into
trailofbits:masterfrom
LouisDeconinck:fix/blank-skip-maxstring-3571

Conversation

@LouisDeconinck

Copy link
Copy Markdown
Contributor

libmagic behavior

A blank in a w or W value skips a whitespace run in the candidate bytes, but the run is not unbounded: mcopy copies the candidate into a MAXstring-byte union (file/src/file.h:179, file/src/softmagic.c:1478-1511), mconvert forces that buffer's last byte to NUL before the comparison (file/src/softmagic.c:1238), and file_strncmp stops the run at the end of the buffer via eb = b + maxlen (file/src/softmagic.c:2070-2072, 2102-2121). A run long enough to reach the buffer's last byte therefore ends there, and the next byte of the value compares against the terminating NUL and fails.

Divergence

PolyFile compiled each blank to an unbounded [ \t\n\v\f\r]+ (or *), so a string/W or string/w test matched a run of any length:

$ printf '0\tstring/W\tA\\ B\tfound\n' > w.magic
$ python3 -c "open('long.bin','wb').write(b'A' + b' ' * 126 + b'B')"
$ LC_ALL=C TZ=UTC file -b -m w.magic long.bin
ASCII text, with no line terminators

PolyFile reported found for that file and for a run of any length.

Fix

StringMatch.matches now runs the compiled pattern over a reproduction of libmagic's buffer: at most MAX_STRING_BYTES - 1 bytes of the candidate plus the terminating NUL (data[:limit].ljust(MAX_STRING_BYTES, b"\x00")). A str_range (string/N) shortens the copy the same way mcopy shortens it (softmagic.c:1480-1481). search is untouched: libmagic reads that buffer in place and bounds the run at the end of the remaining buffer instead (file/src/softmagic.c:2363-2364), which the search path already does.

Test

test_a_blank_run_stops_at_the_end_of_the_string_buffer pins the boundary for both spellings: 125 blanks match, 126 or more fall back to the text result, and a padded file shows the bound is the buffer rather than the file's length. A search/200/W case pins that a search still matches a longer run.

Verified against file 5.46 and the bundled 5.48 source: both flags, negated tests (string/W !A\ B reports found on a 126-blank file, as libmagic does), and multi-blank values agree at every boundary probed.

pytest tests/test_magic.py: 220 passed. flake8 on the touched files reports only the warnings already present on master.

Fixes #3571

A blank in a w or W value skipped a whitespace run of any length, so a
test still matched when the run carried the next byte past the end of
the buffer libmagic compares against. libmagic copies the candidate
bytes into a MAXstring-byte union and forces its last byte to NUL
(file/src/softmagic.c:1238), and file_strncmp stops the run at the end
of that buffer (file/src/softmagic.c:2102-2121), so the byte after the
run compares against the NUL and a run of 126 or more blanks fails.

StringMatch.matches now runs the pattern over a reproduction of that
buffer: at most MAX_STRING_BYTES - 1 bytes of the candidate, plus the
terminating NUL. A search keeps its own bound, the end of the buffer it
reads in place (file/src/softmagic.c:2363-2364), so it is untouched.

Fixes trailofbits#3571
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The w and W blank skip is unbounded, where libmagic stops at MAXstring

1 participant