Resolve an indirect offset inside a named list from the use site - #3602
Merged
Merged
Conversation
A test nested under a `name` block counts its offsets from wherever the
`use` that dispatched it matched. `NamedAbsoluteOffset` does that, but
only for a test whose own offset is an `AbsoluteOffset`. An indirect
offset is not one, so the position it reads its pointer from was left
counting from the start of the file.
Every `(N.x)` inside a named list therefore read the wrong bytes. On a
Mach-O universal binary, `magic_defs/cafebabe:29` declares `>(8.L)
indirect x`, which should read the architecture's file offset 8 bytes
into the fat_arch record the `use` landed on. It read absolute offset 8
instead, which is the CPU type:
/usr/bin/nohup
pointer at absolute 16 = 16384 the first architecture
pointer at absolute 36 = 49152 the second
what was read = 16777223 0x01000007, CPU_TYPE_X86_64
Both architectures dispatched to the same out-of-bounds offset, so
neither reported anything and the brackets came out empty:
file: ... [x86_64:\012- Mach-O 64-bit x86_64 executable, flags:<...>] ...
before: ... [ x86_64:] [ arm64e (caps: 0x2):]
after: ... [ x86_64: Mach-O 64-bit x86_64 executable, flags:<...>] ...
`rebase_in_named_test` walks the offset instead of type-testing it. It
rebases a position and leaves a distance alone: what a relative (`&`)
offset wraps is a distance from the previous match, and rebasing it would
add the `use` site twice.
Two differences from `file` remain on this input, both in how the message
is assembled rather than where it reads: PolyFile writes `[ x86_64` where
libmagic writes `[x86_64`, and libmagic emits its match separator before
each nested verdict. Over 2,000 real files this fix alone moves the
strongest-match agreement by one file, because the Mach-O descriptions
now hold the right content but still do not match byte for byte.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RG45aqyyCLkL5b1UB5GWd7
This was referenced Sep 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A test nested under a
nameblock counts its offsets from wherever theusethat dispatched it matched.NamedAbsoluteOffsetdoes that, butMagicTest.__init__applied it only to a test whose own offset is anAbsoluteOffset. An indirect offset is not one, so the position it reads its pointer from was left counting from the start of the file, and every(N.x)inside a named list read the wrong bytes.What it looked like
magic_defs/cafebabe:29declares>(8.L) indirect x, which should read the architecture's file offset 8 bytes into thefat_archrecord theuselanded on. For/usr/bin/nohup:Both architectures dispatched to the same out-of-bounds offset, so neither reported anything:
The fix
rebase_in_named_testwalks the offset instead of type-testing it. It rebases a position and leaves a distance alone: what a relative (&) offset wraps is a distance from the previous match, so rebasing it would add theusesite twice.The regression test is self-contained rather than reliant on a Mach-O binary. It puts the pointer a named list declares at offset 12 and a decoy at offset 4, which is what an offset counted from the start of the file reads instead.
file5.48 reportsbase [:nestedfor the same input.What this does not fix
Two differences from
fileremain on a universal binary, both in how the message is assembled rather than where it reads:[ x86_64, libmagic writes[x86_64. A\bfollowing auseis not suppressing the separating space.:\012- Mach-O 64-bit ...where PolyFile writes: Mach-O 64-bit ....So this alone moves the numbers by one file, not by forty. Over the same 2,000 real files, strongest-match agreement goes from 1765/1998 to 1766/1998 and containment from 1922/1998 to 1923/1998. The Mach-O descriptions now carry the right content but still do not match byte for byte, so the differential does not count them. Closing that cluster needs the two items above, which I will file separately.
Full suite: 384 passed, 1,511 subtests. Blocking flake8: 0. The test fails with the change reverted.
🤖 Generated with Claude Code