Repository navigation
Every EOF-truncated PCAP-NG file raises an uncaught ValueError: pcapng_block_selector passes a negative __length__ to SchemaField #678
Description
Activity
A second manifestation of the same root shape, found while probing #676's bound: a
captured_lenpast the block makesEnhancedPacketBlock's option-area expressionlength - 32 - captured_len - (4 - captured_len % 4) % 4negative, the negative reaches a_TextFieldtemplate asf'{-N}s', andstructraisesstruct.error: bad char in struct format— again not one ofpcapkit.utilities.exceptions.Measured on 200 Enhanced Packet Blocks with
captured_len = 0xFFFFFFin 8,048 octets, byte-identical before and after #676. Same fix family: clamp the computed length at zero where it is consumed, rather than letting a negative reachstructorread().- added a commit that references this issue
on Sep 22, 2026 Correction to the sweep in the body, from the cross-review of #676. I wrote that every truncation level raises the same
ValueError; it does not.Re-swept
dhcp.pcapngat 1-octet granularity over 501 levels:497 ValueError: read length must be non-negative or -1 2 error: bad char in struct format 2 okThe reviewer swept 376 levels independently and found a third type,
ProtocolError: unknown byteorder magic: 0x0, from cuts reaching into the Section Header Block. So the picture is: overwhelmingly theValueError, withstruct.errorand a byteorder-magicProtocolErrorat particular depths, and a small number of levels that do parse.That strengthens rather than weakens the report — three different exception types out of one root cause, two of them not from
pcapkit.utilities.exceptions— but the original wording was over-general and worth correcting before it becomes the record.- added 4 commits that reference this issue
on Sep 22, 2026 Completing the census in the correction above, since the re-sweep stopped at 501 levels and two of the
three exception types only appear deeper. Swept all 1,509 octet boundaries ofdhcp.pcapngat
1-octet granularity onf0999858e(post-#676, post-#683), classifying by whether the exception is one
ofpcapkit.utilities.exceptions:1479 ValueError: read length must be non-negative or -1 (foreign) 10 struct.error: bad char in struct format (foreign) 8 ProtocolError: unknown byteorder magic: 0x0 and friends (in-library) 4 FormatError: unknown file format (in-library) 2 ProtocolError: PCAP-NG: [if_tsresol] invalid length (in-library) 6 parsedSo 1,489 of 1,509 levels raise from outside the library — the two types this issue names, and the
reviewer's byteorder-magicProtocolErrorwas already in-library. Thestruct.errordepths are 372,
373, 720, 721 and six more; at 372 it arrives by a route the comment above does not name — not the
craftedcaptured_len, but__option_padding__reaching-32onEnhancedPacketBlock.padding_opts,
becauseOptionField.unpacksubtracts each parsed option's real size from the area without checking it
fits. Same root shape, third consumption point.One correction to the "Where it comes from" section: the negative does not originate at
pcapng_block_selector, it only surfaces there.PCAPNG.read'sseek_cur = _seek_set + block.length
seeks past the real end of the file when the last block declares more than the file holds — legal
and silent — so the next block read haspreparederive the remainder asend - tell, which is
already negative beforePCAPNG.typeis read at all. That is why the failure is the whole extraction
rather than the one truncated block, and whymax(packet['__length__'], 0)at the selector alone is not
sufficient: with the length clamped to 0 the block schema fabricates a block from zero padding, and
UnknownBlock.body'spkt['length'] - 12then goes negative instead.Fix in #699: clamp the seek at the octets
_read_filengactually returned, report a tail under the
12-octet block minimum as the quietStreamEOFErrorthe frame loop already catches, and floor every
computed length in the schema at zero. 1,495 of the 1,509 levels parse, and none of the 14 that do not
raises from outside the library.- added 6 commits that reference this issue
on Sep 23, 2026 9 remaining items
- added 12 commits that reference this issue
on Sep 26, 2026
Found while measuring #594's amplification band, and deliberately left out of #676 because it is a
block-level defect rather than an option-level one, and fixing it changes frame counts on truncated
captures.
Every EOF-truncated PCAP-NG file raises an uncaught
ValueErrorNot "loses a frame" — raises out of
Extractor, so the whole extraction is lost. Measured on0c7f2b7c9against the committedexamples/captures/dhcp.pcapng, truncated at every 4-octetboundary from -4 to -400. All 101 levels fail, identically:
ValueErroris not one ofpcapkit.utilities.exceptions, so a caller cannot distinguish it from abug in its own code, and it defeats the
#431/#571standing constraint that a legitimatelytruncated capture must still parse — for PCAP-NG that constraint is currently vacuous at the file
level.
Where it comes from
The negative length originates at
pcapkit/protocols/schema/misc/pcapng.py:228:packet['__length__']is seeded by@preparefrom what is left in the file(
pcapkit/utilities/decorators.py:274) and then decremented by four forPCAPNG.type. A previousblock whose declared Block Total Length ran past the real end of the file leaves the reader
positioned such that fewer than four octets remain, so
__length__goes negative anddata.read(negative)raises.@prepareonly turns a remainder of exactly zero into the quietStreamEOFErrorthat the frame loop is built to catch (decorators.py:264-270); one, two or threeoctets fall through.
The fix that is probably right
max(packet['__length__'], 0), which is the idiompcapkit/protocols/schema/transport/sctp.pyalready uses for the same hazard, and which
Schema.unpackitself already anticipates —schema.py:898-900warns rather than raises when__length__has gone negative, so the negativevalue is expected to be survivable at that layer and only this one call site treats it as fatal.
Worth deciding alongside it: whether a short tail should surface as the quiet
StreamEOFErrortheframe loop already handles (giving the frames before the cut, like the legacy PCAP reader) or as a
ProtocolErrornaming the truncation. The first matches the#431bar; the second is louder. Eitherway it should be an in-library exception rather than a bare
ValueError.Why it is not in #676
#676 bounds an option's payload to the option area its block declares. This is the block's own
span against the file, one layer up. Measured byte-identical before and after #676 at all 101
truncation levels, including the failures — #676 neither causes nor cures it. Fixing it changes frame
counts on truncated captures, which wants its own review and its own
breakingargument.Related
packetis computed then overwritten with b'', so dumping through PCAPIO writes a record header claiming 314 octets followed by none #646 — also PCAP-NG and also about a length being the wrong length, but that one isProtocolBase.packettreating Block Total Length as a pre-payload header length. Different site,different symptom.