You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
NumberField with a callable length cannot pack at 8 octets: _need_process latches True from the placeholder and is never cleared #591
A NumberField whose length is a callable is unpackable once that callable resolves to 8 octets: _need_process is latched True while the field is still a placeholder and is never cleared when the real length arrives, so pre_process returns bytes to a struct template that now expects an integer.
Mechanism
numbers.py:109 sets the flag false in __init__, and the template builder sets it true for any length that is not 8, 4, 2 or 1:
else: # do not unpackstruct_fmt=f'{length}s'self._need_process=True
A callable length is a placeholder of -1 at construction, which takes that branch. When __call__ later resolves the real width the template is rebuilt correctly — but nothing resets _need_process. So pre_process at numbers.py:194 takes the process path, self._length is now 8 rather than negative so the _length < 0 repair at :196-203 is skipped, and :205 returns
Executed on origin/main (9c240a60e), CPython 3.14.7. The comparison is the proof — identical resolved length and identical template, differing only in how the length was supplied:
--- callable length (resolves to 8) ---
length= 8 _need_process= True template= >Q
error: required argument is not an integer
--- static length 8, for comparison ---
length= 8 _need_process= False template= >Q
pack -> 0000010000000000
So NumberField(length=lambda pkt: 8) cannot pack, while NumberField(length=8) packs the same value through the same template.
Impact on the wire
This makes every extended (8-octet) MPTCP DSS form unpackable, since those widths are selected at runtime from the DSS flags and therefore must come from a callable. Reported as:
_make_mptcp_dss(DSS, ack=1 << 40) -> struct.error: required argument is not an integer
Any other schema that resolves a numeric width to 8 octets through a callable has the same failure.
The obvious fix is to recompute _need_process when the resolved length is applied, rather than only at construction. Whether 1/2/4 are equally affected is not something I checked — the reproduction above is the 8-octet case only, and build_template takes the same else branch for any width outside {1, 2, 4, 8}, so a callable resolving to, say, 3 would legitimately still need processing. A fix must distinguish "placeholder" from "genuinely needs byte packing".
Scope correction to this issue, established by the fix in #598 and verified here.
This issue understated the defect in two ways.
1. All four native widths were affected, not only 8. The body said whether 1/2/4 are equally affected "is NOT something I checked". They are: a callable length resolving to 1, 2, 4 or 8 each latched _need_process = True against templates >B, >H, >I and >Q respectively, and each raised required argument is not an integer. Width 3 correctly packed throughout, because a 3-octet field genuinely does need byte processing. The latch has nothing to do with the width it latches into — which is obvious in hindsight and is why the 8-octet DSS case was simply the one that got noticed.
EnumField was affected identically, since it also leaves __template__ unset. The eight subclasses that fix __template__ never latched anything, because __init__ skips build_template for them.
2. Parsing was broken in the mirror direction, not just packing. The title says "cannot pack". The same flag made post_process (numbers.py:220-224) call int.from_bytes on the integer struct.unpack had already produced. So the field could not read what it could not write.
On the fix's shape, since this issue speculated about needing to distinguish "placeholder" from "genuinely needs processing": #598 does not add such a bit. It makes build_templateassign_need_process rather than only ever raise it, so the flag becomes a function of the width currently in force — True for the -1 placeholder, False for 8, True again for 3. The placeholder's True was never wrong; it was only sticky.
One separate defect was found on the same path and is filed independently, because #598 neither fixes nor masks it: pre_process's repair at numbers.py:198 uses math.ceil(value.bit_length() // 8), which is floor division, so every octet boundary is sized one octet short and raises OverflowError.
A
NumberFieldwhoselengthis a callable is unpackable once that callable resolves to 8 octets:_need_processis latchedTruewhile the field is still a placeholder and is never cleared when the real length arrives, sopre_processreturnsbytesto astructtemplate that now expects an integer.Mechanism
numbers.py:109sets the flag false in__init__, and the template builder sets it true for any length that is not 8, 4, 2 or 1:A callable length is a placeholder of
-1at construction, which takes that branch. When__call__later resolves the real width the template is rebuilt correctly — but nothing resets_need_process. Sopre_processatnumbers.py:194takes the process path,self._lengthis now8rather than negative so the_length < 0repair at:196-203is skipped, and:205returnsbytes, into a template that is by then
>Q.Measured
Executed on
origin/main(9c240a60e), CPython 3.14.7. The comparison is the proof — identical resolved length and identical template, differing only in how the length was supplied:So
NumberField(length=lambda pkt: 8)cannot pack, whileNumberField(length=8)packs the same value through the same template.Impact on the wire
This makes every extended (8-octet) MPTCP DSS form unpackable, since those widths are selected at runtime from the DSS flags and therefore must come from a callable. Reported as:
Any other schema that resolves a numeric width to 8 octets through a callable has the same failure.
Notes
pcapkit/corekit/fields/numbers.pywas outside that change's scope, so fix(tcp): correct MPTCP option length arithmetic at all six #576 sites #585 works around it with aSwitchFieldoverUInt32Field/UInt64Fieldrather than touching this file. The workaround leaves this defect live for every other caller, which is why it needs its own fix._need_processwhen the resolved length is applied, rather than only at construction. Whether1/2/4are equally affected is not something I checked — the reproduction above is the 8-octet case only, andbuild_templatetakes the sameelsebranch for any width outside {1, 2, 4, 8}, so a callable resolving to, say, 3 would legitimately still need processing. A fix must distinguish "placeholder" from "genuinely needs byte packing".pcapkit/corekit/fields/field.py. This is a separate defect in a sibling module.