Skip to content

HIP NAT_TRAVERSAL_MODE and ESP_TRANSFORM size list entries at one octet where the RFCs specify 16 bits #472

Description

@JarryShaw

Summary

Two more HIP list fields size their entries at one octet where the RFC
specifies 16 bits, so a single-entry list round-trips with a spurious extra
entry, the packed length disagrees with the declared Length, and parsing
through the public API fails outright.

This is the same defect class as #463, at two sites #463 did not cover. PR #466
fixed it for TransportFormatListParameter; these two remain.

Sites

  • pcapkit/protocols/schema/internet/hip.py:567-570 — NATTraversalModeParameter.modes,
    item_type=EnumField(length=1, namespace=Enum_NATTraversal). :rfc:5770 §5.4
    specifies a 16-bit Mode ID.
  • pcapkit/protocols/schema/internet/hip.py:939-942 — ESPTransformParameter.suites,
    item_type=EnumField(length=1, namespace=Enum_ESPTransformSuite). :rfc:7402
    §5.1.2 specifies a 16-bit Suite ID.

HIPTransportModeParameter.mode in the same file already uses length=2, so the
correct shape is established locally.

Reproduction

Measured on fa128959e, through the public makers. Byte-identical on PR #466's
branch, so pre-existing and not introduced by that PR, which only changed the
length= callback reference at these two sites:

proto = object.__new__(HIP)

s = proto._make_param_nat_traversal_mode(Parameter.NAT_TRAVERSAL_MODE, version=2, modes=[1])
s.pack()          # 0260000400000100000000  -- 11 octets, declared len=4
NATTraversalModeParameter.unpack(s.pack()).modes    # [1, 0]   -- want [1]

s = proto._make_param_esp_transform(Parameter.ESP_TRANSFORM, version=2, suites=[1])
s.pack()          # 0fff000400000100000000  -- 11 octets, declared len=4
ESPTransformParameter.unpack(s.pack()).suites       # [1, 0]   -- want [1]

So a one-element list comes back with a spurious second element.

Through the full HIP() parser it is worse than a wrong value — it reports
AttributeError: 'NATTraversalModeParameter' object has no attribute 'modes',
an outright parse failure.

Mechanism

The same three-way disagreement #466 documented for TRANSPORT_FORMAT_LIST: the
maker computes len on a two-octet-per-entry assumption (2 for the reserved
field plus 2 * count, hence len=4 for one entry), while item_type packs one
octet per entry, and the length callback then reads len - 2 octets as that many
one-octet items — twice as many as the wire holds.

Why the existing tests miss it

They assert against the schema object's Python attribute without packing and
unpacking. That is the same blind spot #466's history identifies as having let the
TransportFormatListParameter bug survive: a suite total cannot distinguish a
correct field from one whose declared length and item width are mutually
inconsistent, because nothing ever round-trips it.

Suggested direction

item_type=EnumField(length=2, ...) at both sites, matching
HIPTransportModeParameter.mode, then a round-trip test through the public
maker
— not a hand-built schema with a self-consistent len the maker would
never produce — covering one, two and three entries. Check whether the generated
options-internet.pcap capture moves, and whether
EXPECTED_FAILURES['hip-parameter/NAT_TRAVERSAL_MODE'] and
['hip-parameter/ESP_TRANSFORM'] change shape; both are currently in the
sixteen-entry RECONSTRUCT: unsupported type <class 'tuple'> group.

Provenance

Surfaced by the review of #466 and verified independently on both trees before
filing. Deliberately not folded into #466, which was already at a
GOOD TO MERGE verdict for the TRANSPORT_FORMAT_LIST half.

Activity

  1. JarryShaw commented on Sep 18, 2026

    @JarryShaw
    OwnerAuthor

    Correcting my own issue text, prompted by #475's author being unable to reproduce one of the three symptoms I listed. They were right to say so rather than quietly match my wording.

    What reproduces exactly as filed, on a clean origin/main worktree at 5182ad0ce:

    modes=[1]  ->  declared len=4, packs 0260000400000100000000 (11 octets)
               ->  round-trips to [<NATTraversal.UDP_ENCAPSULATION: 1>, <NATTraversal.Reserved_0: 0>]
    

    The phantom trailing entry, the wrong declared length, and the 16-bit-vs-1-octet mismatch against RFC 5770 §5.4 and RFC 7402 §5.1.2 are all confirmed. That is the defect, and #475 fixes it.

    What does not reproduce: the AttributeError: ... has no attribute 'modes' I quoted. Through the full HIP() parser on main I get, instead:

    pcapkit.utilities.exceptions.FieldValueError: Field param has an option that consumed no
    data: <Parameter.Unassigned_0: 0> at offset 0 of 736, with 736 octet(s) of the option area
    left to parse
    

    preceded by two SchemaWarning: packet length < 0 warnings. #475's author independently hit two further variants — ProtocolError: HIPv2: invalid format via HIP.make(), since the buggy 11-octets-per-parameter length is not 8-aligned, and a SchemaWarning with silent corruption of a following parameter from a hand-built two-parameter packet.

    So the downstream symptom depends on how the malformed parameter is reached, and none of the three paths produces the AttributeError I recorded. I cannot now reconstruct which packet shape produced it; it may have come from a differently-built fixture during the original investigation, and I should have pinned the exact input alongside the message. Treat that line of the issue as unverified and the wire-level reproduction above as the authoritative one.

    Worth noting the correction cuts in the fix's favour on one point: the parser-level failure on main is an in-library FieldValueError, not the bare stdlib exception an AttributeError would have been. So this issue never had the "bare exception escapes" character that #469 and #465 have — it is purely a wire-format width defect.

  2. added
    bugIssues reporting a defect (set by the bug report template; a default, not an assessment)
    on Sep 22, 2026
  3. added this to the 1.5 milestone on Oct 6, 2026
  4. moved this to Done in PyPCAPKiton Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIssues reporting a defect (set by the bug report template; a default, not an assessment)

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions