Skip to content

fix(toolkit): scapy reads nothing in a zero-length frame, and a hand-built packet off its octets (#1676) - #1683

Merged
JarryShaw merged 1 commit into
mainfrom
fix/1676-scapy-zero-length-frames
Oct 13, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/1676-scapy-zero-length-frames

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly

  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])

  • Use Preview tab to see how your pull request will actually look like

  • Searched for similar pull requests

  • Followed the coding style (make pylint, make mypy, make isort)

  • make test passes, and a test case covers the change — tests cover it; only the scapy, toolkit, engine and integration parity selections were run, not the whole suite

  • Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Description of your pull request and other information

Root cause: Scapy dissects only a non-empty string, so a zero-length frame comes back as a layer of defaults. At RAW and IPV4 that is an IPv4 header whose ihl and len are None, and bytes() builds 20 octets for it out of nothing. A packet built by hand has the same None fields.

  • _frame(): a packet's frame octets. That is packet.original for a packet read from a capture (which has a wirelen), else bytes(packet). _network returns no layer for an empty frame. packet2dict and packet2frame now report b'' for one, and a hand-built packet's frame record carries its built octets, not b''.
  • ipv4_reassembly, _tcp_segment and _within_datagrams read the IHL, Total Length and Data Offset from the octets. That stops the TypeError on hand-built layers.

Measured on 3.14.8 (scapy 2.7.0) and 3.11.15 (scapy 2.8.0), with the same results on both:

  • 104 read-frame cases, comparing default, dpkt and scapy: zero-length and cut-short frames at RAW, IPV4, IPV6, NULL (both byte orders), LOOP and ETHERNET, each followed by control frames. On main, scapy raised on the zero-length RAW and IPV4 frames and agreed on every other case. With the fix, all 104 agree. dpkt agrees on all 104 before and after.
  • 15 hand-built packets, compared with the default engine reading bytes(packet): 19 adapter calls raised TypeError on main, and all now match. The one exception is tracing an IP-rooted packet, which still raises MissingKeyError (Registries mint a permanent member for every unrecognised value: 1,169 extend_enum sites across 113 registries #775, by design).
  • 1590 packets from 22 sample captures: original == bytes(packet) for every one, so packet2dict output does not change.

New tests/toolkit/test_scapy_zero_length_frames_1676_unit.py has 5 tests. All 5 fail on main and pass with the fix, on both interpreters. Reverting any one of the 7 changes makes at least one test fail. Coverage of scapy.py goes from 99.58% to 99.59%.

Closes #1676

…built packet off its octets (#1676)

Scapy dissects a packet only from a non-empty string, so a zero-length frame
comes back as a layer of defaults: at RAW and IPV4 an IPv4 header whose ihl
and len are None, and bytes() of it builds a 20-octet header from nothing.

- _frame: the octets of a packet's frame -- packet.original for one read
  from a capture (which carries a wirelen), else bytes(packet).
- _network: no layer for a frame with no octets, so no adapter hands
  over an input for it and IPv4 reassembly no longer raises TypeError.
- packet2dict, packet2frame: report _frame(), so a zero-length frame
  carries b'' rather than a built header, and a hand-built packet's frame
  record carries the octets it builds rather than none.
- ipv4_reassembly, _tcp_segment, _within_datagrams: read the IHL, Total
  Length and Data Offset off the octets, which Scapy leaves None on a
  hand-built layer, so IPv4 and TCP reassembly and TCP flow tracing no
  longer raise on one.

New tests/toolkit/test_scapy_zero_length_frames_1676_unit.py: 5 tests, all
failing on main and passing with the fix on 3.14 and 3.11.
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 13, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 67cf7b7dd. A Sonnet cross-review checked the Opus worker's change. I re-ran the new test on 3.14 and 3.11 (5 passed each) and test_scapy_unit.py (20 passed).

Root cause. Scapy dissects only a non-empty string. A zero-length frame therefore comes back as a layer of defaults: an IPv4 layer whose ihl and len are None, at RAW and IPV4. ipv4_reassembly then computed None * 4 (scapy.py:856).

The fix. It tells a frame read from a capture from a hand-built packet by wirelen, which every Scapy reader sets. A zero-length read frame then hands over nothing, as the default engine and dpkt do. ihl, the total length and the TCP Data Offset are now read from the octets.

What the reviewer measured. 376 of its own cases, each a zero-length, truncated or malformed frame followed by six controls, across 9 link types and both scapy releases. On main the only disagreements were the two crashes. On the branch there are none, with no new exception.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 89.44% (unit tier, Python 3.14, 67cf7b7dd, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 3 20 3 95.16%
pcapkit/const 18858 954 2366 868 91.06%
pcapkit/corekit 2366 67 760 35 96.48%
pcapkit/dumpkit 275 3 100 3 98.40%
pcapkit/foundation 3031 152 1074 35 94.23%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17166 226 4580 190 98.07%
pcapkit/toolkit 1364 283 564 6 76.71%
pcapkit/utilities 494 4 146 4 98.75%
pcapkit/vendor 4417 2342 1010 157 43.38%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 7aeeaf8 into main Oct 13, 2026
50 checks passed
@JarryShaw
JarryShaw deleted the fix/1676-scapy-zero-length-frames branch October 13, 2026 21:54
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 13, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 13, 2026
JarryShaw added a commit that referenced this pull request Oct 14, 2026
Third-party engines are best effort (#1687), so the scapy code that
exists only to match the default engine comes out:

- revert #1683, #1663, #1652 and #1631, and the scapy hunks of #1614: the
  link-type and loopback-family walk, the frame and extension-header
  bounds, the IPv6 chain walk over the octets, the stops at GRE, VXLAN,
  PPPoE and the other layers the default engine leaves raw and at ICMP
  quotes, and the IPv4 header checks
- keep #1624's walk (TCP behind AH or an IPv6 extension header in IPv4,
  and the later-fragment stop), #1605's keying by the carrying IP header
  and #1589's trailer cut; the walk calls _walk_extension_headers, which
  the pypcapfile revert keeps
- re-land as guards: a zero-length frame is no IPv4 input and has no
  packet2dict octets (#1676); IPv4 in IPv6 ends at the Payload Length
  (#1623); a packet built by hand is read off its octets (#1676);
  ipv6_reassembly takes only its own IPv6 layer's Fragment header, and
  none in a later IPv4 fragment
- delete the four scapy agreement modules and test_scapy_unit's
  default-engine checks; the guards' tests assert constants
- docs: scapy applies the extension-header bound only behind IPv4 and the
  frame-layer bound nowhere, and its silent gaps are listed
  where gaps are said to be announced

The scapy toolkit and engine modules, the shared modules that run scapy,
and the project and tier guards pass on 3.14 (scapy 2.7.0) and 3.11
(scapy 2.8.0).
JarryShaw added a commit that referenced this pull request Oct 14, 2026
Third-party engines are best effort (#1687), so the scapy code that
exists only to match the default engine comes out:

- revert #1683, #1663, #1652 and #1631, and the scapy hunks of #1614: the
  link-type and loopback-family walk, the frame and extension-header
  bounds, the IPv6 chain walk over the octets, the stops at GRE, VXLAN,
  PPPoE and the other layers the default engine leaves raw and at ICMP
  quotes, and the IPv4 header checks
- keep #1624's walk (TCP behind AH or an IPv6 extension header in IPv4,
  and the later-fragment stop), #1605's keying by the carrying IP header
  and #1589's trailer cut; the walk calls _walk_extension_headers, which
  the pypcapfile revert keeps
- re-land as guards: a zero-length frame is no IPv4 input and has no
  packet2dict octets (#1676); IPv4 in IPv6 ends at the Payload Length
  (#1623); a packet built by hand is read off its octets (#1676);
  ipv6_reassembly takes only its own IPv6 layer's Fragment header, and
  none in a later IPv4 fragment
- delete the four scapy agreement modules and test_scapy_unit's
  default-engine checks; the guards' tests assert constants
- docs: scapy applies the extension-header bound only behind IPv4 and the
  frame-layer bound nowhere, and its silent gaps are listed
  where gaps are said to be announced

The scapy toolkit and engine modules, the shared modules that run scapy,
and the project and tier guards pass on 3.14 (scapy 2.7.0) and 3.11
(scapy 2.8.0).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

fix(toolkit): scapy ends the extraction on a zero-length RAW frame when IPv4 reassembly is on

1 participant