Repository navigation
LOCATOR_SET is conformant only by accident: nested Locator.len shadows the parameter in padding, and len is in 4-octet units where RFC Length is bytes #679
Description
Activity
Owner decision: option (b) — narrow #664, leave
LOCATOR_SET's padding aloneThe owner's call, on the recommendation above: narrow #664 so it does not change
LOCATOR_SET's padding, and let this issue carry both defects.Why this is the safe answer, restated so the reasoning survives
The two defects cancel exactly — old total
4 + 24n + 4 = 24n + 8, and because24n ≡ 0 (mod 8)the RFC total forLength = 24nis also24n + 8, identical for everyn. So leaving both in place ships nothing wrong on the wire:LOCATOR_SETstays byte-for-byte whatmainemits today, which is what RFC 7401 §5.2.1 asks for.Option (a) — repairing
leninside #664 — was rejected not because it is wrong but because it is a different change: it alters aLengthfield on the wire, alters the publicData_LocatorSetParameter.length, and needs the reader'sListField(length=lambda pkt: pkt['len'])re-verified, since that consumes the same quantity. Folding that into an already-breaking46-parameter padding correction would make one PR carry two unrelated wire-format changes, and the cancellation means there is no urgency forcing them together.What #664 becomes
A padding correction for 45 of 46 HIP parameters, with
LOCATOR_SETexplicitly excluded and the exclusion documented in the code rather than left as an accident. Thedocs/source/changelog/1.5.0.rstentry on #657 needs its claim narrowed too — it currently says the change makes HIP records conformant, which is true of the 45 and not ofLOCATOR_SET.What this issue now owns
Both defects, together, because neither is fixable alone:
LocatorSetParameter.paddingreading the nestedLocator.leninstead of the parameter's, viaListField's shared packet context.- The parameter's
lenbeingsum(Locator.len)in 4-octet units where the RFC'sLengthis a byte count.
Whoever takes this must fix both in one change and re-verify the reader, and must assert the packed total against
11 + Length - (Length + 3) % 8for several locator counts rather than against a single pinned literal — a pinned literal is what let the cancellation hide for as long as it did.A worker is narrowing #664 now. This issue stays open.
The narrowing has landed, and one measured caveat for whoever takes this
#664 is narrowed as decided:
LOCATOR_SETkeeps the pre-#651 padding expression at
both of its sites —LocatorSetParameter.paddingand the reported record length in
_read_param_locator_set— so that parameter is unchanged in its octets and its
data model. Verified byte-for-byte againsta18846c8fover 15 construction cases and
3 parse cases, not just the three lengths the decision quoted. #664 is now a padding
correction for 45 of 46 parameters, guarded by a test that asserts the exclusion is
exactly one parameter and which one.The caveat: the cancellation is narrower than
24n + 8The decision's arithmetic — old total
4 + 24n + 4 = 24n + 8, RFC total for
Length = 24nalso24n + 8— is correct, and it is what makes leaving the line
alone safe. But it holds only for a set of plain IPv6 locators with n ≥ 1. A
locator carrying an SPI is 28 octets rather than 24, so24n ≡ 0 (mod 8)stops
applying; and an empty set has no nested locator to shadowlenat all. Measured on
this branch and ona18846c8f, byte-identically on both:case declared lenpacked multiple of 8? RFC total for a byte-count Lengthempty, n=0 0 4 no 8 plain, n=1 4 32 yes 32 plain, n=2 8 56 yes 56 SPI, n=1 5 35 no 32 SPI, n=2 10 63 no 64 plain then SPI 9 59 no 56 SPI then plain 9 60 no 56 So four of those seven shapes are not 8-aligned today, and one of them — the
empty set — is the single remaining violation in the RFC-only fixture walk over
options-internet.pcap, which goes 43 → 1 on #664. That frame is worth naming
precisely because it is easy to misread, and I did misread it at first: frame 102's
parameter area is00 c1 00 00 00 c1 00 00, two emptyLOCATOR_SETrecords of four
octets each where11 + 0 - (0 + 3) % 8 = 8is required. ALength-driven reader
advances 8 and consumes the second copy's header as padding.What this means for the fix here. Asserting the packed total against
11 + Length - (Length + 3) % 8for several locator counts, as this issue already
requires, is necessary but not sufficient — a sweep over n = 1..5 of plain locators
passes today by cancellation and would pass a half-fix too. The cases that
discriminate are the ones above: n = 0, the SPI/LocatorDatavariant, and a mixed
set in both orders, since those are the shapes where the two defects do not cancel
and where a fix to one of them alone shows up immediately.Two further things found while measuring, both pre-existing and neither fixed:
- The SPI locator path is hard to reach through the public maker at all:
_make_param_locator_setsetslength = 5whenspiis given but leavestype
at its default0, andlocator_value_selectoraccepts onlytype == 0, len == 4
ortype == 1, len == 5— sospi=without an explicittype=1raises
FieldValueError: invalid locator type or length. Worth knowing before writing the
tests for this issue, since the discriminating cases need that path. _read_param_locator_set's reportedlengthis derived from the same wronglen,
so it moves whenlendoes. It is excluded from fix(hip): pad HIP parameters to the record length RFC 7401 gives, not the contents (#651) #664 for exactly that reason and
should move with the pair rather than separately.
Credit where due: the narrowness of the cancellation was found by the cross-review on
#664, which went looking for locator shapes the24nargument does not cover rather
than re-checking the ones it does.- The SPI locator path is hard to reach through the public maker at all:
Correction to my own analysis above: the cancellation is not universal
I wrote that the two defects cancel "exactly" and that the totals are "identical for every n". That is overstated, and it changes how this issue should be read. The correction came from #664's worker via its cross-review, and I have re-measured the decisive case myself rather than relaying it.
What I verified directly
On
origin/main, CPython 3.14.7, editable finder stripped viagetattr(f, '__module__', ''),pcapkit.__file__asserted to a throwaway worktree before any other import:empty LOCATOR_SET packed = 4 ** NOT 8-ALIGNED ** (RFC wants 8) plain IPv6, n = 1 packed = 32 aligned plain IPv6, n = 2 packed = 56 alignedAn empty locator set packs 4 octets where the RFC wants 8, on
main, today. So the cancellation does not hold for every shape — it holds for homogeneous plain-IPv6 sets, which is what myn = 1, 2, 5measurements happened to sample.What I could not verify, stated as such
#664's worker additionally reports, measured identically on both trees:
SPI n=1→ 35,SPI n=2→ 63, mixed → 59/60 — four of seven shapes not 8-aligned, before and after. I could not reproduce those: my attempt to build an SPI locator used the wrong key and raisedFieldValueError: invalid locator type or length. Those three figures are its measurement, not mine. The empty case above is mine and is sufficient on its own to retract the universal claim.Why this matters for the decision, which does not change
Option (b) — leave
LOCATOR_SET's padding alone — remains the right call, but for a weaker and more honest reason than I gave.I argued it ships "nothing wrong on the wire". That is true only of the homogeneous plain-IPv6 shapes. For the empty set and the SPI shapes,
mainis already non-conformant and stays so. So leaving the line alone is the safe choice — it changes nothing and breaks nothing — rather than the correct one. The parameter is not conformant today and will not be until both defects here are fixed together.The practical consequence for whoever takes this issue: do not assume a passing round trip or an aligned total means the parameter is right. Assert against
11 + Length - (Length + 3) % 8across several locator shapes — empty, plain, SPI-bearing, mixed — not merely several locator counts. Sampling counts within one shape is exactly how I reached the wrong conclusion, and how the cancellation stayed hidden for as long as it did.#664's code comment at the excluded site and its PR body both now carry this narrower framing.
- added 6 commits that reference this issue
on Sep 23, 2026 - added a commit that references this issue
on Sep 23, 2026 3 remaining items
- added 13 commits that reference this issue
on Sep 24, 2026
Two pre-existing defects in
LOCATOR_SETcancel each other exactly, so the parameter is RFC-conformant onmainby coincidence rather than by construction. Fixing either one alone makes it non-conformant. #664 fixes the padding half and the parameter is now four octets short at every locator count.Measured on both trees, independently of the PR author
CPython 3.14.7,
__editable__*stripped fromsys.meta_path,pcapkit.__file__asserted to each worktree before any other import,HIP._make_param_locator_set(Parameter.LOCATOR_SET, version=2, locator_set=[...])then.pack():schema.lenmain@0c7f2b7c9819dbbd6aLength = 24nRFC total from §5.2.1's
Total Length = 11 + Length - (Length + 3) % 8.The two defects, and why they cancel
1.
LocatorSetParameter.paddingreads the nestedLocator.len, not the parameter's.ListFieldpacks eachLocatorinto the shared packet context, and the nestedlenshadows the parameter's own. An IPv6 locator always hasLocator.len == 4, so the old padding expression always appended exactly 4 octets regardless of locator count.2. The parameter's
lenissum(Locator.len)— in 4-octet units — where the RFC'sLengthis a byte count. Visible in the table above: contents of 24, 48 and 120 octets giveschema.lenof 4, 8 and 20, i.e.4nrather than24n.Old total:
4 + 24n + 4 = 24n + 8. And because24n ≡ 0 (mod 8), the RFC total forLength = 24nis also24n + 8. Identical for every n — which is why no test and no round trip has ever caught either defect.Why this is filed rather than folded into #664
The repair is not a one-expression change like
EncryptedParameter.datawas. It changes aLengthfield on the wire, changes the publicData_LocatorSetParameter.length, and requires re-checking the reader'sListField(length=lambda pkt: pkt['len']), which consumes the same quantity. That is the same class of change as #672, not a coupled fix that #664 can absorb safely.Consequence for #664 as it stands
#664 must not merge in its current state. It would take
LOCATOR_SETfrom conformant to four octets short, which is a regression in on-wire output even though every individual change in it is correct. Two ways out, and the choice is the owner's:leninside fix(hip): pad HIP parameters to the record length RFC 7401 gives, not the contents (#651) #664 — correct on both counts, but widens an already-breakingPR into a wireLengthchange plus a reader re-check.LOCATOR_SET's padding alone — keeps the accidental conformance, keeps the PR small, and this issue then carries both defects. My recommendation, because it separates a 46-parameter padding correction from a single parameter's length-unit bug, and because the cancellation means (b) ships nothing wrong.Found by the #664 worker while fixing a stale test pin, disclosed rather than absorbed, and reproduced here independently before filing.