Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,8 @@ rewrites the regenerated constants, not as a check, so nothing verifies import o
request.

One trap: `make vermin` redirects its report into `temp/vermin.txt` rather than to your terminal,
and vermin exits 1 because `vermin.ini` sets `targets = 3.6` against a real floor of 3.11. Make
gives up at the redirect with `make: *** [vermin] Error 1`, the report left in the file and the line
and vermin exits 1 because `vermin.ini` sets `targets = 3.6` against the 3.11 minimum it detects.
Make gives up at the redirect with `make: *** [vermin] Error 1`, the report left in the file and the line
that would have opened it never reached. `make vermin-ci` runs the same flags to stdout, which is why
CI uses it and why it is easier to read at a desk. Running the lot before you push still saves a
review round.
Expand Down Expand Up @@ -199,9 +199,9 @@ BLANK LINE
footer (optional)
```

`type` is one of `feat`, `fix`, `docs`, `test`, `perf`, `refactor`, `ci` or `chore` — those are the
ones in use. `scope` names the part of the package affected — `tcp`, `corekit`, `schema`, `ipv4`,
`vendor` — and several are separated by commas inside the parentheses, as in `fix(link,internet):`
`type` is one of `feat`, `fix`, `docs`, `test`, `perf`, `refactor`, `ci`, `chore` or `release` —
those are the ones in use. `scope` names the part of the package affected — `tcp`, `corekit`,
`schema`, `ipv4`, `vendor` — and several are separated by commas inside the parentheses, as in `fix(link,internet):`
or `test(utilities,foundation):`. The scope may be omitted where nothing narrower than the whole
project applies, as in `docs:`.

Expand Down
6 changes: 2 additions & 4 deletions docs/source/contributing/conventions/documentation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -230,10 +230,8 @@ check the members one at a time.** Several of :issue:`719`'s findings were of ex
shape:

* A sweep asserted that every ``.. module::`` target in the documentation resolved.
One did not: :file:`docs/source/pcapkit/protocols/link/rarp.rst` (its path at the
time; now under :file:`application/`) declared
``pcapkit.protocols.data.link.rarp``, which has never existed, because RARP and
DRARP reuse ARP's data class. Fixed in ``68fbccd90``.
One did not: the RARP page declared ``pcapkit.protocols.data.link.rarp``, which has
never existed, because RARP and DRARP reuse ARP's data class. Fixed in ``68fbccd90``.
* :issue:`911`'s ruling -- export the sentinel objects and leave their types out -- was
read as describing all three modules that then held a sentinel. One ran the other
way: :mod:`pcapkit.corekit.fields.field` exported neither, so applying the rule
Expand Down
2 changes: 1 addition & 1 deletion docs/source/contributing/conventions/process.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Running the Repository
----------------------

The four pages before this one are about writing library code. The rulings here are
The five pages before this one are about writing library code. The rulings here are
about running the repository -- what an install carries, what a changelog entry is,
and what the issue and pull request labels mean. None of them is derivable from a
module, and none fits a code-convention page, so the owner ruled on
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,8 @@ they share (``__data__``, ``__layer__``, ``__schema__``, ``layer``) plus
``register`` and ``_read_protos`` -- but all three also exist on ``ProtocolBase``, so
the strict difference ``Link`` minus ``Application`` minus ``ProtocolBase`` is **empty**
and every one of them still resolves on ``OSPF``. What changes is which registry it
resolves *to*: ``OSPF.__proto__`` is now ``ProtocolBase.__proto__`` rather than
``Link.__proto__``, so OSPF no longer sees Link's EtherType entries. That is inert --
resolves *to*: ``OSPF.__proto__`` is ``ProtocolBase.__proto__`` rather than
``Link.__proto__``, so OSPF does not see Link's EtherType entries. That is inert --
nothing reaches OSPF by EtherType, and a ``-1`` lookup misses in both registries alike
and resolves to :class:`~pcapkit.protocols.misc.raw.Raw`, inserting nothing on the miss.
``RARP`` keeps ``Link``'s through ``ARP``.
Expand Down
7 changes: 3 additions & 4 deletions docs/source/contributing/conventions/registry-protocol.rst
Original file line number Diff line number Diff line change
Expand Up @@ -570,10 +570,9 @@ rather than changing it.
``Command.get`` upper-cases its key
before matching, which makes it look as though the base were case-insensitive --
deliberately, since :rfc:`959#section-5` treats FTP command codes identically
regardless of case. ``Method.get`` used to fold case the same way, but
:issue:`896` made it
case-sensitive instead: :rfc:`9110#section-9.1` says the HTTP method token is
case-sensitive, so ``Method.get('get')`` no longer resolves to
regardless of case. ``Method.get`` does not fold case, as ruled on :issue:`896`:
:rfc:`9110#section-9.1` says the HTTP method token is case-sensitive, so
``Method.get('get')`` does not resolve to
``Method.GET`` -- it builds its own unregistered member, preserving the
caller's exact casing, the same way an unrecognised value always does.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ What reaches users is the **object only**. The owner ruled on GitHub issue :issu
that the objects alone -- ``NULL`` and its siblings -- are exported to users, so a public
sentinel names its instance in its module's ``__all__`` and leaves the type out of it.
The type stays importable by its dotted path, for an annotation or an ``is`` guard; it
is ``import *`` that no longer offers it. A private sentinel such as ``ABSENT`` is in
is only ``import *`` that does not offer it. A private sentinel such as ``ABSENT`` is in
neither, which is what private means here -- dropping its leading underscore did not
add it to either list, and :class:`~pcapkit.corekit.sentinels.AbsentType` and
:data:`~pcapkit.corekit.sentinels.ABSENT` are documented on
Expand Down
15 changes: 8 additions & 7 deletions docs/source/contributing/pep.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,8 @@ whose *layout* is identical -- it holds the whole of the tag, and
``NotImplemented`` folder. **Those files are not a roadmap.** They are
scratch reminders, so a stub's presence does not mean a protocol is planned or
claimed, and its absence does not mean the protocol is unwanted:
``LINUX_SLL``, ``QUIC`` and **DTLS** have no stub and are on the list above,
``LINUX_SLL``, ``LINUX_SLL2``, ``QUIC`` and **DTLS** have no stub and are on the
list above,
while **NGAP** has none either and is implemented. Take this page as the
record and ignore the folder.

Expand Down Expand Up @@ -210,8 +211,8 @@ look like ordinary fields and are not:
DTLS
~~~~

**Not started**, and unlike everything in the list above it has no stub in the
tree at all -- only TLS/SSL does. It earns its own entry because the registries
**Not started**, and it has no stub in the tree -- of the TLS/SSL and DTLS pair,
only TLS/SSL does. It earns its own entry because the registries
have moved ahead of it, and a growing part of SCTP's surface now names a protocol
that does not exist:

Expand Down Expand Up @@ -820,9 +821,9 @@ from *wrong*, or it will cry wolf on the commonest capture there is —
model for how to spell "expected, not an error".

Sequenced for **wave 2 or 3**. The cryptographic half could be done sooner since
ESP has already laid the groundwork, but the checksum half genuinely wants the
parent-access question answered first, and that is worth doing deliberately
rather than as a side effect of a checksum patch.
ESP has already laid the groundwork, but the checksum half needs the IP
pseudo-header class defined first, and that is worth doing deliberately rather
than as a side effect of a checksum patch.

Release Plan — 1.5.0 in Two Steps
---------------------------------
Expand Down Expand Up @@ -969,7 +970,7 @@ dispatcher registers exactly three link types
work above rather than a separate concern — all four are link types the library
enumerates but cannot parse. It surfaced while deciding what an unresolvable
link-layer name should do: neither value is an honest stand-in for "unknown link
type", which is why the toolkit now raises instead of defaulting to
type", which is why the toolkit raises instead of defaulting to
``LinkType.NULL``.

**Wave 3 — the remaining protocols** from the same list, taken three or four at a
Expand Down
16 changes: 8 additions & 8 deletions docs/source/contributing/releasing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -186,10 +186,10 @@ pushed, trading a clean skip for a checkout failure; the equality pair skips
``needs.version_check.result == 'success'`` with no ``|| 'skipped'``. It
produces the evidence the gates read, so a skipped or cancelled
``version_check`` leaves them nothing to decide on. ``unit-tests`` is held to
the same bar for a different reason: it is the release test gate, and since
`#1052 <https://github.com/JarryShaw/PyPCAPKit/issues/1052>`__ it skips when
nothing is publishable, so accepting ``skipped`` there would publish
untested. ``github`` names it in ``needs:`` instead and relies on the implicit
the same bar for a different reason: it is the release test gate, and it skips
when nothing is publishable
(`#1052 <https://github.com/JarryShaw/PyPCAPKit/issues/1052>`__), so accepting
``skipped`` there would publish untested. ``github`` names it in ``needs:`` instead and relies on the implicit
``success()``. The gate's own ``if:`` is the union of the four publishers'
conditions, with ``!= 'true'`` so unknown evidence runs it; should the two
ever drift apart, the failure is a release that did not happen, which
Expand Down Expand Up @@ -257,7 +257,7 @@ Precautions
above is what prevents it: ``tag``, ``pypi`` and ``conda`` each check whether
*their own* artefact is missing rather than whether the ``v*`` tag exists, so an
incomplete release runs the jobs that did not finish instead of skipping them.
A half-finished release now self-heals on the next ``workflow_run``-triggered
A half-finished release self-heals on the next ``workflow_run``-triggered
attempt, or on re-running the workflow by hand -- see `Recovery`_ below.
``release_status`` is the other half: it runs unconditionally and reports,
with a ``::notice``, a ``::warning`` or a failing ``::error``, why a run
Expand All @@ -275,7 +275,7 @@ here), so the push only fires again after the tag is deleted -- which touches a
ref the release automation owns, and lands the retry back on the tag-push path,
where every job's guard is bypassed regardless of what has already gone out.
Neither is worth the risk of a double upload to an index that cannot take one
back, and neither is needed, since a plain re-run now self-heals; see
back, and neither is needed, since a plain re-run self-heals; see
`Recovery`_ below.

**``environment: pypi`` stays even though its reviewer is gone.** ``pypi``
Expand Down Expand Up @@ -305,8 +305,8 @@ actual fix rather than a best-effort suggestion: `Per-Job Evidence, Not a
Shared Proxy`_ above means ``tag``, ``pypi`` and ``conda`` each check whether
*their own* artefact is missing, so a re-run finishes whichever jobs did not
complete last time instead of skipping them on the ``v*`` tag's mere
existence. ``pypi`` was always safe to re-run (``skip-existing: true``);
``conda`` now is too, because each matrix leg checks Anaconda for its own
existence. ``pypi`` is safe to re-run (``skip-existing: true``), and so is
``conda``, because each matrix leg checks Anaconda for its own
platform/Python distribution before uploading and skips only that leg's
upload if it is already there.

Expand Down
3 changes: 1 addition & 2 deletions docs/source/contributing/workflows.rst
Original file line number Diff line number Diff line change
Expand Up @@ -260,8 +260,7 @@ The Skip Cascade (`#888 <https://github.com/JarryShaw/PyPCAPKit/issues/888>`__)

The two relationships above compose into a failure mode worth seeing on its
own graph. Create Release's first job, ``version_check`` -- ahead of the
``uses:`` call above since
`#1052 <https://github.com/JarryShaw/PyPCAPKit/issues/1052>`__ -- carries:
``uses:`` call above -- carries:

.. code-block:: yaml

Expand Down
24 changes: 14 additions & 10 deletions docs/source/ext.rst
Original file line number Diff line number Diff line change
Expand Up @@ -648,9 +648,9 @@ The following code snippet shows how to create a new dumper class:
# dumper registries, and use it to dump the extracted network packets.
class MyDumper(Dumper, fmt='pcap', ext='.pcap'):

# NOTE: This property is to define the file format of the dumper.
# It is expected to be a string, which is used as the file extension
# of the output file.
# NOTE: This property names the file format of the dumper, as a string.
# It is not the file extension -- that is the ``ext`` given at class
# definition above.
@property
def kind(self) -> 'str':
return 'pcap'
Expand Down Expand Up @@ -856,8 +856,8 @@ Callback Functions
~~~~~~~~~~~~~~~~~~

Callback functions can be registered on the reassembly and flow tracing
classes, and run at the end of the respective process -- to inspect the
reassembled datagrams or flows, or to discard some of them.
classes, to inspect the reassembled datagrams or traced flows as they are
produced.

.. seealso::

Expand All @@ -869,9 +869,13 @@ reassembled datagrams or flows, or to discard some of them.
- :func:`~pcapkit.foundation.registry.foundation.register_reassembly_tcp_callback`
- :func:`~pcapkit.foundation.registry.foundation.register_traceflow_tcp_callback`

A callback takes one argument -- the list of reassembled datagrams or flows --
and returns :obj:`None`; any return value is ignored.
A reassembly callback runs on every submission and takes one argument, the list
of datagrams that submission produced. A flow tracing callback runs as each flow
is finalised and takes that flow's
:class:`~pcapkit.foundation.traceflow.data.tcp.Index`. Either returns
:obj:`None`; any return value is ignored.

That list can be modified in place, but prefer not to: if the reassembly or
tracing result itself is wrong for your purpose, subclass the reassembly or
flow tracing class and change its algorithm instead.
A reassembly callback can modify its list in place, which discards datagrams,
but prefer not to: if the reassembly or tracing result itself is wrong for your
purpose, subclass the reassembly or flow tracing class and change its algorithm
instead.
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ Or install the latest version from the git repository:
.. code-block:: shell

git clone https://github.com/JarryShaw/PyPCAPKit.git
cd pypcapkit
cd PyPCAPKit
pip install -e .
# and to update at any time
git pull
Expand Down
2 changes: 2 additions & 0 deletions docs/source/pcapkit/foundation/reassembly/ip/ipv6.rst
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,8 @@ Terminology
| | |--> (int) packet range number
| |--> 'header' : (bytes) header buffer
| |--> 'datagram' : (bytearray) data buffer, holes set to b'\\x00'
| |--> 'timestamp' : (float) capture timestamp of the
| | first-arriving fragment
| |--> 'conflict' : (list) octet ranges on which an arriving
| | fragment disagreed with bytes already in
| | 'datagram'
Expand Down
44 changes: 25 additions & 19 deletions docs/source/pcapkit/foundation/reassembly/tcp.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@
TCP Datagram Reassembly
=======================

.. module:: pcapkit.foundation.reassembly.tcp

:mod:`pcapkit.foundation.reassembly.tcp` contains
:class:`~pcapkit.foundation.reassembly.reassembly.Reassembly` only,
:class:`~pcapkit.foundation.reassembly.tcp.TCP` only,
which reconstructs fragmented TCP packets back to origin.

.. autoclass:: pcapkit.foundation.reassembly.tcp.TCP
Expand Down Expand Up @@ -159,27 +161,31 @@ Terminology

.. code-block:: python

ip_info = ip.info
tcp_info = tcp.info

raw_len = len(tcp.packet.payload)
packet_dict = Info(
bufid = tuple(
ip.src, # source IP address
tcp.srcport, # source port
ip.dst, # destination IP address
tcp.dstport, # destination port
bufid = (
ip_info.src, # source IP address
tcp_info.srcport.port, # source port
ip_info.dst, # destination IP address
tcp_info.dstport.port, # destination port
),
dsn = tcp.seq, # data sequence number
ack = tcp.ack, # acknowledgement number
num = frame.number, # original packet range number
syn = tcp.flags.syn, # synchronise flag
fin = tcp.flags.fin, # finish flag
rst = tcp.flags.rst, # reset connection flag
len = tcp.raw_len, # payload length, header excludes
first = tcp.seq, # first sequence number of payload
last = tcp.seq + tcp.raw_len - 1,
# last sequence number of payload
header = tcp.packet.header, # raw bytes type header
payload = tcp.raw, # raw bytearray type payload
num = frame.info.number, # original packet range number
ack = tcp_info.ack, # acknowledgement
dsn = tcp_info.seq, # data sequence number
syn = tcp_info.flags.syn, # synchronise flag
fin = tcp_info.flags.fin, # finish flag
rst = tcp_info.flags.rst, # reset connection flag
header = tcp.packet.header, # raw bytes type header
payload = bytearray(
tcp.packet.payload), # raw bytearray type payload
first = tcp_info.seq, # first sequence number of payload
last = tcp_info.seq + raw_len - 1, # last sequence number of payload
len = raw_len, # payload length, header excludes
timestamp = float(
frame.time_epoch), # capture timestamp
frame.info.time_epoch), # capture timestamp
)

Both ``first`` and ``last`` are absolute TCP sequence numbers and
Expand Down
Loading
Loading