Repository navigation
perf(const): load the TCP and UDP app-type registries on first use (#1538) - #1560
Merged
Merged
Conversation
Contributor
|
Coverage: 90.03% (unit tier, Python 3.14,
Per-file detail: the |
…1538) pcapkit.const.reg.apptype imported its tcp and udp submodules (about 49,700 generated lines) with the package, though nothing on any import path reads them; they were about 30% of `import pcapkit`. - The package binds TCP and UDP, and the tcp and udp submodules, from a PEP 562 __getattr__ on first access. dir(), __all__ and star-imports answer as before. - AppType.__registries__ becomes a dict subclass that holds a ModuleDescriptor for each lazy registry until a read resolves it, so AppType.get() and PortEnumField load only the transport they dispatch to. Keys, their order and every read view are unchanged. - SCTP and DCCP cost about 3 ms together and stay eager. - tests/conftest.py imports both registries before pinning its module snapshot, so its restore no longer leaves a second TCP and UDP behind. New tests show a cold import leaves both unimported, pin every read path and the conftest pin; the const, corekit, protocols, foundation and project suites pass.
JarryShaw
force-pushed
the
perf/1538-p7-lazy-apptype
branch
from
October 10, 2026 04:05
6bd1988 to
ed83fd8
Compare
Owner
Author
|
Verdict: GOOD TO GO at
Non-blocking nit: the comment above |
5 of 13 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please follow the guide below
You will be asked some questions, please read them carefully and answer honestly
Put an
xinto all the boxes [ ] relevant to your pull request (like that [x])Use Preview tab to see how your pull request will actually look like
Searched for similar pull requests
Followed the coding style (
make pylint,make mypy,make isort)make testpasses, and a test case covers the change (run one directory at a time, not as onemake test)Added a changelog entry under
docs/source/changelog/and regeneratedCHANGELOG.md, if the change is user-visible: N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657What is the purpose of your pull request?
Tick the commit type your subject line carries.
fix— corrects a defectfeat— adds a featureperf— changes performance, not behaviourrefactor— changes neither behaviour nor performancetest— tests onlydocs— documentation onlyci— workflows or build toolingrelease— bumps the version or rolls up a distributionchore— anything elseDescription of your pull request and other information
Refs #1538 (P7).
pcapkit/const/reg/apptype/__init__.pynow importstcp/udpon first use: a PEP 562__getattr__forTCP/UDP/tcp/udp, andAppType.__registries__holds aModuleDescriptorper lazy registry until a read resolves it. Nothing on any import path needed them eagerly (import pcapkit,pcapkit.all,from pcapkit import *, extraction, engines, registry, transport/schema modules all checked). The other proposed importers (registry/protocols.py,fields/numbers.py,corekit/enum.py,const/reg/__init__.py) only import the baseAppType, so they are unchanged.Measured (3 runs each):
-X importtime -c 'import pcapkit'695/700/722 → 516/521/503 ms (tcp+udp were 206–214 ms, 29.5%, not 33%).tests/utilities/test_logging.py+test_const_enum_no_mint.py+test_const_apptype_split_unit.py: 43.0/42.8/43.1 → 37.7/37.4/40.8 s. One run oftests/const: 84.8 → 75.3 s, executingtcp.py77 → 27 times (udp.py77 → 26). Transport + application + capture round trip:tcp.py76 → 16,udp.py76 → 6 executions; 174.9/165.8/166.1 → 183.6/156.9/158.1 s (the first pair ran at load ~6, the rest at ~3). Estimate: ≤ ~0.19 s per cold re-import, minus ~0.1 s back for each registry a class then touches; per leg unmeasured.Unchanged, pinned by tests:
dir(),__all__, star-import,__registries__keys/order/len/inand every read view, pickle/copy of members, a port registered before vs after first use, vendor output (regenerated all four from one IANA CSV under both inits: identical, and identical to the committed files). Suites green one directory at a time: const, corekit, protocols (all), foundation, interface, dumpkit, utilities, vendor, cli, toolkit, project, integration, top-level.Contract changes for the
breakingcall:import pcapkitno longer leavespcapkit.const.reg.apptype.tcp/udpinsys.modules, norTCP/UDPin the package__dict__.AppType.__registries__is adictsubclass;__registries__.getcosts 129 ns vs 56 ns,AppType.get761 vs 691 ns.tests/conftest.pynow imports both registries before pinning its module snapshot. Without that, the restore dropped them after the first test that loaded them, and a later direct import built a secondTCP/UDP; under xdist this failedtest_the_audit_population_figures_are_the_measured_oneswith127 != 129on 3.10–3.13.SharedImportProbe/PinnedSnapshotTestsfail without the warm-up.