Repository navigation
test: a long single-process unittest leg slows as it runs (heap growth across modules) #2750
Description
Describe the bug
The Plain unittest ordering (protocols (rest)) cell ran all its modules in one process, and it slows down the longer that process runs. Between a25eb0e39 and ab7be2fb0 the run step went from 527 s to 1096 s. The modules added in that range account for 267 s. The modules already there went from 525 s to 826 s, a slowdown of about 1.35× near the start of the run and 1.85× at the end. Peak RSS reached 1.85 GiB locally. On dea098b15 the step hit its 20-minute cap, which turned main red.
Reproduction
These are per-module timings from that cell's job logs on the runs above, and a local run of util/run_unittest_leg.py protocols --exclude protocols/internet with the repo venv. The cell is now halved (21ec0a6f1, pushed to main), which fixes the timeout but not the growth.
Expected behavior
A module's run time does not depend on how many modules ran before it in the same process. If something keeps growing across tests, it gets found. It could be a class registry, a cache, held captures or held warnings. If it is library state, it is fixed in the library; if it is test state, it is cleaned up in the tests.
Additional context
util/run_unittest_leg.py already forces a gc.collect every few tests, so whatever grows is reachable, not garbage.
Activity
JarryShaw commentedon Nov 3, 2026
A fix is in progress on a branch off
21ec0a6f1. It profiles the leg (tracemalloc and object counts per module) to find what stays reachable and grows. The fix goes in the library if the state is the library's, and in the tests if it is theirs, with before/after RSS and slowdown numbers.JarryShaw commentedon Nov 3, 2026
The PR is #2752 (tests only). It cuts the end-of-leg generations from 13 to 5. One residual needs your call.
Needs deciding: a long-lived generation, kept by a test module's globals, saves another generation's hook in
_install_excepthook, sometimes two hops deep. That is what still links 3 → 5 generations intransport/. The chaining is that function's documented design._install_excepthookso a re-import replaces the previous hook instead of chaining it. That is a library change.Recommendation: 1. It has no effect on users, and the halved leg now runs well under its cap. I'll go ahead with 1 unless you pick 2.
test: release purged pcapkit generations in a long unittest leg (#2750)…
JarryShaw commentedon Nov 3, 2026
#2752 has merged and closed this. The open question above (the cross-generation excepthook chaining) goes with option 1, as recommended: it is accepted as a test-only artefact. If you want option 2, say so and it gets its own issue.