Repository navigation
ci(timing): the project ordering leg will hit its 20-minute step cap within about a week #2254
Description
Problem. The weekly CI timing report (run 40288101249, on main 91147a182) fails its two hard checks for Plain unittest ordering (project), step "Run tests/project and the root-level modules under plain unittest":
- H2: the step's p90 is 11.8 min, at or above 50% of its 20-minute cap.
- H4: on a linear fit it reaches the 20-minute cap in 7.3 days.
tests/project keeps growing: this leg, run locally, went from 2290 tests to 2388 over rounds 5 and 6. The report also names tests.test_tier_guard as the slowest module (over 30 s summed on 3.13), with several WorkflowAgreementTests cases over 10 s each.
Options
- A. Raise the step cap. This buys time, but the trend continues.
- B. Shard the step into two
projectordering legs by module, as the other ordering legs are split. Each half stays well under the cap. - C. Make the slow tests cheaper.
WorkflowAgreementTestslooks like it re-reads or re-expands the workflows for each case; caching that parse per class would cut most of the module's time.
Recommendation: B plus C. B removes the trend, and C is cheap and helps every leg that runs the tier guard. A alone only defers the problem.
Activity
JarryShaw commentedon Oct 26, 2026
B plus C
JarryShaw commentedon Oct 26, 2026
and might worth review and consolidate and remove unnecessary duplicated tests.
ci(ordering): shard the project ordering leg and cache the tier guard…
ci(ordering): shard the project ordering leg and cache the tier guard…
ci(ordering): shard the project ordering leg and cache the tier guard…
JarryShaw commentedon Nov 1, 2026
The
ci-timing-reportartifact of run 40288101249 is artifact 12636501186; its slowest-module table (Python 3.13, median of 5 push runs, top 8 of the 10 rows) follows.tests.test_tier_guardtests.protocols.test_frame_layer_limit_1610_unittests.protocols.internet.test_ext_chain_limit_rebuild_1619_unittests.protocols.misc.test_pcapng_unittests.test_support_helperstests.utilities.test_exceptions_excepthooktests.const.test_const_enum_builtin_paritytests.foundation.test_extraction_no_eofA fresh baseline was dispatched as run 41303273918 (#2554), after #2258 and #2244 changed the job shape and selection.