Skip to content

ci: record per-test timing and runner CPU for the integration leg - #1272

Merged
mforce merged 2 commits into
mainfrom
ci/integration-timing
Oct 12, 2026
Merged

mforce merged 2 commits into
mainfrom
ci/integration-timing

Conversation

@mforce

@mforce mforce commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Step 0 of #1271. This PR only measures: it changes no test and no product code.

Same-day runs of the integration leg range from 10.7 to 16.6 minutes. The console log gives no per-test start or end times and does not name the CPU. Steps 1 to 3 of #1271 need both to prove their effect on CI.

What the integration leg records now

  • A Show runner CPU step prints Model name and CPU(s) from lscpu. A step-level if limits it to the integration leg. The job has no new if, continue-on-error or fail-fast change (CI: measure where Build and test's 10 minutes go, and what the backend suite covers #775).
  • The dotnet test command for the integration leg gets --logger "trx;LogFileName=integration.trx". The blame-hang arguments stay as they are, and the other three legs expand to the same command as before.
  • An Upload integration timings step uploads TestResults/integration.trx as the integration-trx artifact, kept 14 days, whether the tests pass or fail (always()).

The trx holds test names, timings, outcomes, failure messages and stack traces. xUnit v2 does not capture per-test console output, and no integration test uses ITestOutputHelper. The trx's run-level StdOut repeats the test process console (Testcontainers and host log lines), and the Test step's public log already prints those lines. The disposable-credential argument of the existing hang-diagnostics upload covers this artifact too.

Not included:

  • A step-summary table. tools/test-timing/summarize.py also needs run.json, console.log and Docker events, so it cannot run on the trx alone.
  • CLUCKWORK_TEST_TIMING. It only prints fixture phase timings to the console, and the A/B does not use them.

Verification

  • ci.yml parses with yaml.safe_load.
  • SchemaDocsTests and SolutionTestProjectSplitTests read ci.yml. Run locally with a filter: 5/5 pass.
  • SecurityHeadersTests, run locally with the leg's blame and results-directory arguments plus the new logger: 8/8 pass. The trx lands at <results-dir>/integration.trx, the path the upload names, and every UnitTestResult has startTime and endTime.

Part of #1271

On this PR's CI run

Run 38290032941 at e811bdc passed every job.

  • The integration leg printed Model name: AMD EPYC 7763 64-Core Processor and CPU(s): 4. The leg took 16m39s.
  • The integration-trx artifact (1.2 MB zipped) holds 307 classes. The serial integration collection (128 classes) ran from 17.0 s to 902.5 s, and no other test was running for 509 s of that span. Its largest classes were OAuthFailClosedTests (161 s), McpAuthorizationTests (86 s), RecordDailyEntryToolTests (75 s), McpToolErrorTests (72 s) and AccountLifecycleCommandTests (60 s).

The integration leg prints its runner's CPU model and count, writes a trx
with millisecond per-test start and end times, and uploads it as the
integration-trx artifact (14 days) whatever the outcome. Measurement only:
no test or product code changes.

Part of #1271
@mforce mforce added this to the Platform hardening milestone Oct 12, 2026
@mforce
mforce merged commit 391036d into main Oct 12, 2026
19 checks passed
@mforce
mforce deleted the ci/integration-timing branch October 12, 2026 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant