Repository navigation
ci: record per-test timing and runner CPU for the integration leg - #1272
Merged
Merged
Conversation
The integration leg prints its runner's CPU model and count, writes a trx with millisecond per-test start and end times, and uploads it as the integration-trx artifact (14 days) whatever the outcome. Measurement only: no test or product code changes. Part of #1271
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 0 of #1271. This PR only measures: it changes no test and no product code.
Same-day runs of the integration leg range from 10.7 to 16.6 minutes. The console log gives no per-test start or end times and does not name the CPU. Steps 1 to 3 of #1271 need both to prove their effect on CI.
What the integration leg records now
Show runner CPUstep printsModel nameandCPU(s)fromlscpu. A step-leveliflimits it to the integration leg. The job has no newif,continue-on-errororfail-fastchange (CI: measure where Build and test's 10 minutes go, and what the backend suite covers #775).dotnet testcommand for the integration leg gets--logger "trx;LogFileName=integration.trx". The blame-hang arguments stay as they are, and the other three legs expand to the same command as before.Upload integration timingsstep uploadsTestResults/integration.trxas theintegration-trxartifact, kept 14 days, whether the tests pass or fail (always()).The trx holds test names, timings, outcomes, failure messages and stack traces. xUnit v2 does not capture per-test console output, and no integration test uses
ITestOutputHelper. The trx's run-levelStdOutrepeats the test process console (Testcontainers and host log lines), and the Test step's public log already prints those lines. The disposable-credential argument of the existing hang-diagnostics upload covers this artifact too.Not included:
tools/test-timing/summarize.pyalso needsrun.json,console.logand Docker events, so it cannot run on the trx alone.CLUCKWORK_TEST_TIMING. It only prints fixture phase timings to the console, and the A/B does not use them.Verification
ci.ymlparses withyaml.safe_load.SchemaDocsTestsandSolutionTestProjectSplitTestsreadci.yml. Run locally with a filter: 5/5 pass.SecurityHeadersTests, run locally with the leg's blame and results-directory arguments plus the new logger: 8/8 pass. The trx lands at<results-dir>/integration.trx, the path the upload names, and everyUnitTestResulthasstartTimeandendTime.Part of #1271
On this PR's CI run
Run 38290032941 at e811bdc passed every job.
Model name: AMD EPYC 7763 64-Core ProcessorandCPU(s): 4. The leg took 16m39s.integration-trxartifact (1.2 MB zipped) holds 307 classes. The serialintegrationcollection (128 classes) ran from 17.0 s to 902.5 s, and no other test was running for 509 s of that span. Its largest classes wereOAuthFailClosedTests(161 s),McpAuthorizationTests(86 s),RecordDailyEntryToolTests(75 s),McpToolErrorTests(72 s) andAccountLifecycleCommandTests(60 s).