Repository navigation
[rel/4.5] Improve readability of Azure DevOps skip summaries - #12246
Conversation
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
Scope: the markdown rendering of the Skipped tests by reason section in the AzureDevOpsReport extension. I also checked PACKAGE.md, the acceptance test and the unit tests. I found no blocking issues. No public API, IPC, resource or localization files are touched.
Checked
- The single-module path (
modules.Count == 1) prints the shared context once. The multi-module path prints per-entry context in an indented paragraph. Blank-line handling is consistent in both: the single-module list is tight, and the multi-module list is loose with each entry indented two spaces. - The omitted-tests accounting after the list is unchanged.
- The unit tests cover both layouts, including the cap/omission case and the tie-break ordering by module, framework and attempt. The new
BuildAggregateMarkdown_SingleModuleSkipReasonsShareContexttest uses a raw string literal for the exact multiline comparison, as the repo guidelines ask. - The acceptance test adds a reason containing
<,&,"and|, and a long reason, so the escaping and wrapping behavior is exercised end to end.
Non-blocking observations
- The reason and module identity are still escaped with the table-cell helper
EscapeCell. That helper escapes|as\|and turns newlines into<br>, which only matters inside a table cell. The output is harmless in a list item (\|renders as|), and the tests pin it. A follow-up could rename the helper or split it into a general inline-text escape, so the name no longer misleads. - The skip reasons are user-controlled.
EscapeCelldoes not escape*,_or`, so a reason with an unbalanced**could change the formatting of the rest of the bullet. The table had the same exposure, so this is not a regression. - The
PACKAGE.mdupdate accurately describes the new layout.
I did not build or run the tests; this review is from the diff only.
|
Preliminary test-triage analysis — Azure Pipelines build 1667703 Early analysis of the failed leg microsoft.testfx (Build Windows Debug). Other build legs are still running and may change this conclusion.
|
Pull request
Summary
Backport #12236 to
rel/4.5. Replace the wide Azure DevOps skip-reason table with wrapping list entries and bold counts. Single-module summaries show shared module/platform context once; aggregates keep that context with each reason.Preserve sorting, the ten-group cap, omitted-test accounting, legacy unavailable metadata, result identities, publishing and exit semantics. This is a four-file backport: formatter, package documentation, exact-output unit tests and packed acceptance coverage. Keep the release's
net8.0consumer; do not import main's framework, dependency or API changes.Design and decision record
The existing presentation-fix rationale and alternatives are in #12236; no new feature/design record is needed. The production formatter is byte-identical to the merged source (Git blob
f3b508d625f6fb5f3eebe794c6e89bcac33a4fcb). The only cherry-pick conflict was the packed expectation's framework label, retained asnet8.0.e8db9f8d499b4d675fe176187ee483f8ffdd2b9d.32077b2456a3c5f25ca2e96a5add96e0c4d03f74.770eba45575f97930dcfbb92ec8ab07f54c1e1ab.CB4F6B624AFD0200007D272DC502679C94258BE674CFBEF7BAA4A93FB90472DC.Demonstration
Independently rendered the actual packed 2.5.3-dev reporter's emitted Markdown in a local browser with neutral typography at 1200px and 480px iframe viewports. Visual inspection and DOM checks confirmed readable wrapping, no horizontal overflow, correct count/reason associations, shared context, literal escaped markup/quotes and two focusable issue links. These are local release-product checks, not the live Azure DevOps service/theme. The source PR contains the original illustrative captures; the excerpt below is from the release package output:
The full output contains six results: one passed, one failed and four skipped, grouped as
2,1,1. Both passing outer acceptance cases assert the intentional child's exit2and the complete unabridged Markdown. No external publishing was performed. Aggregate/legacy/empty-output contracts are protected by unit assertions; their hosted visual presentation was not exercised.Validation and regression proof
Windows x64, Release, repository SDK
12.0.100-alpha.1.26480.103. Unit host:net8.0on runtime8.0.33. Acceptance host: repository-defaultnet11.0, using its SDK-selected12.0.0-alpha.1.26480.103runtime; actual packaged consumer:net8.0. Both assemblies retained MethodLevel scheduling with 32 workers. No scheduling, SDK, package versions or source configuration were changed.TRX, JUnit and CTRF identities/counts/outcomes reconcile; JUnit's existing DataRow method/attempt aliases were mapped to their source-declared cases. The green execution verifier also checked freshness, exact planned names, unique execution IDs and counters.
Requirement coverage
BuildAggregateMarkdown_SingleModuleSkipReasonsShareContext,BuildMarkdown_RendersTimedSkippedOutcome,SessionFinishing_GroupsSkipExplanationsWithoutListingSkippedNamesAsync.BuildAggregateMarkdown_SkipReasonTiesUseModuleFrameworkAndAttemptIdentity,BuildAggregateMarkdown_SkipReasonsEscapeCellsAndPreserveLegacyMetadata,BuildAggregateMarkdown_SkipReasonsCapsRowsAndAccountsForObservedOmissions.PackedStandalone_ReportsOutcomesSkipReasonsAndSummedTestTime,FixtureSdkWithoutRequiredArtifactSupport_PreservesPerModuleFallback; both assert the final complete summary and upload command. The latter is explicitly a protocol fixture, not a native SDK invocation.Commands below preserve the executed flags;
$evidencedenotes the task-owned evidence directory rather than a machine-specific absolute prefix. Native Windows PowerShell was launched with process-only PATH entries for.dotnet, Windows PowerShell and System32; no user/global settings changed.Package provenance: the generated consumer's
project.assets.jsonSHA-512 entries match the fresh archives, and both executedlib/net8.0DLL SHA-256 values match those archive entries. The fixture used its unique per-run package cache, not a reused global cache. Archive SHA-256:4476A92F43B978D748337212372581090EF5C2830E3C8B4C38BF984C513B6196.C5FF60798F1C14594FEF0F4B89C90D11695680476B2236699EF47DFB0DDE96E0.Test-value evidence
Executed: detected. On tested head
770eba45575f97930dcfbb92ec8ab07f54c1e1ab, retained all final tests while replacing only the formatter with the byte-exact release-baseline blob8e4e2f5e0ec2e87f9e251759b5045ccc570b5b5a. Counterfactual patch SHA-256:BCFBA2C2CA5764B782669E81D6BA219162506A7B70C2AE58179EB709B8BC84BE.The same 36-case selection failed eight exact-output assertions with the old four-column table, including
BuildAggregateMarkdown_SingleModuleSkipReasonsShareContext, then passed all 36 with the correct implementation. Unit and acceptance test source hashes remained unchanged between states. Failures were executed string-equality assertions, not compilation, restore, discovery, launch, timeout or skipped-test signals. Red intermediates/binaries were isolated underartifacts\obj\skip-redandartifacts\bin\skip-red.This paired proof is at the formatter unit boundary; packed acceptance is additional green consumer proof, not a claimed red/green package comparison. Reports, binlogs, patches, package hashes, consumer assets/config and rendered output are retained in the task's session artifacts; they were not uploaded.
Not claimed: remote CI, Linux/macOS execution, a full-suite run, or the live Azure DevOps hosted page/theme. Static pipeline inspection confirms the normal Windows Release solution test step includes both assemblies when
SkipTests=false; CI execution for this published head is not yet proven.