You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Deferred data expansion repeatedly scans all registered tests #7474
Deferred data expansion scans every registered test once for each placeholder in PR #7471. Large suites with many deferred data sources can therefore spend quadratic time preparing dependencies before test execution begins.
Reproduction
Create many test methods using [MethodDataSource(..., DeferEnumeration = true)], each yielding multiple cases, and measure session preparation as the number of methods grows. No dependency attributes are required: ReplaceDeferredTest still visits every registered test for each expansion.
The scan is visible in TestDependencyResolver.cs at dd9ebe89, called once per placeholder by the session coordinator. The current main branch does not yet contain that replacement API; this follow-up applies when #7471 lands. Native prerequisite: #7409, delivered by #7471.
Proposal
Batch placeholder replacements, invalidate affected dependency edges in one pass, and rebuild lookup indexes once before resolving dependencies. Preserve completed zero-child and failed expansion outcomes. Measure preparation time and allocations before and after, and validate dependency chains in source-generated and reflection modes.
Please make deferred dependency preparation scale with the total number of tests and dependency edges rather than repeated full-suite scans.
Acceptance criteria
Batch deferred placeholder replacements and dependency-edge invalidation; rebuild lookup indexes once before resolving dependencies.
Preserve zero-child and failed expansion outcomes and deferred dependency-chain behavior in generated and reflection modes.
Measure preparation time and allocations before and after on suites with increasing numbers of deferred methods and cases; demonstrate that replacement no longer scans the full registered suite once per placeholder.
Confirmed against origin/main 1f705642be3eec0a1a55034d6210f1532ec5946c: the replacement API is still confined to open PR #7471. Added native prerequisite #7409 (delivered by #7471) and explicit acceptance criteria so this follow-up is claimed after the prerequisite lands.
Closing as already implemented by merged PR #7471, commit 801c0b42c2009726a215423cf5b5eb157db6d29d; verified against current origin/main4c6b757bd1.
The coordinator accumulates all expansions and calls replacement once before dependency resolution.
The resolver builds one replacement set, removes placeholders in one pass, clears lookup indexes once for lazy rebuilding, and invalidates dependency edges in one pass. Completed zero-child and failed placeholders retain their outcome and lose stale outgoing edges.
Fresh SDK 10.0.403/net10.0/macOS arm64 validation: DeferredDependencyTests passed 8/8 in generated mode and 8/8 with --reflection; TestDependencyResolverTests passed 2/2 (empty and failed outcomes). Commands: dotnet test --project tests/TUnit.TestProject/TUnit.TestProject.csproj --framework net10.0 --treenode-filter '/*/*/DeferredDependencyTests/*' (repeat with --reflection) and dotnet test --project tests/TUnit.UnitTests/TUnit.UnitTests.csproj --framework net10.0 --treenode-filter '/*/*/TestDependencyResolverTests/*'. No Engine.Tests run locally; Fix dependencies on deferred enumeration tests #7471's engine/AOT checks passed on CI.
PR #7471 already recorded a lightweight before/after benchmark at 1,000 placeholders x 3 children: 11.536 ms / 162,344 bytes before, 0.119 ms / 139,656 bytes after. An additional isolated Release/net10.0 replacement-loop model (old algorithm from dd9ebe89, batch algorithm from merged commit) verifies increasing method and case counts:
Methods
Cases/method
Before ms
Batch ms
Before allocated B
Batch allocated B
Before full-list visits
Batch full-list visits
1,000
1
3.175
0.048
96,040
73,256
1,000,000
2,000
1,000
3
7.048
0.054
144,088
121,304
2,001,000
4,000
1,000
10
16.859
0.081
336,136
313,352
5,504,500
11,000
5,000
1
81.530
0.253
480,040
322,720
25,000,000
10,000
5,000
3
136.092
0.287
720,088
562,768
50,005,000
20,000
5,000
10
310.354
0.551
1,680,136
1,522,816
137,522,500
55,000
These are single-process illustrative measurements of replacement loops with lightweight objects and no dependency edges, excluding suite discovery/execution and index construction; they are not end-to-end preparation benchmarks. Exact times vary with JIT and machine load. The visit counts establish that replacement now scans once per batch rather than once per placeholder; edge invalidation visits each existing edge at most once. No additional source change is needed.
Reproduce the isolated loop measurement
Run this as a net10.0 console project with implicit usings enabled, using dotnet run -c Release:
usingSystem.Diagnostics;// Isolated replacement-loop model of dd9ebe89 and merged 801c0b42, excluding discovery/execution.foreach(varcountinnew[]{100,1000,5000})foreach(varcasesinnew[]{1,3,10})foreach(varbatchinnew[]{false,true}){varexpansions=Enumerable.Range(0,count).Select(_ =>(Placeholder:newTest(),Children:Enumerable.Range(0,cases).Select(_ =>newTest()).ToArray())).ToArray();vartests=expansions.Select(x =>x.Placeholder).ToList();longvisits=0;varallocated=GC.GetAllocatedBytesForCurrentThread();varwatch=Stopwatch.StartNew();if(batch){varreplaced=newHashSet<Test>();foreach(var(placeholder,children)inexpansions)if(children.Length>0)replaced.Add(placeholder);foreach(vartestintests){visits++;if(test.Empty)test.Dependencies=[];}tests.RemoveAll(replaced.Contains);foreach(var(_,children)inexpansions)tests.AddRange(children);foreach(vartestintests){visits++;foreach(vardependencyintest.Dependencies)if(replaced.Contains(dependency)){test.Dependencies=[];break;}}}else{foreach(var(placeholder,children)inexpansions){tests.Remove(placeholder);tests.AddRange(children);foreach(vartestintests){visits++;if(test.Dependencies.Any(d =>d==placeholder))test.Dependencies=[];}}}watch.Stop();allocated=GC.GetAllocatedBytesForCurrentThread()-allocated;Console.WriteLine($"{count},{cases},{(batch?"batch":"per-placeholder")},{watch.Elapsed.TotalMilliseconds:F3},{allocated},{visits}");}classTest{publicboolEmpty;publicTest[]Dependencies=[];}
Activity
thomhurst commentedon Nov 3, 2026
Confirmed against origin/main
1f705642be3eec0a1a55034d6210f1532ec5946c: the replacement API is still confined to open PR #7471. Added native prerequisite #7409 (delivered by #7471) and explicit acceptance criteria so this follow-up is claimed after the prerequisite lands.thomhurst commentedon Nov 5, 2026
Closing as already implemented by merged PR #7471, commit
801c0b42c2009726a215423cf5b5eb157db6d29d; verified against currentorigin/main4c6b757bd1.DeferredDependencyTestspassed 8/8 in generated mode and 8/8 with--reflection;TestDependencyResolverTestspassed 2/2 (empty and failed outcomes). Commands:dotnet test --project tests/TUnit.TestProject/TUnit.TestProject.csproj --framework net10.0 --treenode-filter '/*/*/DeferredDependencyTests/*'(repeat with--reflection) anddotnet test --project tests/TUnit.UnitTests/TUnit.UnitTests.csproj --framework net10.0 --treenode-filter '/*/*/TestDependencyResolverTests/*'. No Engine.Tests run locally; Fix dependencies on deferred enumeration tests #7471's engine/AOT checks passed on CI.PR #7471 already recorded a lightweight before/after benchmark at 1,000 placeholders x 3 children: 11.536 ms / 162,344 bytes before, 0.119 ms / 139,656 bytes after. An additional isolated Release/net10.0 replacement-loop model (old algorithm from
dd9ebe89, batch algorithm from merged commit) verifies increasing method and case counts:These are single-process illustrative measurements of replacement loops with lightweight objects and no dependency edges, excluding suite discovery/execution and index construction; they are not end-to-end preparation benchmarks. Exact times vary with JIT and machine load. The visit counts establish that replacement now scans once per batch rather than once per placeholder; edge invalidation visits each existing edge at most once. No additional source change is needed.
Reproduce the isolated loop measurement
Run this as a net10.0 console project with implicit usings enabled, using
dotnet run -c Release: