Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@
<PackageVersion Include="OpenTelemetry" Version="1.19.1" />
<PackageVersion Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.19.1" />
<PackageVersion Include="System.Memory" Version="$(SystemMemoryVersion)" />
<PackageVersion Include="System.Reflection.Metadata" Version="8.0.0" />
<PackageVersion Include="System.Text.Json" Version="10.0.12" />
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="$(SystemThreadingTasksExtensionsVersion)" />
</ItemGroup>
Expand Down
329 changes: 329 additions & 0 deletions docs/decisions/assembly-fixture-provider-discovery.md

Large diffs are not rendered by default.

6 changes: 5 additions & 1 deletion docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,11 @@ A shared static helper compiled into MTP extensions via file linking (no NuGet s

### AssemblyFixtureProviderAttribute

An MSTest assembly-level attribute (`[assembly: AssemblyFixtureProvider(typeof(T))]`) in `Microsoft.VisualStudio.TestTools.UnitTesting` that enables cross-assembly assembly fixtures. When applied to a library assembly, it causes any `[AssemblyInitialize]` and `[AssemblyCleanup]` methods on the specified `FixtureType` to be discovered and executed once per consuming test assembly that loads the library at runtime. This allows shared test infrastructure (e.g., database setup, container lifecycle) to be co-located in a test-helper library rather than repeated across every test project. Local `[AssemblyInitialize]`/`[AssemblyCleanup]` declarations always take precedence over provider-contributed ones; the attribute may be applied multiple times on the same assembly to expose multiple fixture types; and it can also be applied on the consuming test assembly to opt into fixtures from a third-party library. Introduced in [PR #8677](https://github.com/microsoft/testfx/pull/8677).
An MSTest assembly-level attribute (`[assembly: AssemblyFixtureProvider(typeof(T))]`) in `Microsoft.VisualStudio.TestTools.UnitTesting` that enables cross-assembly assembly fixtures. Introduced in MSTest 4.3.0 by [PR #8677](https://github.com/microsoft/testfx/pull/8677), it exposes `[AssemblyInitialize]` and `[AssemblyCleanup]` methods on `FixtureType` once per consuming test assembly with a direct or transitive compiled reference to the marked library. The consumer need not execute a call into that library; an otherwise-unused project/package reference omitted by the compiler is insufficient. The attribute opts into **supplying** fixtures, not into **discovering** them.

Local hooks take precedence independently, and providers fill only missing slots. Both local hooks bypass provider discovery; one local hook does not. Multiple markers are supported, but multiple eligible methods for the same unfilled slot produce the existing duplicate-fixture diagnostic rather than a pipeline. The consumer can also apply the marker to expose fixtures from an unmodifiable library. Shared providers are unsupported when dynamic code is unavailable; the AOT guard and MSTEST0072 were added in 4.4.

The original discovery implementation, confirmed in 4.3.0 and 4.5.2, eagerly loads transitive unmarked references, changing `AppDomain.GetAssemblies()` and potentially locking DLLs on Windows/.NET Framework. The metadata-first correction on main avoids those loads for resolvable file-backed references, while retaining runtime-resolution fallback for opaque or ambiguous bindings. There is no dedicated discovery opt-out. Empty local hooks are a historical workaround, not required MSTest boilerplate. See [the discovery decision and validation record](decisions/assembly-fixture-provider-discovery.md) for the exact compatibility limits and servicing status; no released fixed version is implied.

### Assert.Scope (soft assertions)

Expand Down
1 change: 1 addition & 0 deletions src/Adapter/MSTest.TestAdapter/MSTest.TestAdapter.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ $(CommonProductDescription)]]></PackageDescription>
that reference is suppressed above, so they are referenced explicitly here. -->
<PackageReference Include="System.Threading.Tasks.Extensions" Condition=" '$(TargetFramework)' == '$(NetFrameworkMinimum)' OR '$(TargetFramework)' == '$(UwpMinimum)' " />
<PackageReference Include="System.Memory" Condition=" '$(TargetFramework)' == '$(NetFrameworkMinimum)' OR '$(TargetFramework)' == '$(UwpMinimum)' " />
<PackageReference Include="System.Reflection.Metadata" Condition=" '$(TargetFramework)' == '$(NetFrameworkMinimum)' OR '$(TargetFramework)' == '$(UwpMinimum)' " />

<!-- The adapter's VSTest-facing code (VSTestAdapter\*, AdapterTestProperties) and the native Microsoft.Testing.Platform
integration (TestingPlatformAdapter\MSTestFilterContext) use the VSTest object model directly. It used to flow
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Reflection.Metadata;
using System.Reflection.PortableExecutable;
using System.Security;
#if NET && !WINDOWS_UWP
using System.Runtime.Loader;
#endif

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution;

internal sealed partial class TypeCache
{
private static bool TryReadProviderMetadata(string path, out bool hasMarker, out AssemblyName[] references)
{
hasMarker = false;
references = [];
try
{
using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
using var peReader = new PEReader(stream, PEStreamOptions.PrefetchMetadata);
MetadataReader reader = peReader.GetMetadataReader();
AssemblyDefinition definition = reader.GetAssemblyDefinition();
foreach (CustomAttributeHandle handle in definition.GetCustomAttributes())
{
EntityHandle constructor = reader.GetCustomAttribute(handle).Constructor;
EntityHandle type = constructor.Kind switch
{
HandleKind.MemberReference => reader.GetMemberReference((MemberReferenceHandle)constructor).Parent,
HandleKind.MethodDefinition => reader.GetMethodDefinition((MethodDefinitionHandle)constructor).GetDeclaringType(),
_ => default,
};
if (type.Kind == HandleKind.TypeReference)
{
TypeReference attribute = reader.GetTypeReference((TypeReferenceHandle)type);
hasMarker = reader.StringComparer.Equals(attribute.Name, nameof(AssemblyFixtureProviderAttribute))
&& reader.StringComparer.Equals(attribute.Namespace, typeof(AssemblyFixtureProviderAttribute).Namespace!);
}
else if (type.Kind == HandleKind.TypeDefinition)
{
TypeDefinition attribute = reader.GetTypeDefinition((TypeDefinitionHandle)type);
hasMarker = reader.StringComparer.Equals(attribute.Name, nameof(AssemblyFixtureProviderAttribute))
&& reader.StringComparer.Equals(attribute.Namespace, typeof(AssemblyFixtureProviderAttribute).Namespace!);
}

if (hasMarker)
{
break;
}
}

references = reader.AssemblyReferences.Select(handle => reader.GetAssemblyReference(handle).GetAssemblyName()).ToArray();
return true;
}
catch (Exception ex) when (ex is IOException or BadImageFormatException or UnauthorizedAccessException or SecurityException)
{
if (PlatformServiceProvider.Instance.AdapterTraceLogger.IsWarningEnabled)
{
PlatformServiceProvider.Instance.AdapterTraceLogger.Warning(
"TypeCache: Could not read assembly metadata from {0}; using runtime resolution for AssemblyFixtureProvider discovery. {1}",
path,
ex);
}

return false;
}
}

private sealed class ProviderDiscoveryMetadataResolver
{
#if NET && !WINDOWS_UWP
private Dictionary<string, string?>? _trustedAssemblyPaths;
#endif
#if NETFRAMEWORK
private bool? _hasCodeBaseOverrides;
#endif

[SuppressMessage("Performance", "CA1822:Mark members as static", Justification = "The resolver caches runtime-specific metadata paths on other target frameworks.")]
public string? Resolve(Assembly referrer, AssemblyName referenceName)
{
try
{
#if NET && !WINDOWS_UWP
AssemblyLoadContext context = AssemblyLoadContext.GetLoadContext(referrer) ?? AssemblyLoadContext.Default;
// A component's deps/adjacent files do not describe Default's binding policy.
// A later-loaded consumer may instead bind through Default.Resolving.
if (context != AssemblyLoadContext.Default || context.Assemblies.Any(assembly =>
string.Equals(assembly.GetName().Name, referenceName.Name, StringComparison.OrdinalIgnoreCase)))
{
return null;
}

_trustedAssemblyPaths ??= ReadTrustedAssemblyPaths();
return _trustedAssemblyPaths.TryGetValue(referenceName.Name!, out string? path)
&& path is not null
&& HasMatchingProviderDiscoveryIdentity(referenceName, path) ? path : null;
#elif NETFRAMEWORK
_ = referrer;
AppDomain domain = AppDomain.CurrentDomain;
AppDomainSetup setup = domain.SetupInformation;
// Fusion can select a loaded image or the GAC before probing the application base.
// Keep those bindings, culture probing, redirects and host overrides authoritative.
if (domain.GetAssemblies().Any(assembly =>
string.Equals(assembly.GetName().Name, referenceName.Name, StringComparison.OrdinalIgnoreCase))
|| referenceName.GetPublicKeyToken() is { Length: > 0 }
|| !string.IsNullOrEmpty(referenceName.CultureName)
|| setup.DisallowApplicationBaseProbing
|| setup.PrivateBinPathProbe is not null
|| setup.GetConfigurationBytes() is { Length: > 0 }
|| !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("DEVPATH"))
|| !string.Equals(domain.ApplyPolicy(referenceName.FullName), referenceName.FullName, StringComparison.Ordinal)
|| (_hasCodeBaseOverrides ??= HasCodeBaseOverrides()))
{
return null;
}

foreach (string extension in new[] { ".dll", ".exe" })
{
string path = Path.Combine(domain.BaseDirectory, referenceName.Name + extension);
if (File.Exists(path))
{
return HasMatchingProviderDiscoveryIdentity(referenceName, path) ? path : null;
}
}
#else
_ = referrer;
_ = referenceName;
#endif
}
catch (Exception ex) when (ex is IOException or BadImageFormatException or UnauthorizedAccessException or InvalidOperationException or ArgumentException or SecurityException or XmlException)
{
if (PlatformServiceProvider.Instance.AdapterTraceLogger.IsWarningEnabled)
{
PlatformServiceProvider.Instance.AdapterTraceLogger.Warning(
"TypeCache: Could not resolve a metadata path for {0}; using runtime resolution for AssemblyFixtureProvider discovery. {1}",
referenceName.FullName,
ex);
}
}

return null;
}

#if NET && !WINDOWS_UWP
private static Dictionary<string, string?> ReadTrustedAssemblyPaths()
{
Dictionary<string, string?> paths = [with(StringComparer.OrdinalIgnoreCase)];
if (AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES") is not string trustedPaths)
{
return paths;
}

foreach (string path in trustedPaths.Split(Path.PathSeparator))
{
if (string.IsNullOrEmpty(path))
{
continue;
}

string name = Path.GetFileNameWithoutExtension(path);
paths[name] = paths.TryGetValue(name, out string? previous)
&& !string.Equals(previous, path, StringComparison.Ordinal) ? null : path;
}

return paths;
}
#endif

#if NETFRAMEWORK || (NET && !WINDOWS_UWP)
private static bool HasMatchingProviderDiscoveryIdentity(AssemblyName referenceName, string path)
=> string.Equals(referenceName.FullName, AssemblyName.GetAssemblyName(path).FullName, StringComparison.Ordinal);
#endif

#if NETFRAMEWORK
private static bool HasCodeBaseOverrides()
{
string configurationFile = AppDomain.CurrentDomain.SetupInformation.ConfigurationFile;
if (!File.Exists(configurationFile))
{
return false;
}

using var reader = XmlReader.Create(configurationFile, new XmlReaderSettings { XmlResolver = null, DtdProcessing = DtdProcessing.Prohibit });
while (reader.Read())
{
if (reader.NodeType == XmlNodeType.Element && reader.LocalName == "codeBase")
{
return true;
}
}

return false;
}
#endif
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -221,70 +221,80 @@ internal static void CollectFixtureMethodsFromProviderType(

private static IEnumerable<Assembly> EnumerateCandidateAssemblies(Assembly currentAssembly)
{
// BFS over the consumer assembly's reference graph. This bounds the work to assemblies that
// are recorded in the metadata reference table (the runtime reference graph), instead of
// scanning the entire AppDomain. Note that a bare project / PackageReference is NOT
// sufficient: the C# compiler omits references that the compiled IL never uses, so a
// provider library only shows up here if its types are actually touched by the consumer
// (or a `[assembly: TypeForwardedTo]` / similar pulls the reference in). The acceptance
// asset for this feature accordingly calls into the provider library to materialize the
// reference.
//
// Dedup uses AssemblyName.FullName (name + version + culture + public-key-token) so multi-version
// / multi-token references with the same simple name are not collapsed.
var visited = new HashSet<string>(StringComparer.Ordinal);
var queue = new Queue<Assembly>();

var queue = new Queue<ProviderDiscoveryNode>();
var metadataResolver = new ProviderDiscoveryMetadataResolver();
visited.Add(currentAssembly.FullName ?? string.Empty);

// The consumer assembly itself is yielded first so users may also place the marker on the
// test project (handy escape hatch when the library author cannot ship the attribute themselves).
queue.Enqueue(currentAssembly);
// A consumer can register fixtures itself. For references, inspect PE metadata first:
// loading an unmarked library just to read its references changes user-visible runtime state.
yield return currentAssembly;
queue.Enqueue(new(currentAssembly, GetProviderDiscoveryReferences(currentAssembly)));

while (queue.Count > 0)
{
Assembly current = queue.Dequeue();
yield return current;

AssemblyName[] references;
try
ProviderDiscoveryNode node = queue.Dequeue();
Assembly referrer = node.Referrer;
foreach (AssemblyName referenceName in node.References)
{
references = current.GetReferencedAssemblies();
}
catch (Exception ex)
{
if (PlatformServiceProvider.Instance.AdapterTraceLogger.IsWarningEnabled)
string? name = referenceName.Name;
if (name is null || IsFrameworkAssemblyName(name) || !visited.Add(referenceName.FullName))
{
PlatformServiceProvider.Instance.AdapterTraceLogger.Warning(
"TypeCache: Exception occurred while enumerating referenced assemblies of {0} for AssemblyFixtureProvider discovery. {1}",
SafeGetAssemblyName(current),
ex);
continue;
}

continue;
}

foreach (AssemblyName referenceName in references)
{
string? name = referenceName.Name;
if (name is null || IsFrameworkAssemblyName(name))
string? path = metadataResolver.Resolve(referrer, referenceName);
if (path is not null && TryReadProviderMetadata(path, out bool hasMarker, out AssemblyName[] metadataReferences) && !hasMarker)
{
// Keep the nearest runtime referrer for eventual provider loading, but walk
// unmarked intermediaries using only their metadata.
queue.Enqueue(new(referrer, metadataReferences));
continue;
}

if (!visited.Add(referenceName.FullName))
if (path is null && PlatformServiceProvider.Instance.AdapterTraceLogger.IsInfoEnabled)
{
continue;
PlatformServiceProvider.Instance.AdapterTraceLogger.Info(
"TypeCache: No metadata path for referenced assembly {0}; using runtime resolution for AssemblyFixtureProvider discovery.",
referenceName.FullName);
}

Assembly? referenced = TryLoadReferencedAssembly(current, referenceName);
Assembly? referenced = TryLoadReferencedAssembly(referrer, referenceName);
if (referenced is null || referenced.IsDynamic)
{
continue;
}

queue.Enqueue(referenced);
yield return referenced;
queue.Enqueue(new(referenced, GetProviderDiscoveryReferences(referenced)));
}
}
}

private readonly struct ProviderDiscoveryNode(Assembly referrer, AssemblyName[] references)
{
public Assembly Referrer { get; } = referrer;

public AssemblyName[] References { get; } = references;
}

private static AssemblyName[] GetProviderDiscoveryReferences(Assembly assembly)
{
try
{
return assembly.GetReferencedAssemblies();
}
catch (Exception ex)
{
if (PlatformServiceProvider.Instance.AdapterTraceLogger.IsWarningEnabled)
{
PlatformServiceProvider.Instance.AdapterTraceLogger.Warning(
"TypeCache: Exception occurred while enumerating referenced assemblies of {0} for AssemblyFixtureProvider discovery. {1}",
SafeGetAssemblyName(assembly),
ex);
}

return [];
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="System.Reflection.Metadata" Condition=" '$(TargetFramework)' == 'netstandard2.0' OR '$(TargetFramework)' == '$(NetFrameworkMinimum)' OR '$(TargetFramework)' == '$(UwpMinimum)' " />
<PackageReference Include="System.Threading.Tasks.Extensions" Condition=" '$(TargetFramework)' == 'netstandard2.0' OR '$(TargetFramework)' == '$(NetFrameworkMinimum)' OR '$(TargetFramework)' == '$(UwpMinimum)' " />
<!-- Span/MemoryMarshal used by the linked XxHash code below, and Span<T>.Fill used by
Execution\TestDependencyGraph.*, live in System.Memory on these TFMs.
Expand Down
Loading
Loading