Repository navigation
[API Proposal]: Type Mapping API #113362
Description
Background and motivation
The TypeMap is a solution for large interop based frameworks (for example, Java, Swift, WinRT, etc). The issue is attempting to map a type from another platform to the .NET platform and vice versa. The classic COM scenario uses the Guid type as the common key. Note that all .NET types have an associated GUID and therefore one can easily map from Type to Guid with minimal effort. There are two interop scenarios to consider the reverse P/Invoke (unmanaged to managed) or P/Invoke (managed to unmanaged).
For the reverse P/Invoke some type from another environment is entering the managed environment, what type do we map this instance to?
Examples: Guid -> Type (COM) or string -> Type (WinRT)
The P/Invoke means a Type is leaving managed code and being projected into an unmanaged environment, what/how do we project?
Examples: Type -> CCW (built-in COM) or Type -> Type (Where the second Type is the ABI projection, for example WinRT).
Both scenarios above can be relatively simple to solve in a JIT-supported .NET environment. However, for AOT scenarios, where size is a consideration, the above mapping must be trimmable and the simple type mapping solutions using tools like Type.GetType() are not viable.
This API is designed to both support the Type Mapping needed for interop as well as be trimmable for AOT scenarios.
See #110691 for complete details.
Comment describing the workflow this supports - #110691 (comment)
API Proposal
namespace System.Runtime.InteropServices;
/// <summary>
/// Type mapping between a string and a type.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public sealed class TypeMapAttribute<TTypeUniverse> : Attribute
{
/// <summary>
/// Create a mapping between a value and a <see cref="System.Type"/>.
/// </summary>
/// <param name="value">String representation of key</param>
/// <param name="target">Type value</param>
/// <remarks>
/// This mapping is unconditionally inserted into the type map.
/// </remarks>
public TypeMapAttribute(string value, Type target)
{ }
/// <summary>
/// Create a mapping between a value and a <see cref="System.Type"/>.
/// </summary>
/// <param name="value">String representation of key</param>
/// <param name="target">Type value</param>
/// <param name="trimTarget">Type used by Trimmer to determine type map inclusion.</param>
/// <remarks>
/// This mapping is only included in the type map if the Trimmer observes a type check
/// using the <see cref="System.Type"/> represented by <paramref name="trimTarget"/>.
/// </remarks>
[RequiresUnreferencedCode("Interop types may be removed by trimming")]
public TypeMapAttribute(string value, Type target, Type trimTarget)
{ }
}
/// <summary>
/// Declare an assembly that should be inspected during type map building.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = true)]
public sealed class TypeMapAssemblyTargetAttribute<TTypeUniverse> : Attribute
{
/// <summary>
/// Provide the assembly to look for type mapping attributes.
/// </summary>
/// <param name="assemblyName">Assembly to reference</param>
public TypeMapAssemblyTargetAttribute(string assemblyName)
{ }
}
/// <summary>
/// Create a type association between a type and its proxy.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
[AttributeUsage(AttributeTargets.Assembly, AllowMultiple = false)]
public sealed class TypeMapAssociationAttribute<TTypeUniverse> : Attribute
{
/// <summary>
/// Create an association between two types in the type map.
/// </summary>
/// <param name="source">Target type.</param>
/// <param name="proxy">Type to associated with <paramref name="source"/>.</param>
/// <remarks>
/// This mapping will only exist in the type map if the Trimmer observes
/// an allocation using the <see cref="System.Type"/> represented by <paramref name="source"/>.
/// </remarks>
public TypeMapAssociationAttribute(Type source, Type proxy)
{ }
}
/// <summary>
/// Entry type for interop type mapping logic.
/// </summary>
public static class TypeMapping
{
/// <summary>
/// Returns the External type type map generated for the current application.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
/// <returns>Requested type map</returns>
/// <remarks>
/// Call sites are treated as an intrinsic by the Trimmer and implemented inline.
/// </remarks>
[RequiresUnreferencedCode("Interop types may be removed by trimming")]
public static IReadOnlyDictionary<string, Type> GetOrCreateExternalTypeMapping<TTypeUniverse>();
/// <summary>
/// Returns the associated type type map generated for the current application.
/// </summary>
/// <typeparam name="TTypeUniverse">Type map universe</typeparam>
/// <returns>Requested type map</returns>
/// <remarks>
/// Call sites are treated as an intrinsic by the Trimmer and implemented inline.
/// </remarks>
[RequiresUnreferencedCode("Interop types may be removed by trimming")]
public static IReadOnlyDictionary<Type, Type> GetOrCreateProxyTypeMapping<TTypeUniverse>();
}API Usage
Provided by .NET for Android runtime
// Type used to express specific type map universe.
public sealed class JavaTypeMapUniverse { }
// Java capability to enable type instance creation.
public interface IJavaCreateInstance
{
object Create(ref JniObjectReference reference, JniObjectReferenceOptions options);
}
// Java capability implementation for object creation.
public class JavaObjectTypeMapAttribute<[DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] T> : Attribute, IJavaCreateInstance
where T : JavaObject
{
public object Create(ref JniObjectReference reference, JniObjectReferenceOptions options)
{
// Create instance using T.
}
}
// Java capability implementation for string creation.
public class JavaStringTypeAttribute : Attribute, IJavaCreateInstance
{
public object Create(ref JniObjectReference reference, JniObjectReferenceOptions options)
{
IntPtr ptr = /* Extract the underlying string pointer from reference. */
string str = Marshal.PtrToStringUni(ptr);
return str;
}
}
// -- Alternative attribute design
// This would expose the direct as a Type instead of as a generic parameter. It would reduce
// type loading costs and could be the single attribute for all "capabilities".
public class JavaObjectTypeMapAttribute : Attribute
{
public JavaObjectTypeMapAttribute([DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] Type type)
{ }
public [DynamicallyAccessedMembers (DynamicallyAccessedMemberTypes.PublicConstructors | DynamicallyAccessedMemberTypes.NonPublicConstructors)] Type TargetType { get; }
}.NET for Android projection library
// Dependent association between typeof(string) => typeof(StringProxy).
[assembly: TypeMapAssociation<JavaTypeMapUniverse>(typeof(string), typeof(StringProxy))]
// Map string name of Java type to typeof(StringProxy), using typeof(string) to indicate
// if the entry should be included or not.
[assembly: TypeMap<JavaTypeMapUniverse>("java/lang/String", typeof(StringProxy), typeof(string))]
[JavaStringType]
public class StringProxy
{ }User application
// Application reference for assembly to be used in type map generation.
[assembly: TypeMapAssemblyTarget<JavaTypeMapUniverse>("Application.SideCar.dll")]
// User defined type
[JavaObjectTypeMap<MyJavaClass>]
public class MyJavaClass : JavaObject
{ }
var inst = new ClassFromNuGet();Generated Application.SideCar.dll
A sidecar assembly could always be generated to handle the following:
- Specify assembly type map targets (for example, .NET for Android runtime)
- Specify type mappings for types defined in the application.
- Generate proxies and mappings for types in Nugets to support older TFMs.
// Application reference for assembly to be used in type map generation.
[assembly: TypeMapAssemblyTarget<JavaTypeMapUniverse>("Android.Runtime.dll")]
[assembly: TypeMap<JavaTypeMapUniverse>("app/MyJavaClass", typeof(MyJavaClass), typeof(MyJavaClass))]
[assembly: TypeMap<JavaTypeMapUniverse>("lib/classFromNuget", typeof(ClassFromNuGetProxy), typeof(ClassFromNuGet))]
[JavaObjectTypeMap<ClassFromNuGet>]
internal class ClassFromNuGetProxy
{ }Interop runtime (for example, .NET for Android) usage example.
// Stored by the interop runtime in some global static
static IReadOnlyDictionary<string, Type> s_JavaTypeMap = TypeMapping.GetOrCreateProxyTypeMapping<JavaTypeUniverse>();
...
string javaTypeName = /* Get Java type name from jniHandle */
s_JavaTypeMap.TryGetValue(javaTypeName, out Type? projection);
Debug.Assert(projection != null); // Assuming map contains type.
var creator = (IJavaCreateInstance)projection.GetCustomAttribute(typeof(JavaObjectTypeMapAttribute<>));
var objectRef = new JniObjectReference(jniHandle, JniObjectReferenceType.Local);
JavaObject mcw = (JavaObject)creator.Create(ref objectRef, JniObjectReferenceOptions.None);Alternative Designs
API design alternatives:
- The APIs on
TypeMappingcould instead be getter methods instead of exposing theIReadOnlyDictionary<,>. - The APIs on
TypeMappingcould avoid theTrysemantics and always return theIReadOnlyDictionary<,>since there is expected to be a dynamic solution that will create theIReadOnlyDictionary<,>at run-time. - Generics could be removed from several APIs to avoid the additional loading of types for each universe combination.
API alternative
Continue with the bespoke solutions that currently exist in CsWinRT, Android (Java), Swift, and future interop platform targets.
Risks
This has a minimal risk to the current ecosystem as it is new.
The larger risk is creating something that doesn't address all issues for current or future interop platform targets. At present, there is work being done CsWinRT to confirm the mechanics. The Android (Java) target is actively being explored with no obvious concerns. The Trimmer team has been consulted extensively about this API and signed off from the trimmable perspective.
Related to missing a functional need, there is a potential to be less performant than the current bespoke solutions being employed. This performance gap could impact adoption.
Finally, since the trimmable semantics are very strict in this model, it is possible incorrectness in existing TypeMap implementations may be uncovered resulting in new issues for adopting interop frameworks.
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsNo status
Activity
dotnet-policy-service commentedon Mar 11, 2025
Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.
AaronRobinsonMSFT commentedon Mar 11, 2025
@agocke @sbomer @rolfbjarne @jonpryor @jkoritzinsky @MichalStrehovsky @Sergio0694 @jkotas @simonrozsival @ivanpovazan
I'd like to get this API approved ASAP. Please prioritize questions/comments/concerns.
Sergio0694 commentedon Mar 11, 2025
From the CsWinRT side, we've been busy prototyping CsWinRT 3.0 on top of these APIs (see fhl/cswinrt-3.0), and at least from a usability perspective, we're actually quite happy with how well the design integrates with things on our end. Obligatory: we couldn't actually test this (for obvious reasons), but the API shape seems to cover all scenarios we've looked at so far 😄
A couple notes while at it:
GetExternalTypeMappingreturning the type map directly, overTryGetExternalTypeMapping, if we decide to just always let the trimmer build a map or throw an exception if it's missing. Mostly because it makes it a little bit more convenient to use it in static field initializers (since my understanding is, you should cache the type map returned from those methods?). Otherwise yeah we can also live with theTryGet___pattern if there's a strong preference.return (T)foo). I understand that the linker wouldn't "see" those casts with respect to the external type table, which is great, exactly what we want (we should document this too). But are there other considerations with respect to casts in generics? Would they cause the trimmer to keep more stuff in either type map, and if so, in what scenarios exactly?Really looking forward to this API, thank you again @AaronRobinsonMSFT for driving this!
MichalStrehovsky commentedon Mar 11, 2025
I was thinking about that one too. Unless there's a special need for the API to return a dictionary, it might be better to do:
Returning an
IReadOnlyDictionarymakes it possible for someone to write code that iterates it or counts things in it and unless that's needed for something, it's just a way to write code that may not behave the same before/after trimming.I don't know if this is something we could contractually guarantee. We'd consider things casted to something whenever it's possible in some way to get there. For example #110691 (comment), or
IsAssignableFromAPIs, etc.It's my understanding that this is for casts and cast-like behaviors in user code, not just within the interop layer.
Casts in generic code are still casts. If we were to treat it as anything else, it would make things difficult to reason about. Trimming will need to be more conservative to account for generics. It already does that to keep casting in general working. I think the general rule we'll need to end up with is "if you can obtain a System.Type of the type at runtime, it should also be considered used in a cast", modulo maybe things like System.Type used in a
==/!=comparison. Whenever we optimize away a "cast", it's a potential after-trimming behavioral difference, so we'd want as few of them as possible, ideally none.The proposal LGTM otherwise!
rolfbjarne commentedon Mar 11, 2025
I think I see a potential problem, where the assembly attributes can't see nested private types for any typeof expressions:
and this won't work either:
I don't think this would be a problem in IL though, which is probably what we're going to do anyways (emit IL instead of C# code).
Sergio0694 commentedon Mar 11, 2025
If we want to do that (I'm not opposed), at that point can we just use a span too for the external type mapping? 😄
I understand the lookup might still be expensive, but this would still feel like a small free perf win to me. It would've been different if we had wanted to return a dictionary, in which case, sure. But if we want to expose lookup methods directly, why not?
Mmmhhh... I'm kinda starting to worry that putting everything together this might actually end up basically just rooting pretty much almost everything. I suppose the only way to actually find out is to actually try things out for real with a CsWinRT prototype and see what happens, and possibly go from there and see what could be improved. Hopefully it's not actually that bad 😅
You'll likely need to have an analyzer in your interop layer emitting an error for such cases, and telling users that they need their types needing marshalling to be at least accessible to the assembly. Eg. I do the same in ComputeSharp for a similar reason.
agocke commentedon Mar 11, 2025
I shared some feedback offline, I'll copy it here as well.
I have the same concern as @MichalStrehovsky above -- now that we've moved towards generating a universal implementation using reflection, I don't think producing an IReadOnlyDictionary is helpful, and it may be harmful because it exposes more of the implementation than we want.
I also think the combination of generics and reflection is a little awkward. We should consider passing
ITypeMapUniverseas either a System.Type or even just a string. A typeof in an attribute is just a string anyway.jkotas commentedon Mar 11, 2025
My understanding that generation of universal implementation using reflection is not part of the current proposal.
TryGet...APIs returntrueand the type map IDictionary.TryGet...APIs returnfalseand the specific interop runtime has to use some fallback solution. This fallback is not provided by the runtime.@AaronRobinsonMSFT Is this correct?
jkotas commentedon Mar 11, 2025
We do not have to implement all methods on
IReadOnlyDictionary. It is fine to implementIReadOnlyDictionarymethods that we do not like asthrow NotSupportedException.AaronRobinsonMSFT commentedon Mar 11, 2025
@rolfbjarne Yes, that would be an issue. Is this a common pattern? I think it would be an acceptible limitation and as you mention, emitting IL directly is at least a workaround.
@agocke @MichalStrehovsky @Sergio0694 This has evolved for me and I now prefer returning the
IReadOnlyDictionary<,>. I think it is the common shape we want and even though there are APIs that could be ill-advised now, we can throw as suggested in #113362 (comment).I don't see that combination here. The
ITypeMapUniverseis a generic parameter that tells the system whichIReadOnlyDictionary<,>to return. We can of course create an arbitrary key mechanism via astringbut that seems a bit fragile. I can get behind moving it from a generic parameter to aITypeMapUniverseinstance argument if that is preferred though. My desire here is for arbitary interop frameworks to own how they describe their universe. The only way for someone to look-up the type map for an Android scenario, would be through a key the Android team defines.57 remaining items
Sergio0694 commentedon Mar 30, 2025
@josephmoresena out of curiosity, would you be able to link to the logic you use (or some examples) to handle marshalling objects to the right types when no static type information is available? That is, say you have a native API that returns just
object, where that object (at runtime) is actually some specific native object of some type (say,Foo.Bar). When marshalling to managed, how do you lookup the type specific marshalling logic forFoo.Barso you can construct an instance of that type? And also how do you do this in a trimmable way? 🤔josephmoresena commentedon Mar 31, 2025
JNetInterface doesn’t rely on traditional marshalling. Instead, my implementation retrieves the class of the Java object at runtime and determines the most compatible managed type available.
I’ve tried to explain the core ideas in the following documentation:
To resolve metadata, I use a static dictionary populated at runtime. The key is the type’s hash, which, in this case, is a
System.Stringinstance used as buffer for the UTF-8 characters of the type’s JNI name.Each type’s interface defines three static methods that are required to be implemented. These are used by the generic metadata logic to create instances dynamically:
This approach is fully trimmable compatible, as it only depends on the code paths actually used. Even when reflection is not disabled (i.e., the feature switch to avoid it is not enabled), metadata for jagged arrays like
T[]...[]can still be generated at runtime without issues. I wasn’t able to avoid reflection entirely due to the (N+1) problem when dealing with these array types.A consumer-facing example of the mapping can be found here:
That said, it's worth mentioning that the built-in types in JNetInterface use a slightly different initialization strategy:
This alternative method, based on precomputed constants, is likely to yield better performance during initialization.
Sergio0694 commentedon Apr 11, 2025
Closing the loop on this, I skipped the links you shared (thank you!). I think this is the main point:
Now I understand where the disconnect is (probably). This interop mapping system seems to be:
This (especially the first point) are fundamentally incompatible and different with what's proposed here. The point of the type map APIs is to allow projection systems (eg. WinRT, Java, etc.) to declare managed versions of native types, in such a way that:
The external type map gives you (1), (2) for projected types, and (3).
The proxy type map gives you (2) for user defined types, and (3).
Not saying either approach is necessarily better, just that they seem to have a very different design (and ergonomics). I don't think you can achieve all 4 points mentioned above with a fully statically typed system, because you fundamentally can't know all static type information at all time (ie. you can have a native API that takes or returns
object, and you might want to downcast it).josephmoresena commentedon Apr 15, 2025
Yes, I believe that in my case, item 1 would be the most important for me. JNI calls are not fixed, as they depend on the specific invocation made to or from Java.
Marshalling could be a plus—it might improve the experience when declaring native Java methods, but I think using an adapter is acceptable in that context.
Item 3 in my approach holds true as long as the different execution flows are not used. While it's true that once a type is registered it remains available, in practice, only the functionality that was actually used is preserved.
However, I wanted to find a way to optimize the registration process and, if possible, avoid doing it manually. Although duplicate registration is not allowed—and does not throw any error when attempted—it is mainly intended for use by a module initializer, which only makes sense (or makes much more sense) in executable assemblies. This leaves library assemblies somewhat unsupported—at least formally—although it can be addressed through custom strategies defined by each user.
Additionally, there’s the matter of the TypeMetadataBuilder. While I consider it a great fluent alternative at design time, at runtime it's problematic—it should ideally be a compile-time construct. That’s precisely why I’m interested in this strategy, even though marshalling is not one of my goals.
agocke commentedon Apr 15, 2025
I think @josephmoresena's approach has a lot of benefits. The main one is: although the static methods are structured as separate to execution, they don't have to be. For example, if
JVirtualMachine.Register<T>()were called next to the place that the target type were used, then that call would be subject to trimming in the same way as the code using it.What would be ideal is some system where the registration returns some sort of provider, which is actually needed to produce an object of that type. That would mean that casts would only succeed when they go through registration, and there would be a statically typed accessor that enforces that everything goes through registration.
However, this doesn't solve the same problem that the type map solves, which primarily is: back compat. The type map allows things like
isandaschecks to succeed without special handling. However, it's also unsafe. Things likeobject.GetType()can return different results based on trimming. The system @josephmoresena is using is safer and more predictable.Sergio0694 commentedon Apr 15, 2025
Can you elaborate why the type map is inherently more unsafe? I'm not sure I get the difference between:
as,is,(T), etc.) are recognized by linker, which roots the marshalling code for itCast<T>()method, which roots and registers theTfrom thereLike, in both cases all those casts would always work. I understand that it's not "safe" in the sense that you might get slightly different behavior when trimming, but I guess I'm wondering, isn't this at least "safer" than normal trim-unsafe code, in the sense that once you do enable trimming, the behavior is fully deterministic, since the linker will always root the same external types in the map based on the casts you actually do, and not just on "whatever code happens to accidentally be rooted" in the app? 🤔
agocke commentedon Apr 15, 2025
You’re assuming that the system preserves object identity when using the strong cast and that the
ischeck works too. I wasn’t proposing that.Sergio0694 commentedon Apr 15, 2025
Ah, I see, I'm following now. Yeah for us (CsWinRT) that's an important aspect of this implementation, as code will just rely on this all over the place. I can see how an approach where you just have to do
Cast<T>()and that might return a new RCW instance every time would be much simpler to statically implement (though eg. simply way too impractical in our case).MichalStrehovsky commentedon Apr 15, 2025
We were explicitly discussing that casts would not always work in places like generic code (casting to T when T is a generic parameter).
Cast<T>would have the same problem when it comes to generics, but the main difference is that the underlying interop stack could warn whenever it sawCast<T>being used with a generic T. It cannot warn for(T)/is T/as Tbecause that would simply not be practical (most of the time this cast is not an "interop operation").So an explicit
Cast<T>that returns a new RCW every time the T changes allows implementing this in a trimming safe manner. The heuristic that makesissometimes work will always have holes (which is why the proposal marks these APIs as unsafe - the underlying interop stack will need to suppress the warning, but my position is that suppressing the warning will always be a bug since the underlying interop stack cannot guarantee these semantics, but at least we can wash our hands off it on the trimming side).Sergio0694 commentedon Apr 16, 2025
To clarify, I meant with the revised trimming policy you mentioned, where the external type map would also consider all generic type arguments as potentially used in a cast, and preserve them as entries in the table. So those generic casts would also still work.
Sergio0694 commentedon Apr 29, 2025
Small question: how does one suppress warnings when using the assembly attributes here? 🤔
Like, I can just use
[UnconditionalSuppressMessage]when using theTypeMappingAPIs, that part is fine. But I'm not sure how to handle the warnings from the actual attributes. You can use the same attribute on the assembly, but that will suppress all trim warnings anywhere else too, which can hide actual errors and doesn't seem great. You can't just use#pragma warning disablehere in source, because that won't actually stop ILLink/ILC from emitting those warnings when you actually publish....Could we perhaps drop
[RequiresUnreferencedCode]on the attributes? AllTypeMappingAPIs are annotated anyway, so you'd still get warnings when using them. Would that not be sufficient? Applying the attribute itself doesn't seem trim-unsafe.MichalStrehovsky commentedon Apr 29, 2025
I don't think you can. This will require special casing in trimming. Discussion here: #110691 (comment)
Sergio0694 commentedon Apr 29, 2025
Ah, I see, I had missed that message. I'll just ignore them for now then, thank you! 🙂