Add ability to output a shim .exe + .dll for an application (for WinRT activation) #103451
Description
Overview
We're working on enabling more scenarios on modern .NET for WinRT applications (WinUI 3, UWP). One such scenario involves applications that reference one or more WinRT components, and need to both be executed (as an app), as well as act as the implementation binary for those (merged) WinRT components.
To make this clearer, consider this scenario:
- You have a WinUI 3 (or UWP) application
- You reference
MyComponent.csproj, which is a WinRT component. Common scenarios:- Out-of-proc background tasks, ie. exporting
IBackgroundTasktypes - Custom video effects, ie. exporting
IBasicVideoEffecttypes
- Out-of-proc background tasks, ie. exporting
When publishing your app, you need two different binaries being produced:
- A .dll with all the code of your published project (and referenced projects, including those WinRT components)
- A stub .exe, which simply invokes a private
Mainfrom that .dll
This is because there's multiple activation paths for your app:
- When launched (eg. by the user, or by the OS), that .exe will be run
- When used as a WinRT component, that .dll will be loaded directly
To make this all work, we need two features:
- Merging and exporting all WinRT activation factories by all referenced WinRT component projects
- We're adding support for this in WinRT here
- A way to publish as this ".dll + stub .exe" combination
The latter is described in more detail in this .NET Native blog post:
We basically need to match this for modern .NET as well.
Proposal
Talking with @MichalStrehovsky about this, he suggested we might add some UseShimExe property, which would make the build produce these two components. Ideally, this would also make things work as expected when using CoreCLR, and not just NativeAOT.
Internal tracking item (MSFT only): microsoft/OS/48896901.
Alternatives
We could do this entirely from the CsWinRT side (adding some source generator to produce a new [UnmanagedCallersOnly] method to invoke the user-defined entry point, and then bundle some native host to act as stub .exe, which we could copy to the output folder). The main issues with this approach though are that we'd need to ask users to mark their applications as using a library output type, which is very confusing and counter-intuitive, and that it's not entirely clear how we'd also make this work for CoreCLR.
Activity
dotnet-policy-service commentedon Jun 13, 2024
Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.
AaronRobinsonMSFT commentedon Jun 13, 2024
/cc @elinor-fung
AaronRobinsonMSFT commentedon Jun 13, 2024
This doesn't seem like it has much utility beyond a WinRT activation scenario and appears to be a typical AppHost scenario. What is the difference here?
Sergio0694 commentedon Jun 13, 2024
So, there would basically be two different behaviors here depending on CoreCLR vs NativeAOT:
[UnmanagedCallersOnly]methods declared in the application project being compiled (just that one .csproj, as usual).Mainentry point being exported by the .dll).Sergio0694 commentedon Jun 14, 2024
At least in theory, this can also be used whenever someone has some shared logic used by their app that they also want to expose in some other way (eg. library native with whatever ABI, be it WinRT, COM or whatever). Using this approach they can compile everything into a single .dll and not bloat their binary size with basically two copies of all the shared code (plus two copies of the whole runtime/GC, at least in the NativeAOT case).
jkotas commentedon Jun 14, 2024
We had similar problem with iOS apps and NativeAOT. iOS apps have
Mainmethod, but they want to be published as special native libraries because of how the app model is put together. We have introducedCustomNativeMainproperty to make this mode possible in the .NET SDK and left all the heavy lifting (compiling the actual native libraries, etc.) for Xamarin SDK. Can we do the same here - generalize the existing solution used for iOS apps?Sergio0694 commentedon Jun 14, 2024
I remember Jeremy also mentioning that feature and suggesting we might be able to perhaps reuse it here too. Just so I understand, what part of the work would you imagine this generalized version of that feature would handle? Eg. would the shim .exe also be produced by the runtime/SDK, or would that have to be bundled by something else (eg. CsWinRT)? Another question I have is, how do you imagine the CoreCLR scenario would work? It seems to me that that would essentially have to be some combination of a shim .exe + basically DNNE (host + exports forwarder) + managed .dll(s)..?
Just trying to understand how the various pieces would be structured and fit together here 😅
jkotas commentedon Jun 14, 2024
I expect that the .NET SDK would just do the minimum to get out of the way. It would be up to the app model specific SDK (e.g. CsWinRT) to produce the custom host and shim. It is how the current solution for iOS works.
Sergio0694 commentedon Jun 14, 2024
So like, on NativeAOT, the main difference between a fully undocked approach and this would be:
OutputType=ExeIs the above correct, on NativeAOT? And how would things look like with CoreCLR? Would the output still be a .dll (though managed), and that export be..?
MichalStrehovsky commentedon Jun 14, 2024
If we do this the same as iOS/Xamarin, it would mean that
OutputTypecan stayExeand CsWinRT would need to take the responsibility of what to do with the object file generated by ilc (native AOT targets would not run the linker to generate the EXE). CsWinRT would have to generate whatever other glue is needed (something that initializes the runtime) and invoke link.exe to create the final executable (which would be a DLL, even though OutputType is EXE). Then it would also have to create the shim EXE and ensure it all gets published.We don't currently have the same separation of linker arguments on Windows that we have on iDevices, so we'd need to introduce the same protocol (discussed around #88294 (comment)).
Sergio0694 commentedon Jun 14, 2024
Mmmh that kinda sounds like more work on CsWinRT itself alone that just doing everything with the fully decoupled approach we also discussed (source generated export + shim .exe both provided by CsWinRT). It's not entirely clear to me whether this additional work (plus what the runtime would need to do to enable this) is worth it just to allow users to not have to remove that single
OutputTypeline in the .csproj 🤔MichalStrehovsky commentedon Jun 14, 2024
You can certainly require OutputType DLL and source generate the secret UnmanagedCallersOnly entrypoint that the shim EXE will call into. That would probably work for native AOT. I don't know how that would look for JIT based deployments, also with respect to VS integration (would need some prototyping).
But if OutputType is Library, someone will need to figure out what is Main because the C# compiler will no longer do it. And top level statements won't work. So it's a bit of a UX regression, but could be okay.
jkotas commentedon Jun 14, 2024
And also generate code to call Assembly.SetEntryAssembly so that the libraries that call
Assembly.GetEntryAssemblycontinue to work.Sergio0694 commentedon Jun 14, 2024
Right, yeah we were thinking one option could be to try to match this ourselves in some way (eg. assemby name namespace +
Program+Main), or perhaps introduce some attribute (eg.[EntryPoint]or whatever) and tell users to add it to theirMain). Yeah, agreed that it would be a bit clunky, for sure. To clarify, this was just us trying to put together a plan in case we were going to do everything fully decoupled and entirely from CsWinRT, with no runtime changes.Ooh, right 👀
I suppose I was just worried about the cost/complexity of doing all this from CsWinRT, and just thinking about options, is all. It is true though that as I mentioned, support for this is needed by WinUI 3 as well, so it might be worth it. cc. @manodasanW
Could you elaborate on how you'd imagine things to work with CoreCLR? Because to me it seems like the NativeAOT scenario here is more straightforward (either with this custom main, or with a decoupled approach), whereas it's not entirely clear to me how things would be structured with CoreCLR (given that we'd presumably also need 2 native binaries there as well, the .exe and the .dll, so that native consumers can use either just like on NativeAOT, and then a managed .dll with the app code..?). And like, presumably we'd also need to use something like DNNE as well here so that the native exports (
DllGetActivationFactory) from the application project is also exported correctly from the native host .dll or something? 🤔jkotas commentedon Jun 14, 2024
You can either ship template shim .exe and host .dll, and patch the names in them during the build with the actual app name. It would mirror what .NET SDK does for regular console apps today. This works only if the set of the exports is fixed.
Or you can create the shim and host on the fly by compiling C/C++ sources. This is more flexible, but it has a lot more moving parts so it will break more often.
Sergio0694 commentedon Jun 14, 2024
Right, yeah in our case we're thinking we could just bundle the shim .exe in CsWinRT (like we do our native host) 🤔
@MichalStrehovsky could you clarify why couldn't the runtime/SDK take care of building the native .dll, in the NativeAOT scenario? I mean, wouldn't it be possible for it to do that just like it does when building a shared lib (perhaps behind some flag?). Having to manually invoke the linker and do all that from CsWinRT seems potentially error prone (and complicated), but I would imagine the runtime already has all the necessary setup to do this like it does in other scenarios?
jkotas commentedon Jun 15, 2024
It should not be that difficult to tweak regular publish of
<CustomNativeMain>true</CustomNativeMain>to produce a native .dll. It should only require tweaking a few conditions for linker arguments in https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Windows.targets. @Sergio0694 Would you be interested in prototyping it?iOS SDK does not use our linker target. They use their own since they want to customize it heavily. So the current
CustomNativeMainonly goes as far as producing .obj/.o file file with the right shape. The linker step was not updated for it.Sergio0694 commentedon Jun 15, 2024
Yeah I can give it a try! And I can go
botherping Michal to get some more guidance on that 😄I have a few more questions on what the general thing would look like:
CustomNativeMain, possiblyEntryPointSymbolto some well known name (idk like,CsWinRT_StubEntryPointor whatever), and change that linker arg adding/DLLto also be triggered onCustomNativeMainbeingtrue, and not just onNativeLibbeing set. This will make the toolchain produce this NativeAOT .dll with (1) all[UnmanagedCallersOnly]exports from the published project, as usual, and (2) that well known export directly invoking theMainthat the user used (be it top level statements or an explicit one). Then we can bundle our own stub .exe calling that export, and we're done (and just rename it to match the assembly name, and optionally stamp/APPCONTAINERfor UWP).I guess this was just me saying out loud what my understand is on this. Is this first part correct so far?
Anyway I'll try to put up a draft PR for that .targets and we can go from there I assume. Thank you! 🙂