Skip to content

[API Proposal]: Add PersistedAssemblyBuilder type #97015

Closed

Description

@buyaa-n

Background and motivation

We added a new persisted AssemblyBuilder implementation and related new APIs in .NET 9 preview 1. While discussing adding the SetEntryPoint method questions raised for adding SetEntryPoint(MethodInfo, PEFileKinds) the overload that sets PEFileKinds option.

Further we would need to add more options that used in .NET framework, like PortableExecutableKinds or ImageFileMachine that used to set with Save(String, PortableExecutableKinds, ImageFileMachine) overload, plus would need to add APIs for setting AddResourceFile, DefineResource, DefineUnmanagedResource.

Instead of adding all old .NET framework APIs we prefer to let the customers handle their assembly building process by themselves using the PEHeaderBuilder and System.Reflection.PortableExecutable.PEBuilder implementation ManagedPEBuilder options.

public PEHeaderBuilder(
Machine machine = 0,
int sectionAlignment = 0x2000,
int fileAlignment = 0x200,
ulong imageBase = 0x00400000,
byte majorLinkerVersion = 0x30, // (what is ref.emit using?)
byte minorLinkerVersion = 0,
ushort majorOperatingSystemVersion = 4,
ushort minorOperatingSystemVersion = 0,
ushort majorImageVersion = 0,
ushort minorImageVersion = 0,
ushort majorSubsystemVersion = 4,
ushort minorSubsystemVersion = 0,
Subsystem subsystem = Subsystem.WindowsCui,
DllCharacteristics dllCharacteristics = DllCharacteristics.DynamicBase | DllCharacteristics.NxCompatible | DllCharacteristics.NoSeh | DllCharacteristics.TerminalServerAware,
Characteristics imageCharacteristics = Characteristics.Dll,
ulong sizeOfStackReserve = 0x00100000,
ulong sizeOfStackCommit = 0x1000,
ulong sizeOfHeapReserve = 0x00100000,
ulong sizeOfHeapCommit = 0x1000)
and/or
public ManagedPEBuilder(
PEHeaderBuilder header,
MetadataRootBuilder metadataRootBuilder,
BlobBuilder ilStream,
BlobBuilder? mappedFieldData = null,
BlobBuilder? managedResources = null,
ResourceSectionBuilder? nativeResources = null,
DebugDirectoryBuilder? debugDirectoryBuilder = null,
int strongNameSignatureSize = DefaultStrongNameSignatureSize,
MethodDefinitionHandle entryPoint = default(MethodDefinitionHandle),
CorFlags flags = CorFlags.ILOnly,
Func<IEnumerable<Blob>, BlobContentId>? deterministicIdProvider = null)

These options are used to produce the assembly and allow users to configure it any way they want. These covers all options that existed in .NET framework, (but not exactly same way), plus provide many other options that not available in .NET framework that customers may want to set on the final binary.

In order to achieve this, we would need to provide all metadata information produced with Reflection.Emit APIs (MetadataBuilder and ILStreams) so that user could embed them into the corresponding section of PEBuidler. But because the MetadataBuilder and BlobBuilder types are not accessible from within CoreLib we decided to make the PersistedAssemblyBuilder type public and reintroduce the related new APIs there, remove the new APIs from the base AssemblyBuilder type.

API Proposal

namespace System.Reflection.Emit;

public abstract partial class AssemblyBuilder
{
    // Previously approved new APIs
-   public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);

-   public void Save(Stream stream);
-   public void Save(string assemblyFileName);
-   protected abstract void SaveCore(Stream stream);
}

+public sealed class PersistedAssemblyBuilder : AssemblyBuilder
{
+   public PersistedAssemblyBuilder(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);

+   public void Save(Stream stream);
+   public void Save(string assemblyFileName);
+   public MetadataBuilder GenerateMetadata(out BlobBuilder ilStream, out BlobBuilder mappedFieldData);
}

API Usage

PersistedAssemblyBuilder ab = new PersistedAssemblyBuilder(new AssemblyName("MyAssembly"), typeof(object).Assembly);
TypeBuilder tb = ab.DefineDynamicModule("MyModule").DefineType("MyType", TypeAttributes.Public | TypeAttributes.Class);
// ...
MethodBuilder entryPoint = tb.DefineMethod("Main", MethodAttributes.HideBySig | MethodAttributes.Public | MethodAttributes.Static);
ILGenerator il2 = entryPoint.GetILGenerator();
// ...
il2.Emit(OpCodes.Ret);
tb.CreateType();

MetadataBuilder metadataBuilder = ab.GenerateMetadata(out BlobBuilder ilStream, out BlobBuilder fieldData);
PEHeaderBuilder peHeaderBuilder = new PEHeaderBuilder(
                // Header for PEFileKinds.WindowApplication imageCharacteristics and subsystem need to be set
                imageBase: 0x00400000, // this is the default value, was not necessarily needed to set.
                imageCharacteristics: Characteristics.ExecutableImage,
                subsystem: Subsystem.WindowsGui);

ManagedPEBuilder peBuilder = new ManagedPEBuilder(
                header: peHeaderBuilder,
                metadataRootBuilder: new MetadataRootBuilder(metadataBuilder),
                ilStream: ilStream,
                mappedFieldData: fieldData,
                entryPoint: MetadataTokens.MethodDefinitionHandle(entryPoint.MetadataToken));

BlobBuilder peBlob = new BlobBuilder();
peBuilder.Serialize(peBlob);

// in case saving to a file:
using var fileStream = new FileStream("MyAssembly.exe", FileMode.Create, FileAccess.Write);
peBlob.WriteContentTo(fileStream); 

Old AssemblyBuilder.SetEntryPoint(MethodInfo) Proposal folded below:

Add AssemblyBuilder.SetEntryPoint(MethodInfo) method ### Background and motivation

As a new persist able AssemblyBuilder implementation is being added in .NET 9 it also needs an API for setting the Entry point. We could add similar but virtual API we have in .NET framework.

API Proposal

namespace System.Reflection.Emit;

public abstract partial class AssemblyBuilder
{
    // Previously approved new APIs
    public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);

    public void Save(Stream stream);
    public void Save(string assemblyFileName);
    protected abstract void SaveCore(Stream stream);
+   public virtual void SetEntryPoint(MethodInfo entryMethod);
}

API Usage

AssemblyBuilder ab = AssemblyBuilder.DefinePersistedAssembly(new AssemblyName("MyAssembly"), typeof(object).Assembly, null);
TypeBuilder tb = ab.DefineDynamicModule("MyModule").DefineType("MyType", TypeAttributes.Public | TypeAttributes.Class);

MethodBuilder mb1 = tb.DefineMethod("SumMethod", MethodAttributes.Public | MethodAttributes.Static, typeof(int), [typeof(int), typeof(int)]);
ILGenerator il = mb1.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Add);
il.Emit(OpCodes.Ret);

MethodBuilder main = tb.DefineMethod("Main", MethodAttributes.HideBySig | MethodAttributes.Public | MethodAttributes.Static);
ILGenerator il2 = main.GetILGenerator();
il2.Emit(OpCodes.Ldc_I4_S, 10);
il2.Emit(OpCodes.Ldc_I4_1);
il2.Emit(OpCodes.Call, mb1);
il2.Emit(OpCodes.Call, typeof(Console).GetMethod("WriteLine", [typeof(int)])!);
il2.Emit(OpCodes.Ret);
tb.CreateType();

ab.SetEntryPoint(main);
ab.Save("MyAssembly.exe");

Alternative Designs

Or keep the same pattern (have protected virtual *Core method).

namespace System.Reflection.Emit;

public abstract partial class AssemblyBuilder
{
    // Previously approved new APIs
    public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);

    public void Save(Stream stream);
    public void Save(string assemblyFileName);
    protected abstract void SaveCore(Stream stream);
+   public void SetEntryPoint(MethodInfo entryMethod);
+   protected virtual void SetEntryPointCore(MethodInfo entryMethod);
}

Activity

  1. added
    api-suggestionEarly API idea and discussion, it is NOT ready for implementation
    on Jan 16, 2024
  2. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Jan 16, 2024
  3. added this to the 9.0.0 milestone on Jan 16, 2024
  4. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Jan 16, 2024
  5. jkotas commented on Jan 16, 2024

    @jkotas
    Member

    public void SetEntryPoint(MethodInfo entryMethod)

    Does it need to be virtual?

  6. buyaa-n commented on Jan 16, 2024

    @buyaa-n
    ContributorAuthor

    Does it need to be virtual?

    Yes, it needs to be virtual, thank you. Updated the proposal to public virtual void SetEntryPoint(...), or can be protected virtual SetEntryPointCore(...) that will be called from the public void SetEntryPoint(...).

  7. added
    api-ready-for-reviewAPI is ready for review, it is NOT ready for implementation
    and removed
    api-suggestionEarly API idea and discussion, it is NOT ready for implementation
    on Jan 16, 2024
  8. reflectronic commented on Jan 30, 2024

    @reflectronic
    Contributor

    Is there a reason why SetEntryPoint(MethodInfo, PEFileKinds) from .NET Framework is not also being added? I notice that Save(string, PortableExecutableKinds, ImageFileMachine) is missing from the AssemblyBuilder proposal too. These overloads are needed for building certain kinds of .NET Framework assemblies. Is it expected that AssemblyBuilder.Save in .NET 9 will not support this?

  9. buyaa-n commented on Jan 31, 2024

    @buyaa-n
    ContributorAuthor

    PEFileKinds doesn't exist in .NET core and doesn't really have anything useful.

    For now, the assembly is DLL only if no entry point set, if user needs to set more than this we can consider adding API for setting PEHeaderBuilder in the future, but that option doesn't have to be associated with SetEntryPoint or Save method

    These overloads are needed for building certain kinds of .NET Framework assemblies. Is it expected that AssemblyBuilder.Save in .NET 9 will not support this?

    Please try out the default implementation and let us know what exact scenarios is not covered, and also check PEHeaderBuilder if there is an option that could cover your scenarios and let us know if there is anything not covered.

  10. bartonjs commented on Feb 6, 2024

    @bartonjs
    Member

    Video

    • We like the template method pattern approach better, especially because the SetEntryPoint method is already non-virtual in .NET Framework.
    • Since this is being written for future-proofing, we should go fully future-proof and include the PEFileKinds overloads.
    namespace System.Reflection.Emit;
    
    + public enum PEFileKinds
    + {
    +     Dll                = 0x0001,
    +     ConsoleApplication = 0x0002,
    +     WindowApplication = 0x0003,
    + }
    
    public abstract partial class AssemblyBuilder
    {
        // Previously approved new APIs
        public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);
    
        public void Save(Stream stream);
        public void Save(string assemblyFileName);
        protected abstract void SaveCore(Stream stream);
    +   public void SetEntryPoint(MethodInfo entryMethod);
    +   public void SetEntryPoint(MethodInfo entryMethod, PEFileKinds fileKind);
    +   protected virtual void SetEntryPointCore(MethodInfo entryMethod, PEFileKinds fileKind);
    }
  11. added
    api-approvedAPI was approved in API review, it can be implemented
    and removed
    api-ready-for-reviewAPI is ready for review, it is NOT ready for implementation
    on Feb 6, 2024
  12. 10 remaining items

  13. jkotas commented on Feb 20, 2024

    @jkotas
    Member

    Presumably the members on PersistedAssemblyBuilder would still need to be virtual if we want to support extensibility

    I think PersistedAssemblyBuilder should be a sealed type with no extensibility.

  14. buyaa-n commented on Feb 20, 2024

    @buyaa-n
    ContributorAuthor

    API Proposal

    namespace System.Reflection.Emit;
    
    public abstract partial class AssemblyBuilder
    {
        // Previously approved new APIs
    -   public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);
    
    -   public void Save(Stream stream);
    -   public void Save(string assemblyFileName);
    -   protected abstract void SaveCore(Stream stream);
    }
    
    +public sealed class PersistedAssemblyBuilder : AssemblyBuilder
    {
    +   public PersistedAssemblyBuilder(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);
    
    +   public void Save(Stream stream);
    +   public void Save(string assemblyFileName);
    +   public MetadataBuilder GenerateMetadata(out BlobBuilder ilStream, out BlobBuilder mappedFieldData);
    }

    API Usage

    PersistedAssemblyBuilder ab = new PersistedAssemblyBuilder(new AssemblyName("MyAssembly"), typeof(object).Assembly);
    TypeBuilder tb = ab.DefineDynamicModule("MyModule").DefineType("MyType", TypeAttributes.Public | TypeAttributes.Class);
    // ...
    MethodBuilder entryPoint = tb.DefineMethod("Main", MethodAttributes.HideBySig | MethodAttributes.Public | MethodAttributes.Static);
    ILGenerator il2 = entryPoint .GetILGenerator();
    // ...
    il2.Emit(OpCodes.Ret);
    tb.CreateType();
    
    MetadataBuilder metadataBulder = ab.GenerateMetadata(out BlobBuilder ilStream, out BlobBuilder fieldData);
    PEHeaderBuilder peHeaderBuilder = new PEHeaderBuilder(
                    // Header for PEFileKinds.WindowApplication imageCharacteristics and subsystem need to be set
                    imageBase: 0x00400000, // this is the default value, was not necessarily needed to set.
                    imageCharacteristics: Characteristics.ExecutableImage,
                    subsystem: Subsystem.WindowsGui);
    
    ManagedPEBuilder peBuilder = new ManagedPEBuilder(
                    header: peHeaderBuilder,
                    metadataRootBuilder: new MetadataRootBuilder(metadataBuilder),
                    ilStream: ilStream,
                    mappedFieldData: fieldData,
                    entryPoint: MetadataTokens.MethodDefinitionHandle(entryPoint.MetadataToken));
    
    BlobBuilder peBlob = new BlobBuilder();
    peBuilder.Serialize(peBlob);
    
    // in case saving to a file:
    using var fileStream = new FileStream("MyAssembly.exe", FileMode.Create, FileAccess.Write);
    peBlob.WriteContentTo(fileStream); 
    fileStream.Close();
  15. jkotas commented on Feb 20, 2024

    @jkotas
    Member

    public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable? assemblyAttributes = null);

    This can be a regular constructor. I do not think that a factory method serves any purpose here.

  16. added
    blockingMarks issues that we want to fast track in order to unblock other important work
    on Feb 22, 2024
  17. changed the title [-][API Proposal]: Add AssemblyBuilder.SetEntryPoint(MethodInfo) method[/-] [+][API Proposal]: Add PersistedAssemblyBuilder type[/+] on Feb 22, 2024
  18. buyaa-n commented on Feb 22, 2024

    @buyaa-n
    ContributorAuthor

    I have moved the API proposal to the description as it became completely different proposal than the original one, folded the old proposal in the description. Added blocking to get it reviewed ASAP.

  19. masonwheeler commented on Feb 23, 2024

    @masonwheeler
    Contributor

    @buyaa-n

    PersistedAssemblyBuilder ab = PersistedAssemblyBuilder.DefinePersistedAssembly(new AssemblyName("MyAssembly"), typeof(object).Assembly);
    TypeBuilder tb = ab.DefineDynamicModule("MyModule").DefineType("MyType", TypeAttributes.Public | TypeAttributes.Class);
    // ...
    MethodBuilder entryPoint = tb.DefineMethod("Main", MethodAttributes.HideBySig | MethodAttributes.Public | MethodAttributes.Static);
    ILGenerator il2 = main.GetILGenerator();
    

    main has not been defined in this sample code. Is it supposed to be the same thing as entryPoint?

  20. buyaa-n commented on Feb 23, 2024

    @buyaa-n
    ContributorAuthor

    main has not been defined in this sample code. Is it supposed to be the same thing as entryPoint?

    Right, in the new example I changed the main -> entryPoint, forgot to update in ILGenerator il2 = main.GetILGenerator();, thanks!

  21. bartonjs commented on Mar 12, 2024

    @bartonjs
    Member

    Video

    Breaking API compatibility from .NET Framework (by having Save somewhere other than on AssemblyBuilder) is unfortunate; but seems tolerable based on expected usage.

    Given that, looks good as proposed.

    namespace System.Reflection.Emit;
    
    public abstract partial class AssemblyBuilder
    {
        // Previously approved new APIs
    -   public static AssemblyBuilder DefinePersistedAssembly(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);
    
    -   public void Save(Stream stream);
    -   public void Save(string assemblyFileName);
    -   protected abstract void SaveCore(Stream stream);
    }
    
    +public sealed class PersistedAssemblyBuilder : AssemblyBuilder
    {
    +   public PersistedAssemblyBuilder(AssemblyName name, Assembly coreAssembly, IEnumerable<CustomAttributeBuilder>? assemblyAttributes = null);
    
    +   public void Save(Stream stream);
    +   public void Save(string assemblyFileName);
    +   public MetadataBuilder GenerateMetadata(out BlobBuilder ilStream, out BlobBuilder mappedFieldData);
    }
  22. added
    api-approvedAPI was approved in API review, it can be implemented
    and removed
    blockingMarks issues that we want to fast track in order to unblock other important work
    api-ready-for-reviewAPI is ready for review, it is NOT ready for implementation
    on Mar 12, 2024
  23. added
    in-prThere is an active PR which will close this issue when it is merged
    on Mar 13, 2024
  24. locked and limited conversation to collaborators on Apr 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api-approvedAPI was approved in API review, it can be implementedarea-System.Reflection.Emitin-prThere is an active PR which will close this issue when it is merged

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions