Enhance user-facing API for strongly-typed ILogger messages

Is your feature request related to a problem? Please describe.

I'm trying to declare a strongly-typed code path for ILogger messages, but it is difficult to use the Action-returning static method LoggerMessage.Define<T1,T2,...,Tn> in a way that's approachable and maintainable by an entire team.

Describe the solution you'd like

It would be great if there was a strongly-typed struct which could wrap the underlying LoggerMessage.Define call and Action<ILoggerFactory, ...., Exception> field - even completely hide the actual Action signature. Ideally it would also provide more appropriate intellisense on the struct method which calls the action. If would be even cleaner if it could be initialized without repeating the generic argument types, and in most end-user applications if the EventId number was optional it could be derived from the EventId name instead.

internal static class MyAppLoggerExtensions
{
  private readonly static LogMessage<string, int> _logSayHello = (
    LogLevel.Debug, 
    nameof(SayHello), 
    "The program named {ProgramName} is saying hello {HelloCount} times");

  /// <summary>
  /// The program named {ProgramName} is saying hello {HelloCount} times
  /// </summary>
  /// <param name="logger">The logger to write to</param>
  /// <param name="programName">The {ProgramName} message property</param>
  /// <param name="helloCount">The {HelloCount} message property</param>
  public static void SayHello(this ILogger<MyApp> logger, string programName, int helloCount)
  { 
    _logSayHello.Log(logger, programName, helloCount);
  }
}

A LogMessage struct would be a allocation-free. Having implicit operators for tuples which match the constructor parameters enables field initialization without needing to repeat the generic argument types.

public struct LogMessage<T1, ..., Tn>
{
  public LogMessage(LogLevel logLevel, EventId eventId, string formatString);
  public LogMessage(LogLevel logLevel, int eventId, string formatString);
  public LogMessage(LogLevel logLevel, string eventName, string formatString);

  public LogLevel LogLevel {get;}
  public EventId EventId {get;}
  public string FormatString {get;}

  public void Log(ILogger logger, T1 value1, ..., Tn valuen);
  public void Log(ILogger logger, Exception exception, T1 value1, ..., Tn valuen);

  public static implicit operator LogMessage<T1, ..., Tn>((LogLevel logLevel, EventId eventId, string formatString) parameters);
  public static implicit operator LogMessage<T1, ..., Tn>((LogLevel logLevel, int eventId, string formatString) parameters);
  public static implicit operator LogMessage<T1, ..., Tn>((LogLevel logLevel, string eventName, string formatString) parameters);
}

Describe alternatives you've considered

Having one struct per LogLevel could also remove the need to have the first "LogLevel" argument in the initializer - but that would be a larger number of classes and intellisense wouldn't list the level names for you like typing LogLevel. does

@lodejard

@lodejard After giving this a second look I wonder how much this matters given that the logger message is mostly used as an internal detail of these static methods. Would it be better to use a type as IDL for code generating one of these objects.

public partial class Log
{
    [LoggerMessage(LogLevel.Debug, 1, nameof(SayHello), "The program named {ProgramName} is saying hello {HelloCount} times")]
    public static partial void SayHello(ILogger logger, string name, int helloCount);
}

Then it would generate this:

public partial class Log
{
    private static readonly Action<ILogger, string, int, Exception?> _sayHello = 
        LoggerMessage.Define<string, int>(LogLevel.Information, new EventId(1, "SayHello"), "The program named {ProgramName} is saying hello {HelloCount} times");

    public static partial void SayHello(ILogger logger, string programName, int helloCount)
    {
        _sayHello(logger, programName, helloCount, null);
    }
}

Usage would look like:

// Without extension methods
Log.SayHello(logger, "Louis", 10);

// With extension methods
logger.SayHello("Louis", 10);

We need partial extension methods @MadsTorgersen @jaredpar. I keep running into cases where small language limitations make the user experience slightly worse for source generators.

@noahfalk

cc @noahfalk @shirhatti

I've been working on a model that uses C# 9.0 source generators. The developer writes:

    [LogInterface(generatedTypeName: "DiagExtensions", generateExtensionMethods: true)]
    interface IDiagExtensions
    {
        [LogMethod("Could not open socket to `{hostName}`", LogLevel.Critical)]
        void CouldNotOpenSocket(string hostName);
    }

And the following is generated automatically:

namespace LoggingExample
{

    static class DiagExtensions
    {
        
        private readonly struct __CouldNotOpenSocketStruct__ : IReadOnlyList<KeyValuePair<string, object>>
        {
            private readonly string hostName;

            public __CouldNotOpenSocketStruct__(string hostName)
            {
                this.hostName = hostName;

            }

            public string Format()
            {
                return $"Could not open socket to `{hostName}`";
            }

            public KeyValuePair<string, object> this[int index]
            {
                get
                {
                    switch (index)
                    {
                        case 0:
                            return new KeyValuePair<string, object>(nameof(hostName), hostName);

                        default:
                            throw new ArgumentOutOfRangeException(nameof(index));
                    }
                }
            }

            public int Count => 1;

            public IEnumerator<KeyValuePair<string, object>> GetEnumerator()
            {
                yield return new KeyValuePair<string, object>(nameof(hostName), hostName);

            }

            IEnumerator IEnumerable.GetEnumerator()
            {
                return GetEnumerator();
            }
        }
        private static readonly EventId __CouldNotOpenSocketEventId__ = new(0, nameof(CouldNotOpenSocket));

        public static void CouldNotOpenSocket(this ILogger logger, string hostName)
        {
            if (logger.IsEnabled((LogLevel)5))
            {
                var s = new __CouldNotOpenSocketStruct__(hostName);
                logger.Log((LogLevel)5, __CouldNotOpenSocketEventId__, s, null, (s, ex) => s.Format());
            }
        }

    }
}

There is also an option to generate a wrapper type for the ILogger, rather than extension methods:

    [LogInterface(generatedTypeName: "DiagWrapper")]
    interface IDiagWrapper
    {
        [LogMethod("Could not open socket to `{hostName}`", LogLevel.Critical)]
        void CouldNotOpenSocket(string hostName);
    }

which comes out as:

namespace LoggingExample
{

    sealed class DiagWrapper : IDiagWrapper
    {
        private readonly ILogger __logger;

        public DiagWrapper(ILogger logger) => __logger = logger;

        
        private readonly struct __CouldNotOpenSocketStruct__ : IReadOnlyList<KeyValuePair<string, object>>
        {
            private readonly string hostName;

            public __CouldNotOpenSocketStruct__(string hostName)
            {
                this.hostName = hostName;

            }

            public string Format()
            {
                return $"Could not open socket to `{hostName}`";
            }

            public KeyValuePair<string, object> this[int index]
            {
                get
                {
                    switch (index)
                    {
                        case 0:
                            return new KeyValuePair<string, object>(nameof(hostName), hostName);

                        default:
                            throw new ArgumentOutOfRangeException(nameof(index));
                    }
                }
            }

            public int Count => 1;

            public IEnumerator<KeyValuePair<string, object>> GetEnumerator()
            {
                yield return new KeyValuePair<string, object>(nameof(hostName), hostName);

            }

            IEnumerator IEnumerable.GetEnumerator()
            {
                return GetEnumerator();
            }
        }
        private static readonly EventId __CouldNotOpenSocketEventId__ = new(0, nameof(CouldNotOpenSocket));

        public void CouldNotOpenSocket(string hostName)
        {
            if (__logger.IsEnabled((LogLevel)5))
            {
                var s = new __CouldNotOpenSocketStruct__(hostName);
                __logger.Log((LogLevel)5, __CouldNotOpenSocketEventId__, s, null, (s, ex) => s.Format());
            }
        }

    }
}

I believe this code is both more convenient and more efficient than existing approaches used in the wild.

Based on the example I provided above, I propose that we extend the framework with support to automatically generate strongly-type logging methods, based on a developer-authored interface type. The developer provides the minimum amount of information necessary, and the source generator produces a highly efficient set of logging methods. The generated logging methods can be produced as extension methods on ILogger, or can be produced as a type that wraps ILogger and exposes instance logging methods.

More specifically:

Introduce the types LogInterfaceAttribute and LogMethodAttribute to the Microsoft.Extensions.Logging namespace in the Microsoft.Extensions.Logging.Abstraction assembly.

    /// <summary>
    /// Indicates an interface defines strongly-typed logging methods
    /// </summary>
    /// <remarks>
    /// This interface is a trigger to cause code generation of strongly-typed methods
    /// that provide a convenient and highly efficient way to log through an existing
    /// ILogger interface.
    /// 
    /// The declaration of the interface to which this attribute is applied cannot be
    /// nested in another type.
    /// </remarks>
    [System.AttributeUsage(System.AttributeTargets.Interface, AllowMultiple = false)]
    public sealed class LogInterfaceAttribute : Attribute
    {
        /// <summary>
        /// Creates an attribute to annotate an interface as triggering the generation of strongly-typed logging methods.
        /// </summary>
        /// <param name="generatedTypeName">Determines the name of the generated type.</param>
        /// <param name="generateExtensionMethods">Determines whether the generated type uses extension methods or instance methods.</param>
        public LogInterfaceAttribute(string generatedTypeName, bool generateExtensionMethods = false) => (GeneratedTypeName, GenerateExtensionMethods) = (generatedTypeName, generateExtensionMethods);

        /// <summary>
        /// Gets the name of the generated type.. 
        /// </summary>
        public string GeneratedTypeName { get; }

        /// <summary>
        /// Gets whether the generated type exposes the strongly-typed methods as extension methods or instance methods.
        /// </summary>
        public bool GenerateExtensionMethods { get; }
    }

    /// <summary>
    /// Provides information to guide the production of a strongly-typed logging method.
    /// </summary>
    /// <remarks>
    /// This attribute is applied to individual methods in an interface type annotated with [LogInterface].
    /// </remarks>
    [System.AttributeUsage(System.AttributeTargets.Method, AllowMultiple = false)]
    public sealed class LogMethodAttribute : Attribute
    {
        /// <summary>
        /// Creates an attribute to guide the production of a strongly-typed logging method.
        /// </summary>
        /// <param name="message">The message text output by the logging method. This string is a template that can contain any of the method's parameters.</param>
        /// <param name="level">THe logging level produced when invoking the strongly-typed logging method.</param>
        /// <example>
        /// [LogInterface("Diag")]
        /// interface IDiag
        /// {
        ///     [LogMethod("Could not open socket for {hostName}", LogLevel.Critical)]
        ///     void CouldNotOpenSocket(string hostName);
        /// }
        /// </example>
        public LogMethodAttribute(string message, LogLevel level) => (Message, Level) = (message, level);

        /// <summary>
        /// Gets the message text for the logging method.
        /// </summary>
        public string Message { get; }

        /// <summary>
        /// Gets the logging level for the logging method.
        /// </summary>
        public LogLevel Level { get; }
    }

Introduce a source generator in the Microsoft.Extensions.Logging.Generators namespace in a new Microsoft.Extensions.Logging.Generators.dll assembly. This source generator would interpret the [LogInterface] and [LogMethods] attributes to generate strongly-typed logging methods for every logging method defined by the user. The motivation for putting the generator logic into a separate assembly is to avoid pulling in this build-time assembly into the application distribution.

I haven't contributed to this project before, but I'd be happy to learn the ropes and get something like the above checked in.

It does matter if the rationale provided is "complicated logic in the generator".

Again, this wasn't saying "oh man this is going to be hard to implement so we shouldn't expose the API we want", this was saying "the generator handles all the logic so we can expose the API we want".

Regarding:

Error 10: Either generics should work, or it also should check that it's not in a generic type

The constraint is currently that the logging method cannot be generic. The type the method is in can be generic, and method arguments can be of generic types. But the method itself cannot be generic.

The constraint is currently that the logging method cannot be generic

What leads to that constraint being necessary? I'm unclear what it prevents that couldn't be achieved via a type parameter, which is currently allowed. Hence the suggestion that both should be allowed or disallowed equally. Is there a technical limitation?

The constraint is currently that the logging method cannot be generic

What leads to that constraint being necessary? I'm unclear what it prevents that couldn't be achieved via a type parameter, which is currently allowed. Hence the suggestion that both should be allowed or disallowed equally. Is there a technical limitation?

I don't believe there is a technical problem. I didn't see the value in a generic logging method so I didn't put the extra effort necessary to parse, emit, and test this code path. This seemed like something that could clearly be added in the future if a need ever arose. You wouldn't want to complicate the generator logic prematurely, right? :-)

I don't understand why it would be a good idea to say "if you can't do generic methods, than you shouldn't do generic types either". Those seem orthogonal. It's quite possible (and indeed common) for a logging method to be introduced in a generic type. But introducing a generic method by itself seems much less likely.

In the implementation, there is a substantial difference in supporting generic types vs. generic methods. Generic types are mostly free. Support generic methods is more involved as the code generator needs to not only parse these definitions, it needs to generate a corresponding signature. So it would need to parse N generic definitions along with their corresponding constraints, and then emit them back out on the generated method.

You wouldn't want to complicate the generator logic prematurely, right?

I feel like I need to reiterate again that, to my knowledge, no one has expressed concern about how complex the generator is.

why it would be a good idea to say "if you can't do generic methods, than you shouldn't do generic types either"

It depends on the reason for the limitation. One reason, for example, would be that the purpose of such generated logging methods is to enforce strong typing on the structured logging, such that, for example, you could map each argument to its own database column or the like. A T parameter, regardless of whether it comes from a generic method parameter or a generic type parameter, could easily cause problems for that, with various calls each contributing their own unique type that then may not be supported by the underlying store. So if the concern were that, you'd want to ban it for any generic parameter, regardless of where it came from.

Instead, you've stated that's not the reason, but rather purely an artificial constraint based on viewed demand and the difficulty of implementation. In that case, sure, there's no reason to cut off the easier thing that "just works", though I expect in the fullness of time you'll want to remove the generic method limitation as well.

But introducing a generic method by itself seems much less likely.

Really? We do that in a variety of places. For example, we want a semi-structured trace logging method that can be used to trace out messages in a large number of places, with some arguments fixed for all tracing sites and some variable, e.g.

Log(httpConnectionPool, httpConnection, "Trace: {0}", additionalData);

where additionalData may be a value type. We don't want to add overloads for every possible T that additionalData may be, nor do we want to have to add guard calls around every use of Log to prevent additionalData from being boxed. If this Log could actually be Log<T>, it'd be simple to do. We could work around that by putting this Log method on its own generic type, and instead write Tracing<TypeOfAdditionalData>.Log(httpConnectionPool, httpConnection, "Trace: {0}", additionalData), but then we lose type inference and it's more code at every usage site, plus it ends up then needing to be separate from the rest of the logging methods we use.

That said, for the particular cases where the above arises, we're much more likely to use C# 10 interpolation support, write:

Log(ttpConnectionPool, httpConnection, $"Trace: {additionalData}");

and then not only avoid boxing for additionalData, but also not have to add a guard at the call site even if additionalData is actually a complicated and potentially expensive expression.

Which is to say, I believe there are uses, but we ourselves in the core libraries aren't going to use this mechanism, so this feedback is about use cases rather than strong need. If we can't support generic method parameters right now because it actually is too complex for the generator right now and we want to wait for further feedback before investing in it, that's reasonable.

Does the source generator only target the LoggerMessageAttribute or does it also target attributes that inherits LoggerMessageAttribute?
A user could creates its own attribute which inherits from LoggerMessageAttribute and set the constructor with whichever property that the application needs.

public class LogErrorAttribute : LoggerMessageAttribute
{
    public LogErrorAttribute(int eventId, string message)
    {
        EventId = eventId;
        Message = message;
        Level = LogLevel.Error;
    }
}

public class LogInfoAttribute : LoggerMessageAttribute
{
    public LogErrorAttribute(int eventId, string message)
    {
        EventId = eventId;
        Message = message;
        Level = LogLevel.Information;
    }
}

A user could creates its own attribute which inherits from LoggerMessageAttribute and set the constructor with whichever property that the application needs.

Note that this is a sealed attribute

Note that this is a sealed attribute

Oh I missed that. So I think what I'm asking is that could it not be sealed so the users could customize their own log attribute?

@Kahbazi

@Kahbazi, I think that's an intriguing idea, but it wouldn't 'just work' I think. Given how attributes are parsed, there's dedicated logic that looks at ctor parameters or properties, etc. Maybe there's a way to write attribute parsing logic that is inherently more generic in nature and could tolerate inheritance like that, but that's now how the source generator works at the moment.

@lodejard

@lodejard the logging generator got submitted into Microsoft.Extensions.Logging package version 6.0.0-preview.4.21216.3,
and here is a Hello World type gist using it.

Navigation Menu

Repository navigation

Enhance user-facing API for strongly-typed ILogger messages #36022

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

82 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Enhance user-facing API for strongly-typed ILogger messages #36022

Description

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Activity

davidfowl commented on Nov 15, 2020

davidfowl commented on Nov 15, 2020

geeknoid commented on Nov 16, 2020

geeknoid commented on Nov 17, 2020

82 remaining items

stephentoub commented on Apr 10, 2021

geeknoid commented on Apr 10, 2021

stephentoub commented on Apr 10, 2021

geeknoid commented on Apr 10, 2021

geeknoid commented on Apr 10, 2021

stephentoub commented on Apr 10, 2021

Kahbazi commented on Apr 10, 2021

maryamariyan commented on Apr 10, 2021

Kahbazi commented on Apr 10, 2021

geeknoid commented on Apr 10, 2021

maryamariyan commented on Apr 23, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions