New API for single-precision math

Rationale

The .NET framework does not currently provide scalar single-precision floating-point support for many of the trigonometric, logarithmic, and other common mathematical functions.

Single-precision floating-point support should be provided for these mathematical functions in order to better interop with high-performance, scientific, and multimedia-based applications where single-precision floating-points are the only implementation required and/or used.

Since adding these methods to the existing System.Math class would break backwards compatibility, they should be provided through a separate class that provides the same and/or similar functionality.

Providing these new APIs would:

improve code readability (especially in the non-hardware backed implementations of methods in the System.Numerics.Vector structs)
improve code performance in areas where the single-precision implementation is all that is required
provide better interoperability with high-performance, scientific, and multimedia-based applications where data structures and existing code may be dependent on single-precision implementations of such methods.
provide a more advanced infrastructure for mathematics in the CoreCLR

Proposed API

The proposed API here provides feature-parity with the existing double-precision math functions provided by the framework.

System.BitConverter

public static class BitConverter
{
    public static float Int32BitsToSingle(int);
    public static int SingleToInt32Bits(float);
}

System.Single

public static partial class MathF
{
    public const float E = 2.71828183f;
    public const float PI = 3.14159265f;

    // Trigonometric Functions
    public static float Acos(float);
    public static float Asin(float);
    public static float Atan(float);
    public static float Atan2(float, float);
    public static float Cos(float);
    public static float Sin(float);
    public static float Tan(float);

    // Hyperbolic Functions
    public static float Cosh(float);
    public static float Sinh(float);
    public static float Tanh(float);

    // Exponential Functions
    public static float Exp(float);

    // Logarithmic Functions
    public static float Log(float);
    public static float Log(float, float);
    public static float Log10(float);

    // Power Functions
    public static float Pow(float, float);
    public static float Sqrt(float);

    // Rounding Functions
    public static float Ceiling(float);
    public static float Floor(float);
    public static float Round(float);
    public static float Round(float, int);
    public static float Round(float, int, MidpointRounding);
    public static float Round(float, MidpointRounding);
    public static float Truncate(float);

    // Manipulation Functions
    public static int Sign(float);

    // Minimum, Maximum, and Difference Functions
    public static float Max(float, float);
    public static float Min(float, float);

    // Other Functions
    public static float Abs(float);
    public static float IEEERemainder(float, float);
}

Example Usage

Currently to calculate the Tangent for each member of the System.Numerics.Vector4 struct, you currently have to call the double-precision version of the method and cast to the result to a single-precision value:

public static Vector4 Acos(Vector4 value)
{
    return new Vector4((float)(Math.Acos(value.X)),
                       (float)(Math.Acos(value.Y)),
                       (float)(Math.Acos(value.Z)),
                       (float)(Math.Acos(value.W)));
}

With the proposed changes, this would now be simplified to the following:

public static Vector4 Acos(Vector4 value)
{
    return new Vector4(Mathf.Acos(value.X),
                       Mathf.Acos(value.Y),
                       Mathf.Acos(value.Z),
                       Mathf.Acos(value.W));
}

The System.Numerics library itself is filled with similar examples as are various bits of code in the CoreFX and CoreCLR repositories.

Perf Numbers

All performance tests are implemented as follows:

100,000 iterations are executed
The time of all iterations are aggregated to compute the Total Time
The time of all iterations are averaged to compute the Average Time
A single iteration executes some simple operation, using the function under test, 5000 times

The execution time below is the Total Time for all 100,000 iterations, measured in seconds.

Hardware: Desktop w/ 3.7GHz Quad-Core A10-7850K (AMD) and 16GB RAM

Function	Improvment	Execution Time - Double	Execution Time - Single
Abs	0.199243555%	0.63752649s	0.63625626s
Acos	12.30220910%	11.5265412s	10.1085220s
Asin	18.66801808%	11.9472425s	9.71692911s
Atan	21.10350002%	10.9964683s	8.67582861s
Atan2	20.51327307%	24.3328097s	19.3413540s
Ceiling	12.91487191%	1.87116459s	1.62950608s
Cos	5.026665542%	7.19916547s	6.83728750s
Cosh	16.46166555%	13.5416170s	11.3124413s
Exp	33.67586387%	6.65578424s	4.41439140s
Floor	10.39208688%	1.74655247s	1.56504922s
Log	19.81117664%	6.42244806s	5.15008553s
Log10	18.40605725%	6.75118866s	5.50856101s
Pow	47.85595440%	31.8820155s	16.6245727s
Round	0.976398142%	4.22620632s	4.18494172s
Sin	15.49539339%	5.98022268s	5.05356365s
Sinh	17.96609899%	14.6242270s	11.9968239s
Sqrt	4.676516651%	2.51281945s	2.39530703s
Tan	30.33470555%	9.07290178s	6.32066374s
Tanh	0.108182099%	8.12724112s	8.11844890s

I believe some extra perf will be squeezed out when the intrinsics (such as CORINFO_INTRINSIC_Sqrt) are properly implemented in the VM layer for single-precision values. Without such functionality, it falls back to the double-precision functionality (extra precision, reduced performance) for certain calls.

Pull Request

There is a sample pull request covering these changes available: dotnet/coreclr#5492

Additional Details

This will require several changes in the CoreCLR as well to support the new APIs via FCALLs and Intrinsics.

Why would we not add float overloads directly to System.Math?

This would break compatibility. If you call Math.Sqrt(2f) would now call the single precision overload with less precision instead of implicitly casting it to double.

Given the existing API System.Math.Sqrt(double), when you compile the code:
double result = System.Math.Sqrt(2)
The code compiles as a call to System.Math.Sqrt(2.0D).

However, if you add System.Math.Sqrt(float) and recompile the code:
double result = System.Math.Sqrt(2)
The code compiles as a call to System.Math.Sqrt(2.0F).

This results in different behavior and provides no warning as there isn't a data loss in the view of the compiler (when there is an actual loss of precision as compared to the previous release).

Ah, makes sense. Having said that, I wonder if there is some compatible approach that would allow us to have these members on the Math class. Maybe we could add methods with some naming scheme (prefix or suffix) or add a nested type to Math.

Well C uses the sqrt(double) and sqrtf(float)... Although the naming convention doesn't exactly fit (seeing as we have Single and not Float).

Single and Fast make sense as prefixes... 32 could be used as a postfix, but is somewhat ambiguous as it could also imply Int32.

I think a nested type would make it more difficult to access/use the methods. So in a language that doesn't allow using static System.Math.SingleMath, you would have to type Math.SingleMath.Sqrt() (or the equivalent) anytime you wanted to call the method.

A couple of other ideas:

Provide the new APIs in the System.Numerics namespace. It would fit well with the new SIMD enabled types and would likely be used in the same scenarios (e.g. gaming, scientific, and multimedia-based applications).
Provide the new APIs as part of the System.Single struct (and equivalent APIs in System.Double, System.Int32, etc). This would match the behavior of the equivalent APIs in the System.Numerics.Vector types.

Note that Unity already has a Mathf class that does exactly this: https://docs.unity3d.com/ScriptReference/Mathf.html

That's an indicator that such an API is useful and can help influence the vote for a good name.

@tannergooding

@tannergooding why don't you create a formal API proposal? We would discuss it at the next API review. I think, in general, the addition is something we should do. It's just a matter of deciding on the right shape/location/naming of the APIs.

BTW, you can read about the API review process at https://github.com/dotnet/corefx/wiki/API-Review-Process.

@KrzysztofCwalina

@KrzysztofCwalina, I have updated the first comment to more closely resemble a speclet. Please let me know if any additional changes should be made.

Updated the speclet to include the full set of proposed API changes and to provided extended details on the changes that would be provided.

I wouldn't mind having int.Max(a,b) styled methods, myself, because, while you wouldn't have the simplicity of a single starting namespace, you would now be able to easily specify the return type and know that return type just by looking at the methods. It might require more developer work, but sometimes it's better to have the option to be more explicit and descriptive.

Also keep in mind that "helper" classes are only helpful if you know about them. Recent arrivals might not be quick to hunt for a Math class. I've had to point it out to several new recruits myself who expected Min/Max methods on the base type. It lends some credence to the practice of putting methods in the base type when all the arguments and the return type match that base type.

That said, I don't necessarily think that we need to deprecate System.Math. I'd just dislike the code duplication of having the same method in two places (or the overhead of passing a call to a method on the base type). I don't seem to recall any Method redirecting functionality in .NET, which would be helpful for situations like this where we'd like a matching static signature in two separate places to point to the same IL. We could probably gear up for a great round of bike-shedding about proper architecture with this.

I'm not personally opposed to the fleshing out of primitives. As an unrelated example, I've seen argument exceptions occur when calling generic extensions that expected to return integers (Int32) as T and instead encountered shorts (Int16). The same with a double as T but receiving a single. Apparently there are certain scenarios where that conversion isn't automatic when the types aren't known at compile time. But there can be a lot of dragons in any given author's generic extension methods.

@ZigMeowNyan

@AdamsLair, thanks for the support.

I agree that moving them out of a centralized static class is probably not the greatest of options. However, unless the CoreCLR/CoreFX team decides they are willing to break backwards compatibility, it seems that we will end up with at least two distinct locations for any additional math APIs.

I think that, logically, it makes sense to provide these methods through the respective primitive types (as @ZigMeowNyan pointed out, many developers assume that they should exist there anyways).

However, I also agree that in terms of refactoring, ease of use, etc.. It would make sense to also provide them through a centralized class (one that can be easily updated in the future without breaking backwards compatibility).

Example:

public struct Double
{
    public static double Sqrt(double x);
}

public static class MathHelper
{
    public static double Sqrt(double x)
    {
        return double.Sqrt(x);
    }

    public static float Sqrt(float x)
    {
        return float.Sqrt(x);
    }
}

public struct Single
{
    public static double Sqrt(double x);
}

or similar to how System.Numerics.Vector does it:

public struct Double
{
    public static double Sqrt(double x);
}

public static class MathHelper<T> where T : struct
{
    [JitIntrinsic]
    public static T Sqrt(T x)
    {
        object value = (object)(x);

        if (typeof(T) == typeof(double))
        {
            value = double.Sqrt((double)(value));
        }
        else if (typeof(T) == typeof(float))
        {
            value = float.Sqrt((float)(value));
        }
        else
        {
            throw new NotSupportedException();
        }

        return (T)(value);
    }
}

public struct Single
{
    public static double Sqrt(double x);
}

I have updated the speclet to include the current open question of: Should the new APIs be provided through the respective primitive types, through a centralized static class, or both?

Also perhaps something like sincos since it's common to use both.

The core change on the CoreCLR side has gone in, so I think we can close this after the CoreFX side goes in.

There are three bugs tracking the finer implementation details still needed (dotnet/coreclr#7689, dotnet/coreclr#7690, and dotnet/coreclr#7691).

Great, thanks! Keep us posted on the progress.

BTW: It seems that you forgot to update the proposal on the top to MathF with capital F. Can you please do the edit to avoid any confusion? Thanks!

Fixed.

The CoreFX side has gone in as well now. I will continue working on the finer implementation details in my spare time. The additional work being:

https://github.com/dotnet/corefx/issues/12855 - Additional managed UnitTest coverage, to be at least on par with the PAL layer coverage we have
https://github.com/dotnet/coreclr/issues/7689 - Ensuring the intrinsic support is properly piped through
https://github.com/dotnet/coreclr/issues/7690 - Ensuring the CoreCLR UnitTest coverage is on par with the coverage provided for System.Math (most of this is JIT layer tests from the looks of things)
https://github.com/dotnet/coreclr/issues/7691 - Ensuring we have a valid set of Perf tests covering the new APIs

Additionally, it may be useful to review any existing code that currently does (float)Math.SomeCall to see if replacing it with a call to MathF is appropriate (several calls in System.Numerics come to mind here, for the fallback path when SIMD is not available).

Instead of adding SinCos I'd prefer teaching the JITter to optimize them, if it doesn't do so already.

@CodesInChaos

@CodesInChaos I dream of the day a compiler can use a solver to optimize things like trig functions to reduce them through symbolic logic to simpler/faster forms. Like sin(x) * sin(x) - cos(x)_cos(x) = -cos(2_x).

@SunnyWar

@SunnyWar

The optimization I'm talking about is much easier to find than yours. It's essentially a form of common sub-expression elimination, and thus similar to existing optimizations.

Apart from the difficulty of finding them, optimizations like yours are often invalid for floats even when they're valid for real numbers. Avoiding degraded precision and handling all special cases correctly (NaN, infinities, signed zeros, etc.).

@SunnyWar

@SunnyWar In theory analyzers could do that. Some do things like (though much simpler)

@CodesInChaos

@CodesInChaos

Instead of adding SinCos I'd prefer teaching the JITter to optimize them, if it doesn't do so already.

It doesn't but it would have the advantage of working retroactively.

They have had 14 years to implement that seemingly simple optimization though, so I wouldn't hold my breath. An intrinsic is the next best thing, and maybe it has less of a JIT performance impact, but I wouldn't know about that.

CoreFX fixed in dotnet/corefx#12183, closing.

Navigation Menu

Repository navigation

New API for single-precision math #14353

Rationale

Proposed API

System.BitConverter

System.Single

Example Usage

Perf Numbers

Pull Request

Additional Details

42 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

New API for single-precision math #14353

Description

Rationale

Proposed API

System.BitConverter

System.Single

Example Usage

Perf Numbers

Pull Request

Additional Details

Activity

KrzysztofCwalina commented on Mar 16, 2015

pdelvo commented on Mar 16, 2015

tannergooding commented on Mar 16, 2015

KrzysztofCwalina commented on Mar 18, 2015

tannergooding commented on Mar 18, 2015

MikePopoloski commented on Mar 18, 2015

KrzysztofCwalina commented on Mar 19, 2015

KrzysztofCwalina commented on Mar 19, 2015

tannergooding commented on Mar 19, 2015

tannergooding commented on Mar 20, 2015

ZigMeowNyan commented on Mar 21, 2015

tannergooding commented on Mar 21, 2015

GeirGrusom commented on Mar 21, 2015

42 remaining items

tannergooding commented on Oct 18, 2016

karelz commented on Oct 18, 2016

tannergooding commented on Oct 18, 2016

tannergooding commented on Oct 26, 2016

CodesInChaos commented on Oct 28, 2016

SunnyWar commented on Oct 28, 2016

CodesInChaos commented on Oct 29, 2016

shmuelie commented on Oct 30, 2016

GeirGrusom commented on Oct 31, 2016

karelz commented on Nov 28, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions