Repository navigation
Proposal: Add IdnMapping Span-based APIs #32411
Description
The current IdnMapping API accepts/returns strings and throws on invalid input. I propose a set of Span-based APIs to avoid allocations.
namespace System.Globalization
{
public sealed class IdnMapping
{
// Existing API
public string GetAscii(string unicode);
public string GetAscii(string unicode, int index);
public string GetAscii(string unicode, int index, int count);
public string GetUnicode(string ascii);
public string GetUnicode(string ascii, int index);
public string GetUnicode(string ascii, int index, int count);
// Proposed API
public string GetAscii(ReadOnlySpan<char> unicode);
public bool TryGetAscii(ReadOnlySpan<char> unicode, Span<char> destination, out int charsWritten);
public string GetUnicode(ReadOnlySpan<char> ascii);
public bool TryGetUnicode(ReadOnlySpan<char> ascii, Span<char> destination, out int charsWritten);
}
}Both Get and TryGet* methods would throw on invalid input.
TryGet* would return false on insufficient space in the destination span.
This new API would simplify call sites and remove allocations throughout code dealing with internationalized domain names, like Uri and Markdig.
cc: @tarekgh
Reacted by Copilot
Activity
tarekgh commentedon Feb 16, 2020
In general, the proposal looks reasonable.
Why we need the APIs:
I don't think these are useful if we are going to allocate a string anyway. And the other proposed APIs can be used at that time. What do you think about that?
MihaZupan commentedon Feb 16, 2020
Those aren't needed in my use-cases, as
Try*would always be used.I'll remove them as they could easily be exposed later if a use-case presents itself.
stephentoub commentedon Jun 12, 2020
I don't understand these APIs:
What does the
Trymean here? If it's to avoid throwing in the case where the data is somehow invalid, that's a different meaning than the otherTryoverloads would have, which would be based solely on whether the destination is large enough. The Boolean returned from aTryis supposed to convey only one thing, and in such span-based Try methods, it's always used to connote whether the destination was large enough to store the transformed data. I don't think we want two overloads of the same method having a different meaning for theTry.MihaZupan commentedon Jun 12, 2020
The idea was for both the
out stringandSpan destinationto return false on invalid input. The span one would also return false on insufficient space.I agree the Span overload would be confusing to use since you couldn't differentiate between invalid input/insufficient space, without ensuring you supply a worst-case sized buffer.
Do we have a pattern of
Try*methods ever throwing? If so, we could have all overloads throw on invalid input, where theTryonly returns false on insufficient space.Alternatively, we would need an
OperationStatus-style return?stephentoub commentedon Jun 12, 2020
Yes, Try methods can still throw.
Why not just:
?
If the exception for invalid input really is unexceptional, though, with the exception happening so frequently as to be a performance problem in real situations, then yeah, OperationStatus is what you'd want.
Can you share examples where the exception is a meaningful problem?
MihaZupan commentedon Jun 12, 2020
Yes, that is the shape we'd want (lazy copy-pasting, I've edited the comment).
A few examples of exceptional inputs from reading IdnMapping code:
-before.):foo-.barI wouldn't expect the performance overhead of exceptions to be a problem for well-behaving apps.
Edit:
No.
5 remaining items
ericstj commentedon Jul 12, 2021
This is too late for 6.0.0 at this point.
terrajobst commentedon Aug 8, 2023
Video
stephentoub commentedon Feb 6, 2025
@MihaZupan, there are fast paths available for the current methods, where the original string is returned if no encoding is required. What is the intent for the overloads here that take a span and return a string? Won't those now force allocation in cases where the existing APIs didn't?
MihaZupan commentedon Feb 6, 2025
Yes, they would possibly allocate more if the caller changed from
GetAscii(s, i, length)to using spans and happened to have0ands.Lengthoffsets.The idea is similar as with e.g.
Uri.{Un}EscapeDataString(span), so you can use the simple helper with inputs other than string.Looking through the usages in the BCL and Markdig, we don't have a need for it. Inputs are either already just
string, or we'd also want to avoid the result allocation. I'd be fine with just leaving those out.