Repository navigation
SslStream delayed client certificate negotiation #49346
Description
Background and Motivation
Related:
dotnet/aspnetcore#23948
#26210
Http servers like IIS, Http.Sys, HttpListener, etc. can conditionally request a client certificate after receiving the http request. Kestrel does not due to a lack of support in SslStream. Customers ask for this because they only want to require a client certificate for some pages/paths in their application.
Alternative strategies have proven too breaking for applications to adopt when moving from IIS to Kestrel.
Proposed API
namespace System.Net.Security
{
public class SslStream {
+ public Task<X509Certificate2?> GetClientCertificateAsync(CancellationToken cancellationToken)No reads or writes would be allowed in parallel with this operation in order to avoid corrupting internal state.
Should successes and failures be cached? E.g. what happens if you call this API more than once per connection? What if a client cert was provided during the initial handshake?
What validation should happen on this certificate?
Risks
- POST Deadlocks: It's common for Http.Sys users of this feature to deadlock for POST request scenarios if the TCP window fills up with body data before the certificate is requested.
- The app can mitigate this by draining the expected data before requesting the certificate.
- 100-continue mechanics can also be used to delay the request body, but that is timing sensitive.
- SslStream should fail the connection if any application data is received while the client certificate request is outstanding. That avoids all questions about buffering or parallelism.
- What about HTTP pipelining? An app processing one request at a time wouldn't know about pipelined requests.
- In TLS 1.2 and lower the client certificate is requested by renegotiating the TLS context. This is called out as a security risk in the spec. Explore why.
- TLS 1.3 has removed the renegotiation feature and replaced it with a dedicated extension for requesting the client certificate. The same public API should work across protocols.
- HTTP/2 has explicitly banned renegotiation and the new TLS 1.3 extension due to head of line blocking. Kestrel will need to prevent this API from being called for HTTP/2 connections.
- HTTP/3: Edit: Quic requires TLS 1.3 and explicitly prohibits the new extension.
Activity
scalablecory commentedon Mar 9, 2021
This seems like a 99th percentile perf issue waiting to be discovered -- can you go into more detail on why this behavior is needed?
scalablecory commentedon Mar 9, 2021
CC @nibanks any plans for MsQuic delayed client certs?
nibanks commentedon Mar 9, 2021
Can you clarify what the ask is here?
Tratcher commentedon Mar 9, 2021
A) It's unclear if the TLS state would allow data to be encrypted or decrypted during a renegotiation since the ciphers could change, I'll defer to Tomas on that.
B) This avoids the deadlock issue discussed above.
nibanks commentedon Mar 9, 2021
Ok. Read a bit more. No, we don't support that TLS extension, nor do we plan to expose that type of functionality in our API.
Tratcher commentedon Mar 9, 2021
@nibanks Are there known blockers or is it a matter of priorities?
nibanks commentedon Mar 9, 2021
TLS has to support it first. Then expose an API for it. Then we need to understand how it would would with QUIC (if at all; I don't know anything about the mentioned extension) and then we need to figure out how/if we can expose this in our API. And then there is the question of prioritization; which I'd expect this to fall fairly low on the list.
Tratcher commentedon Mar 10, 2021
@nibanks nevermind, Quic (and HTTP/3) also prohibits the use of this extension.
https://quicwg.org/base-drafts/draft-ietf-quic-tls.html#section-4.4
That limits the usefulness of this feature to HTTP/1.x (and non-HTTP protocols).
wfurt commentedon Mar 10, 2021
It is part of base TLS 1.3 RFC https://tools.ietf.org/html/rfc8446#section-4.6 @nibanks. But as @Tratcher mentioned, it seems to be forbidden by QUICK.
I did some experiments and the client certificate comes in via EncryptedHandsahke message e.g. not in cleartext as usual. That may be interesting if privacy is concern regardless of the multiplexing.
This works with with TLS 1.3 - tested with openssl s_client and Windows Insider build. As long as client indicates willingneess via post_handshake_auth. Interestingly, neither Chrome, Safari nor HttpClient sets that. (by default?)
Perhaps HttpClient can do it if LocalCertificateSelectionCallback is set.
Setting RemoteCertificate impacts IsMutuallyAuthenticated. If we re-use (but we don't have to) handshake logic, standard verification logic would apply e.g. we would build a chain and if RemoteCertificateValidationCallback is set, it would be called as well. That may seems redundant but mimics current flow.
This generally seems like server feature. So on client, we would either throw or simply return RemoteCertificate. If we return RemoteCertificate, I would do the same on server e.g. if the certificate exist (was provided during handshake) then return it. Otherwise ask for it. This would effectively block option to ask multiple times, perhaps for different certificates. But I don't know if that is need or if IIS supports that. So I would probably go with one shot for simplicity unless there is indication that something else is needed.
cc: @blowdart @bartonjs @GrabYourPitchforks for any security concerns.
17 remaining items
wfurt commentedon Apr 23, 2021
If validation fails, the
NegotiateClientCertificateAsyncwould throw. I bump to this when writing tests and client did not return certificate. Then it would always throw without validation callback. The behavior for unknown/untrusted certificate would be similar.I can make the certificate appear as SslStream.RemoteCertificate in either case. (validated or not)
Tratcher commentedon Apr 23, 2021
But would the stream still be usable or would it be aborted?
If NegotiateClientCertificateAsync is canceled then the stream is aborted, correct?
Being inconsistent seems concerning. What if validation failures were only reported to RemoteCertificateValidationCallback? Or made available through a parallel property?
Tratcher commentedon Apr 23, 2021
We realized there's a problem with this approach. Kestrel always has a loop running that reads from the SslStream and copies into a Pipe. While we wouldn't want the client to send any data during NegotiateClientCertificateAsync, there's no way to interrupt a pending read in order to call NegotiateClientCertificateAsync.
Would it be possible to allow the pending read to sit paused so long as no data was received during the NegotiateClientCertificateAsync call? Or would we need some way to cancel the read without aborting the SslStream?
wfurt commentedon Apr 23, 2021
The underlying SslStream does not need to be aborted if renegotiation finished (or at least did not fail) even if the
NegotiateClientCertificateAsyncthrows. I think it should be up to the call how they want to treat no or untrusted certificate.For the cancellation, I would need to experiment more.
The
NegotiateClientCertificateAsyncwill initiate the renegotiation, but that can be completed independently as long as the sides try to read and write to the stream. This can already happened today transparently if the renegotiation is requested by the remote peer.And yes, I think we can accommodate the pending read. On Windows the renegotiate code call
ReadAsyncInternalaway to get the processing going. Perhaps I can adjust that and do it only if there is no pending read and guard agains changes .e.g. if there was no read, no new one would be allowed.I have Windows implementation now and I'm working on tests to cover some of the scenarios we discussed. (hoping to get PR up today or early next week)
My goal was to release that early so we can experiment with Kestrel integration and make adjustments. Linux implementation would follow eventually.
davidfowl commentedon Apr 23, 2021
We only do this with HTTP/2 which this feature is blocked for anyways right?
halter73 commentedon Apr 23, 2021
You're right @davidfowl . This would have been a problem back when we had a read loop in Kestrel's HttpsConnectionMiddleware, but I forgot that you changed it to use stream-to-pipe adapter a long time ago when @Tratcher and I were discussing it earlier.
wfurt commentedon Apr 23, 2021
I'm wondering if there is harm in letting the read working. Seems finicky to build feature on latest Kestrel behavior.
As minimum, that could help draining the pipe if there is any lingering traffic from the peer.
davidfowl commentedon Apr 23, 2021
Nah, just make it fail.
wfurt commentedon May 14, 2021
This is reminder of few things missed in initial #51905
missing tests for 2 & 3. This will be impacted by refactoring unrelated to delayed certificate
We should add tests for early certificate with TLS 1.3 - at least on windows. There seems to be strange interaction as TLS 1.3 handshake completion use renegotiation path so as delayed certificate.
Tratcher commentedon May 28, 2021
What happens if AllowRenegotiation is set to false and NegotiateClientCertificateAsync is called? Does it no-op, throw, or renegotiate?
AllowRenegotiation was added for HTTP/2 to prevent the remote party from initiating a renegotiation. Since APLN could result in the connection using HTTP/1.1 it might be OK to allow the server to initiate a renegotiation even if AllowRenegotiation is disabled. It would be the caller's responsibility to prevent that for HTTP/2.
wfurt commentedon May 28, 2021
I'll need to test it but I think it will work on Linux and fail and throw on Windows.
We do copy the provided options so setting them after the stream is negotiated does nothing. There is currently no good way how to enforce the HTTTP/2 requirement AFAIK.
wfurt commentedon Jul 15, 2021
Should be functional on Windows and Linux with current OpenSSL.
There is no support on macOS (and will not be as there is no API support).
2 remaining issues are tracked separately: