gh-136306: Add support for SSL groups #136307
Conversation
This is an initial implementation of the feature proposed in issue python#136306.
Looks like the indentation is required. Got a doc build error without it.
|
I'm not sure why the latest round of tests failed - The only change in this last commit was a doc file change, so I'm guessing it's a transient CI failure. Is there a way to trigger it to retry? |
You can make an empty commit or ask a triager to rerun the CI (but there is none apart from me that is watching the PR I think) |
There was a problem hiding this comment.
This looks great. I was planning to add some PQ support to Python now that ML-KEM/DEM were standardized but this is a good start as well. I'll have a look at the tests when I'm on my laptop again (Friday) but you can consider this PR to be approved unless I find something by then.
Thanks very much! There seems to be less urgency in providing support for ML-DSA and SLH-DSA for authentication than there was for ML-KEM for key agreement mainly because of the "capture now, decrypt later" concerns, but I'd be happy to contribute to an effort around supporting those as well. I'd actually like to learn more about what OpenSSL APIs actually need to change here. |
|
I am currently modernizing the use of OpenSSL HMAC in hashlib and I actually wondered whether we used APIs that were deprecated in 3.0. If you want, you can help me here, at least for the SSL module (I'd prefer that we work on separate modules to avoid merge conflicts), namely look at the calls we make to OpenSSL and check if there are deprecated calls in 3.x. If there are, open an issue and a PR that modernize such calls (for HMAC, we moved from using the old HMAC_* interface to the more generic EVP_MAC interface) |
I don't know if I'll have the cycles to actually take on fixing all the issues that we find, but I gave this a quick look to get a sense for the scope of the problem. There aren't as many changes as I was expecting to find, but it's still fairly complicated if we want to continue to support all the way back to OpenSSL 1.1.1 while avoiding functions deprecated in OpenSSL 3.x. In many cases, the old API calls still exist in 3.x but the new API will only work on 3.x. I'm thinking leaving the old calls in place may be our best bet for now, to avoid conditional compilation. An example is replacing all references to BIO_new() with BIO_new_ex() where we pass in an explicit NULL argument for the library context. It doesn't seem worth a #if everywhere that appears as long as 3.x continues to provide a wrapper for the old BIO_new() call. A similar issue shows up around one use of BIO_new_mem_buf(). Here are some of the items I've found:
|
|
Following up on my last comment: In the SSL module docs, there's a "TLS 1.3" section which actually mentions that |
|
Helping fixing those issues or modernizing API calls is appreciated so choose what you want! |
|
Thanks! I've actually got another change ready which adds support for setting TLS 1.3 ciphers using a new It'll probably be easier if I wait until this PR is merged before I create a new issue and PR to avoid conflicts in the "what's new" entry. Alternately, if you want me to expand the scope of this issue/PR to cover both setting TLS 1.3 cipher and TLS 1.3 groups and check in the additional changes hree, I could do that. While they're independent, both of them are about providing more complete TLS 1.3 support, so I'm good with either option. |
|
I would prefer having separate issues and PRs even though they are related. It makes reverting stuff easier. I will be available tomorrow (maybe this evening, Paris time) |
|
No problem - I'll hold onto the cipher suite change for now. Also, I have a third change planned to add support for getting and setting sigalgs. It should be able to follow the same pattern as ciphers and groups. Once those changes are in, I'll revisit use of deprecated APIs, but for now I'd recommend only rewriting such code if the preferred replacement is supported in 1.1.1 or earlier. That will avoid needing to keep two versions of the code around depending on the OpenSSL version. As an example, I think replacing code which uses DH APIs with EVP APIs should be doable without any compile-time conditionals, but for now I'd avoid replacing calls to BIO_new, as the replacement function BIO_new_ex() doesn't appear until OpenSSL 3.0. |
Lib/test/test_ssl.py
Outdated
| def test_set_groups(self): | ||
| ctx = ssl.create_default_context() | ||
|
|
||
| # Test valid group list | ||
| self.assertIsNone(ctx.set_groups('P-256:X25519')) | ||
|
|
||
| # Test invalid group list | ||
| self.assertRaises(ssl.SSLError, ctx.set_groups, 'P-256:xxx') |
There was a problem hiding this comment.
| def test_set_groups(self): | |
| ctx = ssl.create_default_context() | |
| # Test valid group list | |
| self.assertIsNone(ctx.set_groups('P-256:X25519')) | |
| # Test invalid group list | |
| self.assertRaises(ssl.SSLError, ctx.set_groups, 'P-256:xxx') | |
| def test_set_groups(self): | |
| ctx = ssl.create_default_context() | |
| self.assertIsNone(ctx.set_groups('P-256:X25519')) | |
| self.assertRaises(ssl.SSLError, ctx.set_groups, 'P-256:xxx') |
We can be less verbose.
| if (group_name == NULL) { | ||
| Py_RETURN_NONE; | ||
| } |
There was a problem hiding this comment.
According to the docs, group_name may return NULL in case of an error. Maybe check that there isn't an error?
There was a problem hiding this comment.
Note 2: the docs only mentions the case when the TLS session wasn't established yet, but to be on the safe side, I'd prefer checking if there's an error on OpenSSL's side.
There was a problem hiding this comment.
Are you suggesting adding a unit test here which verifies that OpenSSL returning NULL will hit this code and return None, or are you thinking this should return an error instead?
Right now, this API matches the behavior of SSLSocket.cipher(), returning None when a TLS session is not established (either because of an error or because the application queries this information too early, before the handshake is performed). If there is an error during the handshake, I would expect that to have already raised an error to the calling code. This check is just to handle if the calling code decides to try and query the group despite the incomplete handshake.
There was a problem hiding this comment.
Ah no, I'm just telling to check that we don't have an SSL error set before returning None. If there is an SSL error, we can raise an exception. Otherwise, you can return None as you did.
There was a problem hiding this comment.
Looking at the documentation, I can see where you'd think SSL_get0_group_name() might set an SSL error before returning NULL. Looking at the implementation, though, the functions called by SSL_get0_group_name() return NULL without actually ever setting an error themselves. They just check if other state is NULL or not, and return the name if one is available. I think the error referred to in the docs would have been returned in a prior call, such as when initiating the handshake.
I could put in an explicit call to ERR_get_error here, but I can't find any other code in this module which looks at that to decide whether to return an error. There's always a check for a special error return value in the function before looking at any SSL error information.
In this case, since NULL is returned without setting an SSL error, I would recommend we treat all NULL values as indicating the handshake didn't complete or wasn't attempted yet, and always return None for both cipher and group (and eventually sigalgs when that support is added).
There was a problem hiding this comment.
In this case, since NULL is returned without setting an SSL error
Ah, so it would be hard to detect it. Ok, we can return None, and anyway, if there's an issue, subsequent calls to the interface will fail I think.
I would recommend we treat all NULL values as indicating the handshake didn't complete or wasn't attempted yet,
That could be more useful for the user. Can you make some test for that (namely, you set a group, and try to get its name even though the session doesn't exist and see that it returns None). We should also document that it returns None if no session exists yet.
There was a problem hiding this comment.
The doc about returning None is already in place:
.. method:: SSLSocket.group()
Return the group used for doing key agreement on this connection. If no
connection has been established, returns ``None``.
.. versionadded:: nextThis is very similar to the docs for SSLSocket.cipher().
I should be able to come up with a unit test which creates an SSLSocket without connecting it and then tries to query the group - working on that now.
There was a problem hiding this comment.
Trying to get an SSLSocket created without an underlying connection is actually somewhat messy. I think a more realistic case would actually be to try and query the cipher & group after a handshake failure, like no matching ciphers between the client and server. There's already a test for that, and it's straightforward to add in checks that both cipher() and group() return None in that case. Checking that in momentarily.
This is an initial implementation of the feature proposed in issue #136306.
📚 Documentation preview 📚: https://cpython-previews--136307.org.readthedocs.build/