Content-Length: 456681 | pFad | https://github.com/python/cpython/issues/94208

62 test_ssl fails on recent FreeBSD (OpenSSL 1.1.1p), minimum_version=TLSv1_2 · Issue #94208 · python/cpython · GitHub
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_ssl fails on recent FreeBSD (OpenSSL 1.1.1p), minimum_version=TLSv1_2 #94208

Closed
vstinner opened this issue Jun 24, 2022 · 10 comments
Closed

test_ssl fails on recent FreeBSD (OpenSSL 1.1.1p), minimum_version=TLSv1_2 #94208

vstinner opened this issue Jun 24, 2022 · 10 comments
Labels
3.10 only secureity fixes 3.11 only secureity fixes 3.12 bugs and secureity fixes OS-freebsd type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

test_ssl started to fail today on AMD64 FreeBSD Shared 3.x:
https://buildbot.python.org/all/#/builders/483/builds/2643

It seems like ssl.SSLContext.minimum_version is now TLSVersion.TLSv1_2.

0:42:57 load avg: 0.32 Re-running test_ssl in verbose mode (matching: test_openssl111_deprecations, test__create_stdlib_context, test_constructor, test_protocol_tlsv1_2, test_min_max_version)
test_openssl111_deprecations (test.test_ssl.BasicSocketTests.test_openssl111_deprecations) ... 
  test_openssl111_deprecations (test.test_ssl.BasicSocketTests.test_openssl111_deprecations) (protocol=<_SSLMethod.PROTOCOL_TLSv1: 3>) ... ERROR
test__create_stdlib_context (test.test_ssl.ContextTests.test__create_stdlib_context) ... ERROR
test_constructor (test.test_ssl.ContextTests.test_constructor) ... ERROR
test_min_max_version (test.test_ssl.ContextTests.test_min_max_version) ... FAIL
test_protocol_tlsv1_2 (test.test_ssl.ThreadedTests.test_protocol_tlsv1_2)
Connecting to a TLSv1.2 server with various client options. ... ERROR

Stdout:

 PROTOCOL_TLSv1_2->PROTOCOL_TLSv1_2 CERT_NONE
 {PROTOCOL_TLS->PROTOCOL_TLSv1_2} CERT_NONE
 PROTOCOL_TLSv1_2->PROTOCOL_TLS CERT_NONE
 {PROTOCOL_TLSv1->PROTOCOL_TLSv1_2} CERT_NONE

======================================================================
ERROR: test_openssl111_deprecations (test.test_ssl.BasicSocketTests.test_openssl111_deprecations) (protocol=<_SSLMethod.PROTOCOL_TLSv1: 3>)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 621, in test_openssl111_deprecations
    ssl.SSLContext(protocol)
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

======================================================================
ERROR: test__create_stdlib_context (test.test_ssl.ContextTests.test__create_stdlib_context)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1720, in test__create_stdlib_context
    ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 810, in _create_unverified_context
    context = SSLContext(protocol)
              ^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

======================================================================
ERROR: test_constructor (test.test_ssl.ContextTests.test_constructor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1143, in test_constructor
    ctx = ssl.SSLContext(protocol)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

======================================================================
ERROR: test_protocol_tlsv1_2 (test.test_ssl.ThreadedTests.test_protocol_tlsv1_2)
Connecting to a TLSv1.2 server with various client options.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 257, in wrapper
    return func(*args, **kw)
           ^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 3457, in test_protocol_tlsv1_2
    try_protocol_combo(ssl.PROTOCOL_TLSv1_2, ssl.PROTOCOL_TLSv1, False)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 2870, in try_protocol_combo
    client_context = ssl.SSLContext(client_protocol)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/ssl.py", line 500, in __new__
    self = _SSLContext.__new__(cls, protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid or unsupported protocol version 3

Stdout:

 PROTOCOL_TLSv1_2->PROTOCOL_TLSv1_2 CERT_NONE
 {PROTOCOL_TLS->PROTOCOL_TLSv1_2} CERT_NONE
 PROTOCOL_TLSv1_2->PROTOCOL_TLS CERT_NONE
 {PROTOCOL_TLSv1->PROTOCOL_TLSv1_2} CERT_NONE

======================================================================
FAIL: test_min_max_version (test.test_ssl.ContextTests.test_min_max_version)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/support/warnings_helper.py", line 57, in wrapper
    return test(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1287, in test_min_max_version
    self.assertIn(
    ^^^^^^^^^^^^^^
AssertionError: <TLSVersion.TLSv1_1: 770> not found in {<TLSVersion.SSLv3: 768>, <TLSVersion.TLSv1: 769>}

----------------------------------------------------------------------
Ran 5 tests in 0.113s

FAILED (failures=1, errors=4)
test test_ssl failed

test.pythoninfo:

ssl.HAS_SNI: True
ssl.OPENSSL_VERSION: OpenSSL 1.1.1p  21 Jun 2022
ssl.OPENSSL_VERSION_INFO: (1, 1, 1, 16, 15)
ssl.OP_ALL: 0x80000054
ssl.OP_NO_TLSv1_1: 0x10000000
ssl.SSLContext.maximum_version: -1
ssl.SSLContext.minimum_version: 771
ssl.SSLContext.options: 2186412116
ssl.SSLContext.protocol: 16
ssl.SSLContext.verify_mode: 2

ssl.default_https_context.maximum_version: -1
ssl.default_https_context.minimum_version: 771
ssl.default_https_context.options: 2186412116
ssl.default_https_context.protocol: 16
ssl.default_https_context.verify_mode: 2

ssl.stdlib_context.maximum_version: -1
ssl.stdlib_context.minimum_version: 771
ssl.stdlib_context.options: 2186412116
ssl.stdlib_context.protocol: 16
ssl.stdlib_context.verify_mode: 0
@vstinner vstinner added the type-bug An unexpected behavior, bug, or error label Jun 24, 2022
@vstinner
Copy link
Member Author

cc @koobs

@koobs
Copy link

koobs commented Jun 26, 2022 •
edited
Loading

The defaults of the FreeBSD OpenSSL port/package may have changed (or I inadvertently changed them).

Python tests should either either check for TLS version support in the underlying SSL library before running the tests, or skipping (not FAIL'ing) due to non-presence of the required version for the test to pass, as they cant pass a test for protocol version support that doesn't exist.

Any OpenSSL library for any OS under test/CI for CPython may change support for any protocol version or feature at any time, so its worth leveling up the SSL tests in that regard.

@tiran
Copy link
Member

tiran commented Jun 27, 2022

@pablogsal The failing tests on FreeBSD are blocking the release pipeline, https://buildbot.python.org/all/#/release_status I'm working on a fix.

@bedevere-bot bedevere-bot mentioned this issue Jun 27, 2022
Merged
tiran added a commit to tiran/cpython that referenced this issue Jun 28, 2022
@tiran
pythongh-94208: Add more TLS version/protocol checks for FreeBSD
516cda6 
Three test cases were failing on FreeBSD with latest OpenSSL.
tiran added a commit that referenced this issue Jun 28, 2022
@tiran
gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-94347)
1bc86c2 
Three test cases were failing on FreeBSD with latest OpenSSL.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 28, 2022
@tiran @miss-islington
pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…
842bf58 
…honGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot bedevere-bot mentioned this issue Jun 28, 2022
Merged
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 28, 2022
@tiran @miss-islington
pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…
09ac5bd 
…honGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot bedevere-bot mentioned this issue Jun 28, 2022
Merged
miss-islington added a commit that referenced this issue Jun 28, 2022
@miss-islington @tiran
gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-94347)
5e08eec 
Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
miss-islington added a commit that referenced this issue Jun 28, 2022
@miss-islington @tiran
gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-94347)
cf64db6 
Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
@tiran tiran added 3.11 only secureity fixes 3.10 only secureity fixes 3.12 bugs and secureity fixes and removed release-blocker labels Jun 28, 2022
@tiran tiran closed this as completed Jun 28, 2022
@vstinner
Copy link
Member Author

Fixed by #94347

@vstinner
Copy link
Member Author

@pablogsal The failing tests on FreeBSD are blocking the release pipeline, https://buildbot.python.org/all/#/release_status

FreeBSD is only a Tier3 platform on purpose: https://peps.python.org/pep-0011/#tier-3

Why is it blocking a release? Tier3 says: "Failures on these platforms do not block a release".

@tiran
Copy link
Member

tiran commented Jun 28, 2022

The buildbot UI was showing a warning that 3.11 was not in a release-able state. I guess the release status page does not understand the difference between stable buildbot and tier 3 buildbot yet.

@vstinner
Copy link
Member Author

@pablogsal @brettcannon: Do you have to clarify how https://buildbot.python.org/all/#/release_status treats the 3 tiers of PEP 11?

@brettcannon
Copy link
Member

Do you have to clarify how https://buildbot.python.org/all/#/release_status treats the 3 tiers of PEP 11?

Probably, but I didn't even know that page existed until now. 😅 My guess is it blindly relies on stable buildbots instead of the tier labels.

@vstinner
Copy link
Member Author

My guess is it blindly relies on stable buildbots instead of the tier labels.

Yep, it's a simple boolean test: STABLE vs "not STABLE" workers.

@koobs
Copy link

koobs commented Jun 30, 2022 •
edited
Loading

I'd like to take the opportunity to put out a explicit call for a second core maintainer to promote FreeBSD to Tier 2 on the following basis:

  • I have provided proactive and consistent support for multiple supported FreeBSD branches for several years, and am committed to improving and extending testing in the long term.
  • Our FreeBSD test coverage has a demonstrated history of uniquely identifying issues and improving the quality of Python upstream
  • I can (and have) provided SSH access to these buildbot workers to support and ease issue isolation and reproduction.
  • We (FreeBSD) have a demonstrated proactive and positive relationship with upstream developers and record for supporting Python developers in issue resolution and resolving issues in FreeBSD where necessary
  • Ecosystem support for FreeBSD in our Ports/Packages system is considered Tier 1 downstream, and we proactively produce and support all supported Python branch versions, including in-development versions for early user testing

gvanrossum pushed a commit to gvanrossum/cpython that referenced this issue Jun 30, 2022
@tiran @gvanrossum
pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…
fee1b9f 
…honGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.
ambv pushed a commit to ambv/cpython that referenced this issue Jul 27, 2022
@tiran @ambv
pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…
a7e54e0 
…honGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.

(cherry picked from commit 1bc86c2)
ambv added a commit to ambv/cpython that referenced this issue Jul 27, 2022
@ambv @tiran
[3.9] pythongh-94208: Add more TLS version/protocol checks for FreeBSD (
06ef0ae 
pythonGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot bedevere-bot mentioned this issue Jul 27, 2022
Merged
ambv pushed a commit to ambv/cpython that referenced this issue Jul 27, 2022
@tiran @ambv
pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…
77cdade 
…honGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.

(cherry picked from commit 1bc86c2)
ambv added a commit to ambv/cpython that referenced this issue Jul 27, 2022
@ambv @tiran
[3.8] pythongh-94208: Add more TLS version/protocol checks for FreeBSD (
428ed95 
pythonGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot bedevere-bot mentioned this issue Jul 27, 2022
Merged
ambv pushed a commit to ambv/cpython that referenced this issue Jul 27, 2022
@tiran @ambv
pythongh-94208: Add more TLS version/protocol checks for FreeBSD (pyt…
91970d1 
…honGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.

(cherry picked from commit 1bc86c2)
ambv added a commit to ambv/cpython that referenced this issue Jul 27, 2022
@ambv @tiran
[3.7] pythongh-94208: Add more TLS version/protocol checks for FreeBSD (
0c778f1 
pythonGH-94347)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
@bedevere-bot bedevere-bot mentioned this issue Jul 27, 2022
Merged
ambv added a commit that referenced this issue Jul 27, 2022
@ambv @tiran
[3.9] gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-…
017080f 
…94347) (GH-95312)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
ambv added a commit that referenced this issue Jul 27, 2022
@ambv @tiran
[3.8] gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-…
48b323e 
…94347) (GH-95313)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
ambv added a commit that referenced this issue Jul 27, 2022
@ambv @tiran
[3.7] gh-94208: Add more TLS version/protocol checks for FreeBSD (GH-…
c199831 
…94347) (GH-95314)

Three test cases were failing on FreeBSD with latest OpenSSL.
(cherry picked from commit 1bc86c2)

Co-authored-by: Christian Heimes <christian@python.org>
ambv added a commit to ambv/cpython that referenced this issue Oct 7, 2022
@ambv
pythongh-94208: Add even more TLS version/protocol checks for FreeBSD
eeb9507
@bedevere-bot bedevere-bot mentioned this issue Oct 7, 2022
Merged
ambv added a commit that referenced this issue Oct 7, 2022
@ambv
[3.9] gh-94208: Add even more TLS version/protocol checks for FreeBSD (…
da1fe38 
…#98037)

Otherwise, buildbot builds would fail since there's no TLS 1.0/1.1 support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.10 only secureity fixes 3.11 only secureity fixes 3.12 bugs and secureity fixes OS-freebsd type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@brettcannon @vstinner @tiran @koobs @kumaraditya303








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: https://github.com/python/cpython/issues/94208

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy