Opened 2 years ago
Last modified 4 months ago
#30556 new defect
Meta-ticket: SSL issues
Reported by: | slelievre | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.7 |
Component: | packages: standard | Keywords: | openssl, python |
Cc: | gh-mwageringel, mkoeppe, slelievre, was, dimpase | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #30557, #30560, #30383 | Stopgaps: |
Description (last modified by )
Building Sage with its own Python 3 can easily produce
a Python 3 missing its _ssl
module.
Python's _ssl
module is only built if OpenSSL (with headers)
is available, either via the system or via the openssl
spkg.
The result is reduced functionality:
- it hinders installing pip packages
- it hinders working with Jupyter
See also:
- many reports on Ask Sage and sage-support
- sage-release: cannot start Jupyter due to missing _ssl module
Tickets:
- #30557: Add spkg-configure.m4 for openssl
- #31094 openssl: Make build more robust by clearing some environment variables
- #30674: Patch tornado so it can be imported in Python without an ssl module
- #29555: Upgrade: OpenSSL 3.0, make it a standard package
- #29291: Check for more strictly required extension module "ssl" in Python build
- #24107: Inclusion of OpenSSL, stage 1
- #32311: Upgrade: OpenSSL 3.0.0.beta2
- #23893: make openssl a dependency for pip packages
- #22620: Document that SSL is optional
- #17690: A SSL section in the "install from source code" documentation
- #30950: sage-download-file: Proper initialization of SSL certificates
- #32374 Remove pip package
pyopenssl
References:
- https://www.python.org/dev/peps/pep-0644/ - Require OpenSSL 1.1 or newer
Change History (29)
comment:1 Changed 2 years ago by
- Dependencies set to #30557
comment:2 Changed 2 years ago by
- Description modified (diff)
comment:3 follow-up: ↓ 11 Changed 2 years ago by
- Description modified (diff)
- Summary changed from Build Python's ssl module to Build Python's ssl module or protest loudly
comment:4 follow-up: ↓ 5 Changed 2 years ago by
- Description modified (diff)
Even when configuring with --enable-openssl
,
I think openssl
may be built after python3
.
So currently one really has to separate into two steps:
$ make openssl $ make
or later have to repair with
$ sage -i openssl $ sage -f python3
comment:5 in reply to: ↑ 4 Changed 2 years ago by
Replying to slelievre:
Even when configuring with
--enable-openssl
, I thinkopenssl
may be built afterpython3
.
Yes, this is a problem. We do not have a mechanism to order "optional dependencies".
comment:6 Changed 2 years ago by
See #21700 - Packages with "optional" dependencies (a package manager's suggested/recommended packages)
comment:7 follow-up: ↓ 8 Changed 2 years ago by
We could add openssl
to TOOLCHAIN_DEPS as a workaround. Then it would be built at the very beginning.
comment:8 in reply to: ↑ 7 Changed 2 years ago by
comment:9 Changed 2 years ago by
- Dependencies changed from #30557 to #30557, #30560
comment:10 Changed 2 years ago by
- Dependencies changed from #30557, #30560 to #30557, #30560, #30383
If we are building python3
and openssl
is not available as a system package and not enabled, then we should disable the packages that depend on openssl -- Jupyter notebook (and dependencies) and all source=pip
packages. The latter are already optional packages by definition. But we would need give the Jupyter notebook packages a new package type: optional-enabled-by-default
(#30383).
comment:11 in reply to: ↑ 3 Changed 2 years ago by
Replying to slelievre:
Could there be a
configure
flag to explicitly require building python3 without its_ssl
module?
This would effectively make openssl a standard package. I don't think we should do that because of license reasons. Warnings about the disabled features are better.
comment:12 Changed 2 years ago by
I would like to mention that the SSL requirement for Jupyter seems to be a recent change – presumably the upgrade of tornado or other Jupyter packages in 9.2.beta9. This was not clear to me and part of why I was confused on the mailing list. I was actually using the Python 3 SPKG without SSL all along and until now I have not experienced any problem with this. In particular, this is not related to the Python 3.8 upgrade as I had assumed at first.
comment:13 follow-up: ↓ 14 Changed 2 years ago by
This might be worth investigating more.
comment:14 in reply to: ↑ 13 ; follow-up: ↓ 15 Changed 23 months ago by
Replying to mkoeppe:
This might be worth investigating more.
As suspected, this is a consequence of the Jupyter upgrades in #26919, merged in 9.2.beta9. The dependency on ssl
in tornado/httpserver.py
was added in this commit.
comment:15 in reply to: ↑ 14 Changed 23 months ago by
Replying to gh-mwageringel:
Replying to mkoeppe:
This might be worth investigating more.
As suspected, this is a consequence of the Jupyter upgrades in #26919, merged in 9.2.beta9. The dependency on
ssl
intornado/httpserver.py
was added in this commit.
Looks like this would be easy to patch out.
comment:17 Changed 23 months ago by
- Milestone changed from sage-9.2 to sage-9.3
comment:18 Changed 20 months ago by
Building a Python without ssl module is happening on ubuntu-trusty
(https://github.com/sagemath/sage/runs/1553902103) in 9.3.beta4
Our configure finds system openssl; system python3 is too old, so python 3.8 is built from source; but apparently it fails to build the ssl module. (This should be investigated!)
The symptom is that pytest
(a pip package that is a check-only dependency of some standard package) fails to install:
[pytest] installing. Log file: /sage/logs/pkgs/pytest.log [pytest] error installing, exit status 1. End of log file: [pytest] WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. [pytest] WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
It is likely that this problem has actually been around for a while and was only masked by the faulty SAGE_CHECK logic fixed in #31020.
comment:19 Changed 20 months ago by
Same in ubuntu-xenial-standard
.
Also in various -minimal
builds, of course, where the SSL headers are not available. For example ubuntu-hirsute-minimal
(https://github.com/sagemath/sage/runs/1553902375)
comment:20 Changed 20 months ago by
#31062 "tox / GH Actions: Disable testsuites of packages depending on pip packages (pytest, ...) if there is no ssl" proposes a workaround
comment:21 Changed 20 months ago by
- Cc was dimpase added
- Description modified (diff)
- Summary changed from Build Python's ssl module or protest loudly to Meta-ticket: SSL issues
comment:22 Changed 20 months ago by
- Description modified (diff)
comment:23 Changed 19 months ago by
- Description modified (diff)
comment:24 Changed 17 months ago by
- Milestone changed from sage-9.3 to sage-9.4
- Priority changed from critical to major
Moving it to 9.4; we seem to have an OK solution for now
comment:25 Changed 13 months ago by
- Milestone changed from sage-9.4 to sage-9.5
comment:26 Changed 12 months ago by
- Description modified (diff)
comment:27 Changed 12 months ago by
- Description modified (diff)
comment:28 Changed 7 months ago by
- Milestone changed from sage-9.5 to sage-9.6
comment:29 Changed 4 months ago by
- Milestone changed from sage-9.6 to sage-9.7
Many builds of the python3 spkg without its
_ssl
module likely result from an oversight.Could there be a
configure
flag to explicitly require building python3 without its_ssl
module?Without that flag, consider openssl a dependency of python3, and let the build fail if it's not there.