Opened 2 years ago
Last modified 4 months ago
#30556 new defect
Meta-ticket: SSL issues
Reported by: | Samuel Lelièvre | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.8 |
Component: | packages: standard | Keywords: | openssl, python |
Cc: | Markus Wageringel, Matthias Köppe, Samuel Lelièvre, William Stein, Dima Pasechnik | Merged in: | |
Authors: | Reviewers: | ||
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #30557, #30560, #30383 | Stopgaps: |
Description (last modified by )
Building Sage with its own Python 3 can easily produce
a Python 3 missing its _ssl
module.
Python's _ssl
module is only built if OpenSSL (with headers)
is available, either via the system or via the openssl
spkg.
The result is reduced functionality:
- it hinders installing pip packages
- it hinders working with Jupyter
See also:
- many reports on Ask Sage and sage-support
- sage-release: cannot start Jupyter due to missing _ssl module
Tickets:
- #30557: Add spkg-configure.m4 for openssl
- #31094 openssl: Make build more robust by clearing some environment variables
- #30674: Patch tornado so it can be imported in Python without an ssl module
- #29555: Upgrade: OpenSSL 3.0, make it a standard package
- #29291: Check for more strictly required extension module "ssl" in Python build
- #24107: Inclusion of OpenSSL, stage 1
- #32311: Upgrade: OpenSSL 3.0.0.beta2
- #23893: make openssl a dependency for pip packages
- #22620: Document that SSL is optional
- #17690: A SSL section in the "install from source code" documentation
- #30950: sage-download-file: Proper initialization of SSL certificates
- #32374 Remove pip package
pyopenssl
References:
- https://www.python.org/dev/peps/pep-0644/ - Require OpenSSL 1.1 or newer
Change History (30)
comment:1 Changed 2 years ago by
Dependencies: | → #30557 |
---|
comment:2 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:3 follow-up: 11 Changed 2 years ago by
Description: | modified (diff) |
---|---|
Summary: | Build Python's ssl module → Build Python's ssl module or protest loudly |
comment:4 follow-up: 5 Changed 2 years ago by
Description: | modified (diff) |
---|
Even when configuring with --enable-openssl
,
I think openssl
may be built after python3
.
So currently one really has to separate into two steps:
$ make openssl $ make
or later have to repair with
$ sage -i openssl $ sage -f python3
comment:5 Changed 2 years ago by
Replying to slelievre:
Even when configuring with
--enable-openssl
, I thinkopenssl
may be built afterpython3
.
Yes, this is a problem. We do not have a mechanism to order "optional dependencies".
comment:6 Changed 2 years ago by
See #21700 - Packages with "optional" dependencies (a package manager's suggested/recommended packages)
comment:7 follow-up: 8 Changed 2 years ago by
We could add openssl
to TOOLCHAIN_DEPS as a workaround. Then it would be built at the very beginning.
comment:8 Changed 2 years ago by
comment:9 Changed 2 years ago by
Dependencies: | #30557 → #30557, #30560 |
---|
comment:10 Changed 2 years ago by
Dependencies: | #30557, #30560 → #30557, #30560, #30383 |
---|
If we are building python3
and openssl
is not available as a system package and not enabled, then we should disable the packages that depend on openssl -- Jupyter notebook (and dependencies) and all source=pip
packages. The latter are already optional packages by definition. But we would need give the Jupyter notebook packages a new package type: optional-enabled-by-default
(#30383).
comment:11 Changed 2 years ago by
Replying to slelievre:
Could there be a
configure
flag to explicitly require building python3 without its_ssl
module?
This would effectively make openssl a standard package. I don't think we should do that because of license reasons. Warnings about the disabled features are better.
comment:12 Changed 2 years ago by
I would like to mention that the SSL requirement for Jupyter seems to be a recent change – presumably the upgrade of tornado or other Jupyter packages in 9.2.beta9. This was not clear to me and part of why I was confused on the mailing list. I was actually using the Python 3 SPKG without SSL all along and until now I have not experienced any problem with this. In particular, this is not related to the Python 3.8 upgrade as I had assumed at first.
comment:14 follow-up: 15 Changed 2 years ago by
Replying to mkoeppe:
This might be worth investigating more.
As suspected, this is a consequence of the Jupyter upgrades in #26919, merged in 9.2.beta9. The dependency on ssl
in tornado/httpserver.py
was added in this commit.
comment:15 Changed 2 years ago by
Replying to gh-mwageringel:
Replying to mkoeppe:
This might be worth investigating more.
As suspected, this is a consequence of the Jupyter upgrades in #26919, merged in 9.2.beta9. The dependency on
ssl
intornado/httpserver.py
was added in this commit.
Looks like this would be easy to patch out.
comment:17 Changed 2 years ago by
Milestone: | sage-9.2 → sage-9.3 |
---|
comment:18 Changed 2 years ago by
Building a Python without ssl module is happening on ubuntu-trusty
(https://github.com/sagemath/sage/runs/1553902103) in 9.3.beta4
Our configure finds system openssl; system python3 is too old, so python 3.8 is built from source; but apparently it fails to build the ssl module. (This should be investigated!)
The symptom is that pytest
(a pip package that is a check-only dependency of some standard package) fails to install:
[pytest] installing. Log file: /sage/logs/pkgs/pytest.log [pytest] error installing, exit status 1. End of log file: [pytest] WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. [pytest] WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
It is likely that this problem has actually been around for a while and was only masked by the faulty SAGE_CHECK logic fixed in #31020.
comment:19 Changed 2 years ago by
Same in ubuntu-xenial-standard
.
Also in various -minimal
builds, of course, where the SSL headers are not available. For example ubuntu-hirsute-minimal
(https://github.com/sagemath/sage/runs/1553902375)
comment:20 Changed 2 years ago by
#31062 "tox / GH Actions: Disable testsuites of packages depending on pip packages (pytest, ...) if there is no ssl" proposes a workaround
comment:21 Changed 2 years ago by
Cc: | William Stein Dima Pasechnik added |
---|---|
Description: | modified (diff) |
Summary: | Build Python's ssl module or protest loudly → Meta-ticket: SSL issues |
comment:22 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:23 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:24 Changed 22 months ago by
Milestone: | sage-9.3 → sage-9.4 |
---|---|
Priority: | critical → major |
Moving it to 9.4; we seem to have an OK solution for now
comment:25 Changed 19 months ago by
Milestone: | sage-9.4 → sage-9.5 |
---|
comment:26 Changed 18 months ago by
Description: | modified (diff) |
---|
comment:27 Changed 18 months ago by
Description: | modified (diff) |
---|
comment:28 Changed 13 months ago by
Milestone: | sage-9.5 → sage-9.6 |
---|
comment:29 Changed 9 months ago by
Milestone: | sage-9.6 → sage-9.7 |
---|
comment:30 Changed 4 months ago by
Milestone: | sage-9.7 → sage-9.8 |
---|
Many builds of the python3 spkg without its
_ssl
module likely result from an oversight.Could there be a
configure
flag to explicitly require building python3 without its_ssl
module?Without that flag, consider openssl a dependency of python3, and let the build fail if it's not there.