#29418 closed enhancement (fixed)
sage-download-file: Fix certificate problems with https downloads from upstream_url when sage-system-python is XCode's python3 on macOS
Reported by: | mkoeppe | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.1 |
Component: | build | Keywords: | |
Cc: | vbraun, dimpase, fbissey, jhpalmieri, vdelecroix, chapoton, gh-kliem | Merged in: | |
Authors: | Matthias Koeppe | Reviewers: | Jonathan Kliem |
Report Upstream: | N/A | Work issues: | |
Branch: | 90ea00b (Commits, GitHub, GitLab) | Commit: | |
Dependencies: | Stopgaps: |
Description (last modified by )
#26351 added an optional upstream_url
field to build/pkgs/*/checksums.ini
. It streamlines the procedure for testing upgrade tickets: Developers or automatic testing facilities can pass an extra flag -o
to sage-spkg
to allow downloading from upstream rather than from Sage mirrors (where the updated ticket will be made available later only).
Many upstream package URLs use the https
protocol - in contrast to the http
protocol used when downloading from the Sage mirrors. The downloading is done via build/bin/sage-download-file
, which uses the urllib
module. It supports the https protocol.
However, SSL certificate problems are common on test systems. For example, if one uses XCode's python3
as the system python, then urllib
does not automatically uses the standard system certificates. (This is apparently a known issue -- which is considered "wontfix" by Apple as reported here:
https://github.com/HandBrake/HandBrake/issues/2216#issuecomment-527114519)
We add an option --no-check-certificate
to sage-download-file
, disabling certificate checking (https://stackoverflow.com/questions/36600583/python-3-urllib-ignore-ssl-certificate-verification).
Developers can set this option using the environment variable SAGE_DOWNLOAD_FILE_OPTIONS when installing packages (either by make
or by using sage -i
).
We note that even with SSL certificates disabled, there is still cryptographic protection because of the checksums recorded in checksums.ini
.
Other possible workarounds considered:
- switching from using urllib directly to the requests library
- passing cafile=..., capath=... to urllib, perhaps coming from an environment variable
- using python instead of python3 on macOS as system python
As of #29090 (sage-system-python fixup) prefers /usr/bin/python3
over /usr/bin/python
, leading to:
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake self._sslobj.do_handshake() OSError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)
Change History (38)
comment:1 Changed 3 years ago by
Description: | modified (diff) |
---|
comment:2 Changed 3 years ago by
comment:3 Changed 3 years ago by
comment:4 Changed 3 years ago by
Description: | modified (diff) |
---|---|
Summary: | sage-download-file: Add option --no-check-certificate → sage-download-file: Fix certificate problems with https downloads on macOS python3 |
comment:5 Changed 3 years ago by
Branch: | → u/mkoeppe/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3 |
---|
comment:6 follow-up: 8 Changed 3 years ago by
Commit: | → bd24641560065f4ea4cd78139c11b324b98303b6 |
---|
that's why I suggested not to bother with MacOS/Xcode's python(3). Can't one get Python3 from python.org in MacOS by running a script?
New commits:
bd24641 | build/bin/sage-download-file: Add option --no-check-certificate
|
comment:7 Changed 3 years ago by
Authors: | → Matthias Koeppe |
---|---|
Branch: | u/mkoeppe/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3 |
Cc: | dimpase fbissey jhpalmieri added |
Commit: | bd24641560065f4ea4cd78139c11b324b98303b6 |
Description: | modified (diff) |
Status: | new → needs_review |
comment:8 follow-up: 12 Changed 3 years ago by
Replying to dimpase:
that's why I suggested not to bother with MacOS/Xcode's python(3). Can't one get Python3 from python.org in MacOS by running a script?
Please... the idea is to make Sage installation easier, not harder.
comment:9 Changed 3 years ago by
One often runs into these certificate errors also with random Linux images when I run them on Docker. So this workaround is valuable for the automatic testing in any case.
comment:10 Changed 3 years ago by
Branch: | → u/mkoeppe/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3 |
---|
comment:11 Changed 3 years ago by
Commit: | → e143939a2d434211a7607c6f9830496bce34dacd |
---|
Testing this at https://github.com/mkoeppe/sage/actions/runs/65346154
New commits:
bd24641 | build/bin/sage-download-file: Add option --no-check-certificate
|
e143939 | build/bin/sage-spkg: Append to SAGE_DOWNLOAD_FILE_OPTIONS instead of overwriting it
|
comment:12 follow-up: 13 Changed 3 years ago by
Replying to mkoeppe:
Replying to dimpase:
that's why I suggested not to bother with MacOS/Xcode's python(3). Can't one get Python3 from python.org in MacOS by running a script?
Please... the idea is to make Sage installation easier, not harder.
well, it seems that using system's python would make a slighly cryptographically broken Sagemath, one way or another.
comment:13 Changed 3 years ago by
comment:14 Changed 3 years ago by
And, of course, the present ticket is about sage-system-python
, which is only used for bootstrapping and build. Unrelated to the use of a system python3 for venv (#27824), which has a completely separate test for what it accepts.
comment:16 Changed 3 years ago by
Summary: | sage-download-file: Fix certificate problems with https downloads on macOS python3 → sage-download-file: Fix certificate problems with https downloads when sage-system-python is XCode's python3 on macOS |
---|
comment:17 Changed 3 years ago by
comment:18 Changed 3 years ago by
Commit: | e143939a2d434211a7607c6f9830496bce34dacd → 90ea00b82c4d1ade6ba5c9bbecbc6388dbedefde |
---|
comment:20 Changed 3 years ago by
Cc: | vdelecroix added |
---|
comment:21 Changed 3 years ago by
Cc: | chapoton added |
---|
comment:22 Changed 3 years ago by
Cc: | gh-kliem added |
---|
comment:23 Changed 3 years ago by
Okay, I'm really confused, or my Sage installation is broken, or both. If I use ./sage -i beautifulsoup4
with a build using the system Python on OS X, it fails, but not because of the issues here. Instead, I see this:
make[2]: *** No rule to make target `/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/SYSTEM/sage-9.1.beta9/local/var/lib/sage/installed/beautifulsoup4-none', needed by `all-sage'. Stop.
If instead I run make beautifulsoup4
, it succeeds. When I ran ./sage -i tornado
it didn't seem to even try to install it, but ./sage -f tornado
worked.
So: why does ./sage -i ...
not work? And what is the point of this ticket: what problem does it solve?
comment:26 Changed 3 years ago by
Description: | modified (diff) |
---|
comment:27 Changed 3 years ago by
If I run bootstrap
, then running ./sage -i benzene
acts like ./sage -i tornado
: it doesn't install. There is a message
running ./configure '--enable-beautifulsoup4' '--enable-biopython' '--enable-benzene'
and then if I search my terminal window for benzene
, the only other hit is
benzene-20130630: does not support check for system package; optional, use "./configure --enable-benzene" to install
I don't think I tested #29113 well enough when I was reviewing it.
comment:28 follow-up: 31 Changed 3 years ago by
This fixes it for me:
-
src/bin/sage
diff --git a/src/bin/sage b/src/bin/sage index 10acddcd96..6ccff3789c 100755
a b if [ "$1" = '-i' ]; then 404 404 # 'CC=gcc -Wall' '--enable-e_antic' 405 405 CONFIG_CMD="./configure $(./config.status --config) $ENABLE_ARGS" 406 406 echo >&2 "running $CONFIG_CMD" 407 bash -c "$CONFIG_CMD" && $MAKE all-build407 bash -c "$CONFIG_CMD" && $MAKE $PACKAGES 408 408 else 409 409 echo "New packages may have been installed." 410 410 echo "Re-running configure and make in case any dependent packages need updating."
(or maybe it could be $MAKE all-build $PACKAGES
?)
By the way, I still don't understand the goal of this ticket.
comment:29 Changed 3 years ago by
Let's please take the issues with sage -i
to the new ticket #29481. Thanks for catching this.
comment:30 Changed 3 years ago by
Description: | modified (diff) |
---|---|
Summary: | sage-download-file: Fix certificate problems with https downloads when sage-system-python is XCode's python3 on macOS → sage-download-file: Fix certificate problems with https downloads from upstream_url when sage-system-python is XCode's python3 on macOS |
comment:31 follow-up: 32 Changed 3 years ago by
Replying to jhpalmieri:
By the way, I still don't understand the goal of this ticket.
Reworked the ticket description, please take a look
comment:32 Changed 3 years ago by
Replying to mkoeppe:
Replying to jhpalmieri:
By the way, I still don't understand the goal of this ticket.
Reworked the ticket description, please take a look
Sorry, I wasn't clear enough. What system configuration and commands lead to the problem in the description? I can't reproduce the problem, so I can't tell if the solution here works.
comment:33 follow-up: 34 Changed 3 years ago by
From what I understand this is a problem that only occurs when one has python3 from Xcode. This doesn't ship certify and never will (at least Apple doesn't have any intentions). So it is really difficult to establish an ssl connection with that.
Do you have successfully tested this ticket?
Do I understand correctly that the problem occurring could be theoretically fixed by manually downloading the correct packages into the upstream folder? (Yes this is not a good approach for testing environments.)
Also this flag would never be needed for a normal user as long as the sage mirrors don't use ssl?
comment:34 Changed 3 years ago by
Replying to gh-kliem:
From what I understand this is a problem that only occurs when one has python3 from Xcode. This doesn't ship certify and never will (at least Apple doesn't have any intentions). So it is really difficult to establish an ssl connection with that.
That's a correct description for macOS. But the problem also appears on Linux distributions if one does not install ca-certificates, or those are outdated.
Do you have successfully tested this ticket?
Yes, I have been using this as part of #29417 since end of March.
Do I understand correctly that the problem occurring could be theoretically fixed by manually downloading the correct packages into the upstream folder? (Yes this is not a good approach for testing environments.)
That's correct for the macOS tests using tox local
. For the Linux tests with tox docker
note that /upstream
is in .gitignore
(therefore in .dockerignore
) and is therefore not provided to the container.
Also this flag would never be needed for a normal user as long as the sage mirrors don't use ssl?
That's right, normal users would not use the upstream_url
at all; it is a feature for developers only.
comment:35 Changed 3 years ago by
Reviewers: | → Jonathan Kliem |
---|---|
Status: | needs_review → positive_review |
LGTM.
Btw, I have been using this ticket to test my tickets, e.g. here: https://github.com/kliem/sage-test-27122/actions/runs/72102779
comment:37 Changed 3 years ago by
Branch: | u/mkoeppe/sage_download_file__fix_certificate_problems_with_https_downloads_on_macos_python3 → 90ea00b82c4d1ade6ba5c9bbecbc6388dbedefde |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
comment:38 Changed 2 years ago by
Commit: | 90ea00b82c4d1ade6ba5c9bbecbc6388dbedefde |
---|
Follow up: #30950
https://stackoverflow.com/questions/36600583/python-3-urllib-ignore-ssl-certificate-verification