#23615 closed enhancement (fixed)
Update pip to 9.0.1
Reported by: | mderickx | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-8.1 |
Component: | packages: standard | Keywords: | |
Cc: | Merged in: | ||
Authors: | Maarten Derickx | Reviewers: | Jeroen Demeyer |
Report Upstream: | N/A | Work issues: | |
Branch: | b6b9954 (Commits, GitHub, GitLab) | Commit: | |
Dependencies: | Stopgaps: |
Description (last modified by )
At #20913 pip was patched so that it works without ssl support, and this patch was also submitted upstream. Upstream has since merged the patch at https://github.com/pypa/pip/issues/1165 into 9.0.1. So it makes sense to upgrade pip to this new version in order to have an unpatched pip in sage.
The pip tarbal can be found at https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9
Change History (25)
comment:1 Changed 5 years ago by
Description: | modified (diff) |
---|
comment:2 Changed 5 years ago by
Branch: | → u/mderickx/23615 |
---|---|
Commit: | → 84ecb824e35535288dc980fa94cff9a0fec31190 |
comment:3 Changed 5 years ago by
Commit: | 84ecb824e35535288dc980fa94cff9a0fec31190 → a8d1697b4b56dbe20f8332bc1590bc86ba48d815 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
a8d1697 | Removed use of decrepated pip list format: trac 23615
|
comment:4 Changed 5 years ago by
Status: | new → needs_review |
---|
comment:5 Changed 5 years ago by
Status: | needs_review → needs_work |
---|
This looks very scary: eval(stdout)
. If you want to parse JSON, I would recommend the json module.
comment:6 Changed 5 years ago by
Commit: | a8d1697b4b56dbe20f8332bc1590bc86ba48d815 → b6b9954bb690cad965f4ffa135cd8c46b5d8cd3f |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
b6b9954 | use json for parsing json instead of eval: trac 23615
|
comment:7 Changed 5 years ago by
Status: | needs_work → needs_review |
---|
Yeah, I shouldn't have written that. Changed it into json.loads(stdout)
.
comment:8 follow-up: 9 Changed 5 years ago by
Builds fine and passes tests for me on OS X. Note that on this machine, the python2
log says that ssl is not built:
Python build finished, but the necessary bits to build these modules were not found: _bsddb _ssl dl gdbm imageop linuxaudiodev ossaudiodev spwd sunaudiodev
and import ssl
raises an ImportError
. So it's good that this new pip
works.
comment:9 follow-up: 10 Changed 5 years ago by
Replying to jhpalmieri:
Builds fine and passes tests for me on OS X.
Build from scratch or just a regular upgrade? I'm going to test a build from scratch on Linux.
comment:10 Changed 5 years ago by
Replying to jdemeyer:
Replying to jhpalmieri:
Builds fine and passes tests for me on OS X.
Build from scratch or just a regular upgrade? I'm going to test a build from scratch on Linux.
Build from scratch.
comment:11 Changed 5 years ago by
Reviewers: | → Jeroen Demeyer |
---|---|
Status: | needs_review → positive_review |
comment:12 Changed 5 years ago by
Branch: | u/mderickx/23615 → b6b9954bb690cad965f4ffa135cd8c46b5d8cd3f |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
comment:13 Changed 5 years ago by
Commit: | b6b9954bb690cad965f4ffa135cd8c46b5d8cd3f |
---|
It seems that this breaks something with python3... :(
chapoton@icj-laptop:~/sage3$ ./sage -br cd . && export \ SAGE_ROOT=/doesnotexist \ SAGE_SRC=/doesnotexist \ SAGE_SRC_ROOT=/doesnotexist \ SAGE_DOC_SRC=/doesnotexist \ SAGE_BUILD_DIR=/doesnotexist \ SAGE_PKGS=/home/chapoton/sage3/build/pkgs \ SAGE_CYTHONIZED=/home/chapoton/sage3/src/build/cythonized \ && sage-python23 -u setup.py --no-user-cfg build install Usage: pip list [options] no such option: --format ************************************************************************ Traceback (most recent call last): File "setup.py", line 69, in <module> from module_list import ext_modules, library_order, aliases File "/home/chapoton/sage3/src/module_list.py", line 166, in <module> from sage_setup.optional_extension import OptionalExtension File "/home/chapoton/sage3/src/sage_setup/optional_extension.py", line 24, in <module> all_packages = list_packages(local=True) File "/home/chapoton/sage3/src/sage/misc/package.py", line 226, in list_packages installed = installed_packages(exclude_pip) File "/home/chapoton/sage3/src/sage/misc/package.py", line 286, in installed_packages installed.update(pip_installed_packages()) File "/home/chapoton/sage3/src/sage/misc/package.py", line 148, in pip_installed_packages return {package['name'].lower():package['version'] for package in json.loads(stdout)} File "/home/chapoton/sage3/local/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/home/chapoton/sage3/local/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/home/chapoton/sage3/local/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ************************************************************************ Error building the Sage library ************************************************************************ Makefile:34 : la recette pour la cible « sage » a échouée make: *** [sage] Erreur 1
comment:15 Changed 5 years ago by
Apparently, this really did break the python3 build. I am not amused.
comment:16 Changed 5 years ago by
Hi Chapoton,
I would find it weird if this really did break the python3 build. The error message you posted is one that one would get if the installed pip is not actually 9.0.1:
For example on my machine with sage before this was merged
Maartens-MacBook-Pro:sagedev mderickx$ sage -pip -V pip 8.1.2 from /Applications/sage/local/lib/python2.7/site-packages (python 2.7) Maartens-MacBook-Pro:sagedev mderickx$ sage -pip list --format json Usage: pip list [options] no such option: --format
and after it is merged
Maartens-MacBook-Pro:sagedev mderickx$ ./sage -pip -V pip 9.0.1 from /Applications/sagedev/local/lib/python2.7/site-packages (python 2.7) Maartens-MacBook-Pro:sagedev mderickx$ ./sage -pip list --format json [{"version": "0.7.8", "name": "alabaster"}, ..., {"version": "4.2.0", "name": "zope.interface"}]
Could you give me the output of sage -pip -V
and ls $SAGE_ROOT/upstream | grep pip
?
comment:17 Changed 5 years ago by
A quick solution would be:
sage -pip uninstall pip sage -i pip
This avoids doing:
make distclean make install
comment:18 Changed 5 years ago by
chapoton@icj-laptop:~/sage3$ ./sage -pip -V pip 9.0.1 from /home/chapoton/sage3/local/lib/python2.7/site-packages (python 2.7) chapoton@icj-laptop:~/sage3$ ls upstream/pip* upstream/pip-8.1.2.tar.gz upstream/pip-9.0.1.tar.gz chapoton@icj-laptop:~/sage3$ ./sage -pip list --format json [{"version": "1.0.0", "name": "cypari2"}, {"version": "1.6.5", "name": "cysignals"}, {"version": "0.26", "name": "Cython"}, {"version": "9.0.1", "name": "pip"}, {"version": "33.1.1", "name": "setuptools"}]
so maybe this is a matter of pip versus pip3 ?
comment:19 Changed 5 years ago by
Weird, then the only possibility is that sage is picking up some pip from outside its installation.
Could you add the lines
proc = subprocess.Popen(["pip", "-V"], stdout=subprocess.PIPE) stdout = str(proc.communicate()[0]) print(stdout)
before the lines:
proc = subprocess.Popen(["pip", "list", "--no-index", "--format", "json"], stdout=subprocess.PIPE) stdout = str(proc.communicate()[0])
in src/sage/misc/package.py
in order to see which pip is being picked up at the point where the build fails?
Alternatively could you describe how to reproduce this error so I can look at it myself?
comment:20 Changed 5 years ago by
The result of the added print is
b'pip 9.0.1 from /home/chapoton/sage3/local/lib/python2.7/site-packages (python 2.7)\n'
To reproduce, I think this is enough:
- git clone a new sage - export SAGE_PYTHON3=yes - make build
comment:21 Changed 5 years ago by
Ah I see the problem it is a unicode thing. If you do:
stdout = proc.communicate()[0].decode()
then everything should work. I will create a new ticket with a fix.
comment:23 Changed 4 years ago by
I am hurt by all this again when going from sage 8.4.beta0 to 8.4.beta1.
comment:25 Changed 4 years ago by
I got build failures for sagelib on the very same line
return {package['name'].lower():package['version'] for package in json.loads(stdout)}
but I have now realized that this is probably due to the existence of a personal "pip.conf".
New commits:
Upgrade pip to 9.0.1 trac #23615