Opened 5 years ago
Closed 5 years ago
#23822 closed defect (fixed)
Python3 build error: unicode problem in pip_installed_packages()
Reported by: | mderickx | Owned by: | |
---|---|---|---|
Priority: | critical | Milestone: | sage-8.1 |
Component: | python3 | Keywords: | |
Cc: | Merged in: | ||
Authors: | Maarten Derickx | Reviewers: | Frédéric Chapoton |
Report Upstream: | N/A | Work issues: | |
Branch: | f3e5282 (Commits, GitHub, GitLab) | Commit: | f3e5282c30fe45aa34b611f052341fb546e8e43b |
Dependencies: | Stopgaps: |
Description
The python 3 build was broken by #23615 because of incorrect unicode usage causing the building of sage failing in the following way
chapoton@icj-laptop:~/sage3$ ./sage -br cd . && export \ SAGE_ROOT=/doesnotexist \ SAGE_SRC=/doesnotexist \ SAGE_SRC_ROOT=/doesnotexist \ SAGE_DOC_SRC=/doesnotexist \ SAGE_BUILD_DIR=/doesnotexist \ SAGE_PKGS=/home/chapoton/sage3/build/pkgs \ SAGE_CYTHONIZED=/home/chapoton/sage3/src/build/cythonized \ && sage-python23 -u setup.py --no-user-cfg build install ************************************************************************ Traceback (most recent call last): File "setup.py", line 69, in <module> from module_list import ext_modules, library_order, aliases File "/home/chapoton/sage3/src/module_list.py", line 166, in <module> from sage_setup.optional_extension import OptionalExtension File "/home/chapoton/sage3/src/sage_setup/optional_extension.py", line 24, in <module> all_packages = list_packages(local=True) File "/home/chapoton/sage3/src/sage/misc/package.py", line 226, in list_packages installed = installed_packages(exclude_pip) File "/home/chapoton/sage3/src/sage/misc/package.py", line 286, in installed_packages installed.update(pip_installed_packages()) File "/home/chapoton/sage3/src/sage/misc/package.py", line 148, in pip_installed_packages return {package['name'].lower():package['version'] for package in json.loads(stdout)} File "/home/chapoton/sage3/local/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/home/chapoton/sage3/local/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/home/chapoton/sage3/local/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ************************************************************************ Error building the Sage library ************************************************************************ Makefile:34 : la recette pour la cible « sage » a échouée make: *** [sage] Erreur 1
Change History (8)
comment:1 Changed 5 years ago by
Branch: | → u/mderickx/23822 |
---|---|
Commit: | → f3e5282c30fe45aa34b611f052341fb546e8e43b |
Status: | new → needs_review |
comment:2 Changed 5 years ago by
Authors: | → Maarten Derickx |
---|
comment:3 Changed 5 years ago by
Reviewers: | → Frédéric Chapoton |
---|---|
Status: | needs_review → positive_review |
ok, let it be
comment:4 Changed 5 years ago by
Do you need the .decode
, or can you just use
stdout = proc.communicate()[0]
?
comment:5 Changed 5 years ago by
In python 2 it is not needed but good praxis. In python 3 it is needed since stdout is a bytestring before decoding and the json library only takes unicode strings and not bytestrings as input.
comment:6 Changed 5 years ago by
When I was looking at #23876, I came across this bug and didn't use .decode
, and it seemed to work with Python 3. That is, before making any changes, Sage crashed on startup, complaining about this problem, but when I switched to stdout = proc.communicate()[0]
, it didn't complain about that. (It crashed elsewhere, as described at #23876.) So I am not convinced that it is needed with Python 3.
Okay, I checked the Python documentation. I agree that stdout
should be a binary stream, but according to the json documentation, json.loads
can take something of type bytes
as input.
Edit: this behavior of json
is new as of Python 3.6.
comment:7 Changed 5 years ago by
I could have sworn that I only added the decode() because it was failing otherwise. But sage was already on 3.6 when I wrote this. I dont think either solution is clearly better then the other. But the one with a decode already has positive review so I propose to leave it as is.
comment:8 Changed 5 years ago by
Branch: | u/mderickx/23822 → f3e5282c30fe45aa34b611f052341fb546e8e43b |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
New commits:
Trac #23822: fix python3 build error: unicode problem in pip_installed_packages()