Opened 5 years ago

Closed 5 years ago

#24901 closed defect (fixed)

optional package notedown fail to install

Reported by: Vincent Delecroix Owned by:
Priority: blocker Milestone: sage-8.2
Component: packages: optional Keywords:
Cc: Merged in:
Authors: Vincent Delecroix Reviewers: Ralf Stephan
Report Upstream: Reported upstream. No feedback yet. Work issues:
Branch: e60f898 (Commits, GitHub, GitLab) Commit: e60f898c4c56bf94675c8952ff6c21e0692d2023
Dependencies: Stopgaps:

Status badges

Description (last modified by Vincent Delecroix)

the machine is running archlinux

Found local metadata for notedown-1.5.1
Using cached file /opt/sage/upstream/notedown-1.5.1.tar.gz
notedown-1.5.1
====================================================
Setting up build directory for notedown-1.5.1
Finished extraction
No patch files found in ../patches
****************************************************
Host system:
Linux mangouste 4.14.15-1-ARCH #1 SMP PREEMPT Tue Jan 23 21:49:25 UTC 2018 x86_64 GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (GCC) 
****************************************************
Installing notedown-1.5.1
Traceback (most recent call last):
  File "setup.py", line 10, in <module>
    readme = pandoc.communicate()[0].decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3031: ordinal not in range(128)
Error: could not determine package name
********************************************************************************
Error installing notedown-1.5.1
********************************************************************************

real	0m0.896s
user	0m0.437s
sys	0m0.103s
************************************************************************
Error installing package notedown-1.5.1
************************************************************************
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and including the log file
  /opt/sage/logs/pkgs/notedown-1.5.1.log
Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/opt/sage/local/var/tmp/sage/build/notedown-1.5.1 and type 'make' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
  (cd '/opt/sage/local/var/tmp/sage/build/notedown-1.5.1' && '/opt/sage/sage' --sh)
When you are done debugging, you can type "exit" to leave the subshell.
************************************************************************

Upstream report at https://github.com/aaren/notedown/issues/76

Change History (8)

comment:1 Changed 5 years ago by François Bissey

There was a similar issue with fpylll unicode in the readme was causing a fit. This is an issue that needs fixing upstream by removing the offending stuff from the readme file.

comment:2 Changed 5 years ago by Vincent Delecroix

The README.md is perfectly ASCII. Apparently, pandoc is producing those weird characters

>>> pandoc = subprocess.Popen(['pandoc', 'README.md', '--to', 'rst'],
... stdout=subprocess.PIPE)
>>> readme = pandoc.communicate()[0]
>>> readme.decode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3325: ordinal not in range(128)
>>> readme[3320:3330]
'ython\xe2\x80\x99s '

comment:3 Changed 5 years ago by Vincent Delecroix

Description: modified (diff)
Report Upstream: N/AReported upstream. No feedback yet.

comment:4 Changed 5 years ago by Vincent Delecroix

Authors: Vincent Delecroix
Branch: u/vdelecroix/24901
Commit: e60f898c4c56bf94675c8952ff6c21e0692d2023
Status: newneeds_review

New commits:

e60f89824901: patch for avoiding pandoc in setup.py

comment:5 in reply to:  2 ; Changed 5 years ago by vklein

Replying to vdelecroix:

The README.md is perfectly ASCII. Apparently, pandoc is producing those weird characters

>>> pandoc = subprocess.Popen(['pandoc', 'README.md', '--to', 'rst'],
... stdout=subprocess.PIPE)
>>> readme = pandoc.communicate()[0]
>>> readme.decode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3325: ordinal not in range(128)
>>> readme[3320:3330]
'ython\xe2\x80\x99s '


For the record i am not sure if this example alone show a bug in pandoc as pandoc seems to work in utf-8 by default (https://pandoc.org/MANUAL.html#character-encoding). Pandoc don't seem to claim to keep the input encoding. In your exemple doing readme.decode('utf-8') instead of readme.decode() works fine.
On ubuntu 16.4 with pandoc 1.16.0.2 sage -i notedown works fine (sage version 8.2.beta7)

Version 0, edited 5 years ago by vklein (next)

comment:6 in reply to:  5 Changed 5 years ago by Vincent Delecroix

Replying to vklein:

Replying to vdelecroix:

The README.md is perfectly ASCII. Apparently, pandoc is producing those weird characters

>>> pandoc = subprocess.Popen(['pandoc', 'README.md', '--to', 'rst'],
... stdout=subprocess.PIPE)
>>> readme = pandoc.communicate()[0]
>>> readme.decode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3325: ordinal not in range(128)
>>> readme[3320:3330]
'ython\xe2\x80\x99s '


For the record i am not sure if this example alone show a bug in pandoc as pandoc seems to work in utf-8 by default (https://pandoc.org/MANUAL.html#character-encoding).

Pandoc don't seem to claim to keep the input encoding. In your exemple doing readme.decode('utf-8') instead of readme.decode() works fine.

On ubuntu 16.4 with pandoc 1.16.0.2 sage -i notedown works fine (sage version 8.2.beta7)

I never claimed that it was a bug in pandoc. The commands are extracted from the setup.py and produce an error on my computer. As specified in the documentation the behavior of decode depends on some default encoding that might differ between platforms.

Replacing readme.decode() with readme.decode('utf-8') in setup.py solves the problem mentioned in the description. However, I am not sure that utf-8 characters are encouraged in the long_description argument of the setup (there is no way to specify encoding there).

comment:7 Changed 5 years ago by Ralf Stephan

Reviewers: Ralf Stephan
Status: needs_reviewpositive_review

This branch installs notedown fine here, both without and with system pandoc 1.19.2.1. LGTM.

comment:8 Changed 5 years ago by Volker Braun

Branch: u/vdelecroix/24901e60f898c4c56bf94675c8952ff6c21e0692d2023
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.