#24901 closed defect (fixed)

optional package notedown fail to install

Reported by: vdelecroix Owned by:
Priority: blocker Milestone: sage-8.2
Component: packages: optional Keywords:
Cc: Merged in:
Authors: Vincent Delecroix Reviewers: Ralf Stephan
Report Upstream: Reported upstream. No feedback yet. Work issues:
Branch: e60f898 (Commits) Commit: e60f898c4c56bf94675c8952ff6c21e0692d2023
Dependencies: Stopgaps:

Description (last modified by vdelecroix)

the machine is running archlinux

Found local metadata for notedown-1.5.1
Using cached file /opt/sage/upstream/notedown-1.5.1.tar.gz
notedown-1.5.1
====================================================
Setting up build directory for notedown-1.5.1
Finished extraction
No patch files found in ../patches
****************************************************
Host system:
Linux mangouste 4.14.15-1-ARCH #1 SMP PREEMPT Tue Jan 23 21:49:25 UTC 2018 x86_64 GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp
Thread model: posix
gcc version 7.3.0 (GCC) 
****************************************************
Installing notedown-1.5.1
Traceback (most recent call last):
  File "setup.py", line 10, in <module>
    readme = pandoc.communicate()[0].decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3031: ordinal not in range(128)
Error: could not determine package name
********************************************************************************
Error installing notedown-1.5.1
********************************************************************************

real	0m0.896s
user	0m0.437s
sys	0m0.103s
************************************************************************
Error installing package notedown-1.5.1
************************************************************************
Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and including the log file
  /opt/sage/logs/pkgs/notedown-1.5.1.log
Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, *don't* just cd to
/opt/sage/local/var/tmp/sage/build/notedown-1.5.1 and type 'make' or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
  (cd '/opt/sage/local/var/tmp/sage/build/notedown-1.5.1' && '/opt/sage/sage' --sh)
When you are done debugging, you can type "exit" to leave the subshell.
************************************************************************

Upstream report at https://github.com/aaren/notedown/issues/76

Change History (8)

comment:1 Changed 17 months ago by fbissey

There was a similar issue with fpylll unicode in the readme was causing a fit. This is an issue that needs fixing upstream by removing the offending stuff from the readme file.

comment:2 follow-up: Changed 17 months ago by vdelecroix

The README.md is perfectly ASCII. Apparently, pandoc is producing those weird characters

>>> pandoc = subprocess.Popen(['pandoc', 'README.md', '--to', 'rst'],
... stdout=subprocess.PIPE)
>>> readme = pandoc.communicate()[0]
>>> readme.decode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3325: ordinal not in range(128)
>>> readme[3320:3330]
'ython\xe2\x80\x99s '

comment:3 Changed 17 months ago by vdelecroix

  • Description modified (diff)
  • Report Upstream changed from N/A to Reported upstream. No feedback yet.

comment:4 Changed 17 months ago by vdelecroix

  • Authors set to Vincent Delecroix
  • Branch set to u/vdelecroix/24901
  • Commit set to e60f898c4c56bf94675c8952ff6c21e0692d2023
  • Status changed from new to needs_review

New commits:

e60f89824901: patch for avoiding pandoc in setup.py

comment:5 in reply to: ↑ 2 ; follow-up: Changed 17 months ago by vklein

Replying to vdelecroix:

The README.md is perfectly ASCII. Apparently, pandoc is producing those weird characters

>>> pandoc = subprocess.Popen(['pandoc', 'README.md', '--to', 'rst'],
... stdout=subprocess.PIPE)
>>> readme = pandoc.communicate()[0]
>>> readme.decode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3325: ordinal not in range(128)
>>> readme[3320:3330]
'ython\xe2\x80\x99s '


For the record i am not sure if this example alone show a bug in pandoc as pandoc seems to work in utf-8 by default (https://pandoc.org/MANUAL.html#character-encoding).
Pandoc don't seem to claim to keep the input encoding. In your exemple doing readme.decode('utf-8') instead of readme.decode() works fine.

On ubuntu 16.4 with pandoc 1.16.0.2 sage -i notedown works fine (sage version 8.2.beta7)

Last edited 17 months ago by vklein (previous) (diff)

comment:6 in reply to: ↑ 5 Changed 17 months ago by vdelecroix

Replying to vklein:

Replying to vdelecroix:

The README.md is perfectly ASCII. Apparently, pandoc is producing those weird characters

>>> pandoc = subprocess.Popen(['pandoc', 'README.md', '--to', 'rst'],
... stdout=subprocess.PIPE)
>>> readme = pandoc.communicate()[0]
>>> readme.decode()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3325: ordinal not in range(128)
>>> readme[3320:3330]
'ython\xe2\x80\x99s '


For the record i am not sure if this example alone show a bug in pandoc as pandoc seems to work in utf-8 by default (https://pandoc.org/MANUAL.html#character-encoding).

Pandoc don't seem to claim to keep the input encoding. In your exemple doing readme.decode('utf-8') instead of readme.decode() works fine.

On ubuntu 16.4 with pandoc 1.16.0.2 sage -i notedown works fine (sage version 8.2.beta7)

I never claimed that it was a bug in pandoc. The commands are extracted from the setup.py and produce an error on my computer. As specified in the documentation the behavior of decode depends on some default encoding that might differ between platforms.

Replacing readme.decode() with readme.decode('utf-8') in setup.py solves the problem mentioned in the description. However, I am not sure that utf-8 characters are encouraged in the long_description argument of the setup (there is no way to specify encoding there).

comment:7 Changed 17 months ago by rws

  • Reviewers set to Ralf Stephan
  • Status changed from needs_review to positive_review

This branch installs notedown fine here, both without and with system pandoc 1.19.2.1. LGTM.

comment:8 Changed 16 months ago by vbraun

  • Branch changed from u/vdelecroix/24901 to e60f898c4c56bf94675c8952ff6c21e0692d2023
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.