Opened 2 years ago

Closed 6 months ago

#31267 closed defect (invalid)

ubuntu-bionic: python 3.8.5 readline module does not build after installing ncurses, readline spkgs

Reported by: guenterrote Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: build Keywords:
Cc: dimpase, gh-zlscherr, gh-kliem Merged in:
Authors: Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #32137 Stopgaps:

Status badges

Description (last modified by mkoeppe)

running on Ubuntu 18.04. I get the same result on the "master" and the current "develop" branch

config-log and build-log is attached.

The bug seems to have something to do with these error messages:

*** WARNING: renaming "readline" since importing it failed: /home/rote/Dokumente/Software/sage/local/lib/libreadline.so.8: undefined symbol: UP


[python3-3.8.5] Following modules built successfully but were removed because they could not be imported:
[python3-3.8.5] readline   

This error can be reproduced using:

tox -e docker-ubuntu-bionic-standard -- ncurses readline python3

Attachments (2)

python3-3.8.5.log (1.5 MB) - added by guenterrote 2 years ago.
python build logfile
config.log (171.6 KB) - added by guenterrote 2 years ago.
config.log

Download all attachments as: .zip

Change History (22)

Changed 2 years ago by guenterrote

Attachment: python3-3.8.5.log added

python build logfile

Changed 2 years ago by guenterrote

Attachment: config.log added

config.log

comment:1 Changed 2 years ago by guenterrote

Description: modified (diff)

comment:2 Changed 2 years ago by guenterrote

Description: modified (diff)

comment:3 Changed 2 years ago by mkoeppe

Cc: dimpase added

From python3...log:

checking for getc_unlocked() and friends... yes
checking how to link readline libs... -lreadline -ltinfo
...

but when python3 actually links readline, it is doing something else:

building 'readline' extension
gcc -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -O2 -g -march=native -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I./Include -I/home/rote/Dokumente/Software/sage/local/include -I. -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/home/rote/Dokumente/Software/sage/local/var/tmp/sage/build/python3-3.8.5/src/Include -I/home/rote/Dokumente/Software/sage/local/var/tmp/sage/build/python3-3.8.5/src -c /home/rote/Dokumente/Software/sage/local/var/tmp/sage/build/python3-3.8.5/src/Modules/readline.c -o build/temp.linux-x86_64-3.8/home/rote/Dokumente/Software/sage/local/var/tmp/sage/build/python3-3.8.5/src/Modules/readline.o
gcc -pthread -shared -Wl,-rpath-link,/home/rote/Dokumente/Software/sage/local/lib -L/home/rote/Dokumente/Software/sage/local/lib -Wl,-rpath,/home/rote/Dokumente/Software/sage/local/lib -L. -Wl,-rpath-link,/home/rote/Dokumente/Software/sage/local/lib -L/home/rote/Dokumente/Software/sage/local/lib -Wl,-rpath,/home/rote/Dokumente/Software/sage/local/lib -L. -Wl,-rpath-link,/home/rote/Dokumente/Software/sage/local/lib -L/home/rote/Dokumente/Software/sage/local/lib -Wl,-rpath,/home/rote/Dokumente/Software/sage/local/lib build/temp.linux-x86_64-3.8/home/rote/Dokumente/Software/sage/local/var/tmp/sage/build/python3-3.8.5/src/Modules/readline.o -L/usr/lib/termcap -L. -L/home/rote/Dokumente/Software/sage/local/lib -L/usr/lib/x86_64-linux-gnu -L/usr/local/lib -lreadline -lncursesw -o build/lib.linux-x86_64-3.8/readline.cpython-38-x86_64-linux-gnu.so

You could check if uninstalling libncursesw5-dev solves this problem

Last edited 2 years ago by mkoeppe (previous) (diff)

comment:4 Changed 2 years ago by mkoeppe

https://github.com/python/cpython/blob/master/configure.ac#L4887 checks it in the order "" tinfo ncursesw ncurses curses termcap

https://github.com/python/cpython/blob/master/setup.py#L975 detect_readline_curses redoes the checks in a different way

comment:5 Changed 2 years ago by mkoeppe

Günter, on a typical linux machine, one would actually expect that ncurses and readline can be used and do not have to be built from scratch. So I would recommend to do make ncurses-clean readline-clean and then re-run configure. This should be more robust than the current configuration

comment:6 Changed 2 years ago by mkoeppe

Description: modified (diff)

comment:7 Changed 2 years ago by mkoeppe

Description: modified (diff)
Summary: python 3.8.5 subpackage does not buildubuntu-bionic: python 3.8.5 readline module does not build after installing ncurses, readline spkgs

comment:8 Changed 2 years ago by mkoeppe

root@4d2478da7517:/sage# ldd local/lib/libreadline.so.8
	linux-vdso.so.1 (0x00007ffc7a3c8000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fee30793000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fee30dd5000)
root@4d2478da7517:/sage# ldd /usr/lib/x86_64-linux-gnu/libreadline.so 
	linux-vdso.so.1 (0x00007ffe23ded000)
	libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007f100ed8d000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f100e99c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f100f200000)

The issue likely arises because find_library_file inspects the system library but then the Sage-built library is used for linking https://github.com/python/cpython/blob/master/setup.py#L977

I think this is a limitation of distutils - it does not know about LIBRARY_PATH

comment:9 Changed 2 years ago by mkoeppe

Cc: gh-zlscherr added

I think this would need a similar fix as it needed in the packaging of python3 in homebrew...

comment:10 Changed 2 years ago by mkoeppe

Cc: gh-kliem added

comment:11 in reply to:  5 ; Changed 2 years ago by guenterrote

Replying to mkoeppe:

Günter, on a typical linux machine, one would actually expect that ncurses and readline can be used and do not have to be built from scratch. So I would recommend to do make ncurses-clean readline-clean and then re-run configure. This should be more robust than the current configuration

I tried this; the python installation seemed to go through fine, but eventually the build failed.

cd ../.. && sage-logger -p './sage --docbuild --no-pdf-links all html ' logs/dochtml.log
[dochtml] Traceback (most recent call last):
...
[dochtml]   File "sage/matrix/matrix_mpolynomial_dense.pyx", line 1, in init sage.matrix.matrix_mpolynomial_dense (build/cythonized/sage/matrix/matrix_mpolynomial_dense.cpp:8672)
[dochtml] ImportError: libreadline.so.8: cannot open shared object file: No such file or directory

Trying to start sage anyway displays a nice banner but crashes with the same error

---------------------------------------------------------------------------
ImportErrorPython 3.8.5: /home/rote/Dokumente/Software/sage/local/bin/python3
                                                   Thu Jan 21 07:42:55 2021
A problem occurred executing Python code.  Here is the sequence of function
...
...
~/Dokumente/Software/sage/local/lib/python3.8/site-packages/sage/matrix/matrix_mpolynomial_dense.pyx in init sage.matrix.matrix_mpolynomial_dense (build/cythonized/sage/matrix/matrix_mpolynomial_dense.cpp:8672)()
----> 1 """
      2 Dense matrices over multivariate polynomials over fields
...
ImportError: libreadline.so.8: cannot open shared object file: No such file or directory

comment:12 in reply to:  11 ; Changed 2 years ago by mkoeppe

Replying to guenterrote:

Replying to mkoeppe:

Günter, on a typical linux machine, one would actually expect that ncurses and readline can be used and do not have to be built from scratch. So I would recommend to do make ncurses-clean readline-clean and then re-run configure. This should be more robust than the current configuration

I tried this; the python installation seemed to go through fine, but eventually the build failed.

cd ../.. && sage-logger -p './sage --docbuild --no-pdf-links all html ' logs/dochtml.log
[dochtml] Traceback (most recent call last):
...
[dochtml]   File "sage/matrix/matrix_mpolynomial_dense.pyx", line 1, in init sage.matrix.matrix_mpolynomial_dense (build/cythonized/sage/matrix/matrix_mpolynomial_dense.cpp:8672)
[dochtml] ImportError: libreadline.so.8: cannot open shared object file: No such file or directory

Trying to start sage anyway displays a nice banner but crashes with the same error

---------------------------------------------------------------------------
ImportErrorPython 3.8.5: /home/rote/Dokumente/Software/sage/local/bin/python3
                                                   Thu Jan 21 07:42:55 2021
A problem occurred executing Python code.  Here is the sequence of function
...
...
~/Dokumente/Software/sage/local/lib/python3.8/site-packages/sage/matrix/matrix_mpolynomial_dense.pyx in init sage.matrix.matrix_mpolynomial_dense (build/cythonized/sage/matrix/matrix_mpolynomial_dense.cpp:8672)()
----> 1 """
      2 Dense matrices over multivariate polynomials over fields
...
ImportError: libreadline.so.8: cannot open shared object file: No such file or directory

This looks like make singular-clean sagelib-clean build could fix it

comment:13 in reply to:  12 Changed 2 years ago by guenterrote

Replying to mkoeppe:

This looks like make singular-clean sagelib-clean build could fix it

This worked! Thanks!

comment:14 Changed 23 months ago by mkoeppe

Milestone: sage-9.3sage-9.4

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

comment:15 Changed 18 months ago by mkoeppe

Milestone: sage-9.4sage-9.5

comment:16 Changed 14 months ago by mkoeppe

Milestone: sage-9.5sage-9.6

comment:17 Changed 9 months ago by mkoeppe

Milestone: sage-9.6sage-9.7

comment:18 Changed 7 months ago by mkoeppe

Dependencies: #32137
Milestone: sage-9.7sage-duplicate/invalid/wontfix
Status: newneeds_review

Probably because of our ncurses/readline update in #32137, tox -e docker-ubuntu-bionic-standard -- ncurses readline python3 now works.

comment:19 Changed 7 months ago by dimpase

Reviewers: Dima Pasechnik
Status: needs_reviewpositive_review

ok

comment:20 Changed 6 months ago by mkoeppe

Resolution: invalid
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.