Opened 4 years ago

Closed 11 months ago

#25996 closed defect (invalid)

32-bit binaries broken

Reported by: vbraun Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: build Keywords: sdl
Cc: fbissey, dimpase Merged in:
Authors: Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

Looks like something is wrong when building SAGE_FAT_BINARY=yes SAGE_INSTALL_GCC=yes and the system gcc is < 7 on linux 32-bit (works on 64-bit linux):

ImportError: /lib/i386-linux-gnu/libgcc_s.so.1: version `GCC_7.0.0' not found (required by /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libSingular-4.1.0.so)

***************************************************************************

History of session input:
*** Last line of input (may not be in above history):
buildbot@sagebd09_32s02:~/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2$ ldd /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libSingular-4.1.0.so
	linux-gate.so.1 (0xf7783000)
	libpolys-4.0.3.so => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libpolys-4.0.3.so (0xf727c000)
	libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf7271000)
	libfactory-4.1.0.so => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libfactory-4.1.0.so (0xf702e000)
	libsingular_resources-4.0.3.so => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libsingular_resources-4.0.3.so (0xf7027000)
	libomalloc-0.9.6.so => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libomalloc-0.9.6.so (0xf701c000)
	libflint.so.13 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libflint.so.13 (0xf6b36000)
	libmpfr.so.6 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libmpfr.so.6 (0xf6ab7000)
	libgmp.so.23 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libgmp.so.23 (0xf6a2e000)
	libntl.so.33 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libntl.so.33 (0xf6810000)
	libreadline.so.6 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libreadline.so.6 (0xf67c8000)
	librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0xf67bf000)
	libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf67a2000)
	libstdc++.so.6 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libstdc++.so.6 (0xf6627000)
	libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf65d2000)
	libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf641b000)
	libgcc_s.so.1 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libgcc_s.so.1 (0xf63fe000)
	/lib/ld-linux.so.2 (0xf7785000)
	libgf2x.so.1 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libgf2x.so.1 (0xf63ef000)
	libtinfo.so.6 => /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libtinfo.so.6 (0xf63bd000)
buildbot@sagebd09_32s02:~/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2$ ldd /lib/i386-linux-gnu/libm.so.6
	linux-gate.so.1 (0xf7788000)
	libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7571000)
	/lib/ld-linux.so.2 (0xf778a000)
buildbot@sagebd09_32s02:~/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2$ ldd /lib/i386-linux-gnu/libc.so.6
	/lib/ld-linux.so.2 (0xf7720000)
	linux-gate.so.1 (0xf771e000)

libSingular dosen't directly depend on libgcc_s so it must be that some other library required it first...

Change History (22)

comment:1 Changed 4 years ago by jdemeyer

Duplicate of #25304?

comment:2 Changed 4 years ago by vbraun

Maybe, except that its not Debian 9 specific

Looks like the only direct dependency on the wrong libgcc_s is from gcc itself:

buildbot@sagebd09_32s02:~/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2$ ldd local/lib/libstdc++.so
	linux-gate.so.1 (0xf77c7000)
	libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf75ed000)
	libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf7436000)
	/lib/ld-linux.so.2 (0xf77c9000)
	libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0xf7416000)

comment:3 Changed 4 years ago by vbraun

Unsurprisingly, the rpath is not used when linking:

libtool: link:  /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/./gcc/xgcc -shared-libgcc -B/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulau
jayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/./gcc -nostdinc++ -L/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/i686-p
c-linux-gnu/libstdc++-v3/src -L/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/i686-pc-linux-gnu/libstdc++-v3/src/.libs -L/home/buildbot/slave/binary_pkg/bu
ild/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/i686-pc-linux-gnu/libstdc++-v3/libsupc++/.libs -B/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas
391yaj2/local/i686-pc-linux-gnu/bin/ -B/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/i686-pc-linux-gnu/lib/ -isystem /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia8
8eourzeqip0oidmas391yaj2/local/i686-pc-linux-gnu/include -isystem /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/i686-pc-linux-gnu/sys-include     -fPIC -DPIC -D_GLIBCXX_SHARED -shared -nostdlib
 /usr/lib/i386-linux-gnu/crti.o /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/./gcc/crtbeginS.o  .libs/compatibility.o .libs/compatibility-debug_list.o .l
ibs/compatibility-debug_list-2.o .libs/compatibility-c++0x.o .libs/compatibility-atomic-c++0x.o .libs/compatibility-thread-c++0x.o .libs/compatibility-chrono.o .libs/compatibility-condvar.o  -Wl,--whole-archive ../libsupc++/.libs/libsupc++convenience.a .
./src/c++98/.libs/libc++98convenience.a ../src/c++11/.libs/libc++11convenience.a -Wl,--no-whole-archive  -L/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/i
686-pc-linux-gnu/libstdc++-v3/libsupc++/.libs -L/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/i686-pc-linux-gnu/libstdc++-v3/src -L/home/buildbot/slave/bi
nary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/i686-pc-linux-gnu/libstdc++-v3/src/.libs -lm -L/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeq
ip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/./gcc -L/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib -L/usr/lib/i386-linux-gnu -L/lib/i386-linux-gnu -lc -lgcc_s /home/buildbo
t/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/var/tmp/sage/build/gcc-7.2.0/gcc-build/./gcc/crtendS.o /usr/lib/i386-linux-gnu/crtn.o  -Wl,-O1 -Wl,-z -Wl,relro -Wl,--gc-sections -Wl,--version-script=libstdc++
-symbols.ver   -Wl,-soname -Wl,libstdc++.so.6 -o .libs/libstdc++.so.6.0.24

comment:5 Changed 4 years ago by vbraun

After rebuilding LD_RUN_PATH=$SAGE_ROOT/local/lib ./sage -p gcc I get the right rpath on libstdc++.so.6 but still crashes on startup

comment:6 Changed 4 years ago by vbraun

Apparently libgcc_s is dlopen()-ed from libpthread.

$ LD_DEBUG=all ./sage
[...]
     15757:	opening file=/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/python2.7/site-packages/cysignals/signals.so [0]; direct_opencount=1
     15757:	
     15757:	symbol=initsignals;  lookup in file=/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/python2.7/site-packages/cysignals/signals.so [0]
     15757:	binding file /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/python2.7/site-packages/cysignals/signals.so [0] to /home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/python2.7/site-packages/cysignals/signals.so [0]: normal symbol `initsignals'
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/bin/python2 [0]
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/home/buildbot/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidmas391yaj2/local/lib/libpython2.7.so.1.0 [0]
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/lib/i386-linux-gnu/libpthread.so.0 [0]
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/lib/i386-linux-gnu/libdl.so.2 [0]
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/lib/i386-linux-gnu/libutil.so.1 [0]
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/lib/i386-linux-gnu/libm.so.6 [0]
     15757:	symbol=__libc_dlopen_mode;  lookup in file=/lib/i386-linux-gnu/libc.so.6 [0]
     15757:	binding file /lib/i386-linux-gnu/libpthread.so.0 [0] to /lib/i386-linux-gnu/libc.so.6 [0]: normal symbol `__libc_dlopen_mode' [GLIBC_PRIVATE]
     15757:	
     15757:	file=libgcc_s.so.1 [0];  dynamically loaded by /lib/i386-linux-gnu/libpthread.so.0 [0]
     15757:	find library=libgcc_s.so.1 [0]; searching
     15757:	 search cache=/etc/ld.so.cache
     15757:	  trying file=/lib/i386-linux-gnu/libgcc_s.so.1
     15757:	
     15757:	file=libgcc_s.so.1 [0];  generating link map
     15757:	  dynamic: 0xf48feeec  base: 0xf48e2000   size: 0x0001d254
     15757:	    entry: 0xf48e40a0  phdr: 0xf48e2034  phnum:          7

The gcc support library contains, low-level auxiliary stuff like long long division on 32-bit. So that would explain why it is 32-bit specific, presumably libpthread doesn't need it on 64-bit.

To fix this we need to load libgcc_s before it can be indirectly required through pthread, e.g. via a special cython module or by setting LD_PRELOAD if we compiled our own gcc. The LD_PRELOAD verision works, btw, but is fugly...

Last edited 4 years ago by vbraun (previous) (diff)

comment:7 Changed 4 years ago by vbraun

  • Authors set to fbissey

comment:8 Changed 4 years ago by vbraun

  • Authors fbissey deleted
  • Cc fbissey added

comment:9 Changed 4 years ago by fbissey

Lovely. Looks to me like gcc should have at least bumped the soname of libgcc_s in gcc-7, may be some other as well. I guess we can force a LD_LIBRARY_PATH at linking time for everything from inside gcc by changing the spec but that's quite drastic.

comment:10 follow-up: Changed 4 years ago by vbraun

Not sure what you mean...

Another option is to abandon 32-bit binaries, we don't have OSX 32-bit binaries so how much effort should we spend on Linux 32-bit?

comment:11 in reply to: ↑ 10 ; follow-up: Changed 4 years ago by fbissey

Replying to vbraun:

Not sure what you mean...

I thought that it may have been why I was copied. Making gcc insert rpath by default.

Another option is to abandon 32-bit binaries, we don't have OSX 32-bit binaries so how much effort should we spend on Linux 32-bit?

Support for x86 32 bits is certainly winding down. Also distro packaging has picked up and may be enough for these platforms at this stage.

comment:12 in reply to: ↑ 11 Changed 4 years ago by vbraun

Replying to fbissey:

I thought that it may have been why I was copied. Making gcc insert rpath by default.

We do set rpath already, but libgcc_s is loaded indirectly via the system libpthread. Since rpath is not transitive, pthread picks up the wrong libgcc_s...

I think the only solution would be to have a direct dependency on libgcc_s before libpthread is loaded.

comment:13 follow-up: Changed 4 years ago by tmonteil

For what is worth, Sage Debian Live is still 32-bit (in order to run on most computers). I would be happy not to be alone to track the issues appearing there :)

comment:14 in reply to: ↑ 13 Changed 4 years ago by fbissey

Replying to tmonteil:

For what is worth, Sage Debian Live is still 32-bit (in order to run on most computers). I would be happy not to be alone to track the issues appearing there :)

The issue is really only about binaries produced by a newer compiler (gcc-7.0+) than the host system. If your live cd compiled sage with the system compiler it is fine.

Now for more wild ideas! What if we build 32bit images with clang? Extra complexity but it may avoid some troubles.

comment:15 follow-up: Changed 4 years ago by vbraun

Still happens on sage 8.4, for the record

comment:16 in reply to: ↑ 15 Changed 4 years ago by fbissey

Replying to vbraun:

Still happens on sage 8.4, for the record

Unsurprisingly. LD_PRELOAD is the only real way out of that rabbit hole apart from relying on a system compiler.

comment:17 Changed 3 years ago by tmonteil

  • Keywords sdl added

comment:18 Changed 2 years ago by mkoeppe

Is this still an unresolved problem?

comment:19 Changed 2 years ago by mkoeppe

Is it the same as #25304?

comment:20 Changed 11 months ago by mkoeppe

  • Cc dimpase added
  • Milestone changed from sage-8.4 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

Outdated, and in any case cannot move forward without cooperation by the binary builder; should close

comment:21 Changed 11 months ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

comment:22 Changed 11 months ago by mkoeppe

  • Resolution set to invalid
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.