Opened 5 years ago

Closed 3 years ago

Last modified 2 years ago

#22679 closed task (wontfix)

port Sage to FreeBSD 11

Reported by: dimpase Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: porting: BSD Keywords:
Cc: stephen, fbissey, jdemeyer, jpflori Merged in:
Authors: Dima Pasechnik Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: u/dimpase/fbsd-support (Commits, GitHub, GitLab) Commit: 4aaeeb491635abea35e2804eeee7c93a80af38ea
Dependencies: #12426, #23700 Stopgaps:

Status badges

Description (last modified by dimpase)

Please see #26249 for the followup (for FreeBSD 12).

FreeBSD as of version 10 uses clang by default, and is closer to OSX in other ways too, so it would be a good free testing ground for the use of clang with Sage.

preparing tools:

  • install gcc6 and create fake gfortran by making a link to gfortran6
  • make sure as is new enough: pkg install binutils will install as version 2.28, and you need it to come first in PATH (i.e. make sure that /usr/local/bin comes before /usr/bin).
  • remove stale /lib/libgcc_s.so.1 and point it to the right one in /usr/local/lib/gcc6/ (perhaps there is a better way...)
  • install gmake, gawk, gsed, and create fake make and sed to point to gmake and gsed
  • make sure /bin/sh is actually bash
  • install pkgconf (FreeBSD's replacement of pkg-config)
  • install wget (needed for AtlasRep GAP package).
  • install icu (needed for R)

building Sage itself

starting from a clean git repo:

export CFLAGS="-fPIC"
export CXXFLAGS="-fPIC"
export LDFLAGS="-L/usr/local/lib -Wl,--add-needed -lm"
./configure CC=cc CXX=c++  #(this is clang 3.8.0 a.t.m.)
MAKE="make -j7" make build
MAKE="make -j4" make # building docs is very memory-hungry

HISTORICAL: Sage itself:

  • hack configure.ac to make freebsd allowed platform
  • deal with cephes' complex.h conflicting with numpy - #22786
  • gf2x: only builds with SAGE_FAT_BINARY on
  • upgrade sqlite to version 3.17 - otherwise there are compiling problems with editline (see #22687)
  • flint: remove "-ascii -pedantic" flags from CFLAGS in configure
  • libraries (libgap, cddlib, etc) need -fPIC for CFLAGS (so we build with CC="cc -fPIC -pipe" ; (-pipe is probably not really needed)) - see #22784
  • but gap cannot be built with CC="cc -fPIC -pipe", as it does something weird in ./configure, setting BASECC to be cc, and generating broken Makefile (with -fPIC and -pipe on two separate lines); so build gap with CC=cc
  • giac needs CFLAGS="-I/usr/local/include" to find libintl.h and LDFLAGS="-L/usr/local/lib -lintl" to link.
  • either provide /bin/bash or fix this properly (#22689)
  • Sage builds and runs, modulo linking issues with flint and arb, which can be fixed ad hoc by hand.
  • deal with cephes' libm not pulling system's libm, unless some arcane linker flags are used (see comments below)
  • old freebsd workaround for libgd removed in #22785
  • ...

Change History (99)

comment:1 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:2 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:3 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:4 Changed 5 years ago by dimpase

  • Description modified (diff)
  • Summary changed from port Sage to FreeBSD 11. to port Sage to FreeBSD 11

comment:5 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:6 Changed 5 years ago by dimpase

cf my comments at https://github.com/numpy/numpy/issues/8530

for the need for /lib/libgcc_s.so.1 to be replaced.

Basically, gcc/gfortran and clang's toolchains are slightly incompatible; while workarounds are in place for OSX, linking on freebsd makes it more or less mandatory.

Last edited 5 years ago by dimpase (previous) (diff)

comment:7 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:8 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:9 Changed 5 years ago by dimpase

  • Description modified (diff)

one of internal giac tests fail (harmlessly, I hope):

< -sqrt(6)*I*sqrt(sqrt(13)+3),sqrt(6)*I*sqrt(sqrt(13)+3),-sqrt(6)*sqrt(sqrt(13)-3),sqrt(6)*sqrt(sqrt(13)-3),0,
---
> sqrt(6)*I*sqrt(sqrt(13)+3),-sqrt(6)*I*sqrt(sqrt(13)+3),-sqrt(6)*sqrt(sqrt(13)-3),sqrt(6)*sqrt(sqrt(13)-3),0,
FAIL: chk_fhan16

comment:10 Changed 5 years ago by dimpase

  • Description modified (diff)

comment:11 Changed 4 years ago by dimpase

  • Authors set to Dima Pasechnik
  • Dependencies set to #22687
  • Description modified (diff)

comment:12 Changed 4 years ago by dimpase

  • Branch set to u/dimpase/freebsd_experimental
  • Commit set to 90234a3bb7502f31b625d7315a94d6737579e2c0

New commits:

a44a934updating sqlite to version 3.17
a32b0c7let Sage accept freebsd
3178fbcremove -ascii -pedantic flags from configure
4691d92allow clang++ and libc++ combo on freebsd
90234a3using libintl from /usr/local/

comment:13 Changed 4 years ago by git

  • Commit changed from 90234a3bb7502f31b625d7315a94d6737579e2c0 to df76ac248d308d0d51a76f43018ed9f714b7bf13

Branch pushed to git repo; I updated commit sha1. New commits:

df76ac2temporary fix to mke building on freebsd work

comment:14 Changed 4 years ago by git

  • Commit changed from df76ac248d308d0d51a76f43018ed9f714b7bf13 to f5a9b80bef908a59ae04c4f4090ac9293d6f2bc6

Branch pushed to git repo; I updated commit sha1. New commits:

f5a9b80disable cephes installation on freebsd version >= 10

comment:15 Changed 4 years ago by dimpase

  • Description modified (diff)

comment:16 Changed 4 years ago by git

  • Commit changed from f5a9b80bef908a59ae04c4f4090ac9293d6f2bc6 to 9b622a862a8476e9394c4e25aa2863f96de07ef5

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

516f029Upgraded to R 3.3.3
cbe9134Merge branch 'public/r311' of git://trac.sagemath.org/sage into finalr
f029f66This simple patch removes an overkill check in R's configure.
c8a0b1bMerge branch 'public/r311' of trac.sagemath.org:sage into sqliteupdate
d0d5ca7update fplll/fpylll to 5.1.0/0.2.4dev
9f702f3Merge branch 'u/malb/22643-fplll-upgrade' of trac.sagemath.org:sage into sqliteupdate
1201132LONG_LONG_MAX is an obsolete gnu-ism
276e658R on freebsd needs this for libcurl
0c0c603Write custom create_extension() for Cython
9b622a8Merge branch 'u/jdemeyer/use_create_extension___from_cython' of trac.sagemath.org:sage into sqliteupdate

comment:17 Changed 4 years ago by dimpase

  • Dependencies changed from #22687 to #22687, #22643, #22554
  • Description modified (diff)

builds and runs, modulo ad hoc fixing arb and flint linking issue:

$ ln -sf $SAGE_LOCAL/lib/libarb.so.1.1.1 $SAGE_LOCAL/lib/libarb.so.1
$ ln -sf $SAGE_LOCAL/lib/libflint.so.13.5.2 $SAGE_LOCAL/lib/libflint.so.13

comment:18 Changed 4 years ago by git

  • Commit changed from 9b622a862a8476e9394c4e25aa2863f96de07ef5 to a391ad68175a75057647230aedd758e8a648f03b

Branch pushed to git repo; I updated commit sha1. New commits:

a391ad6cephes still is needed for cpow/clog

comment:19 Changed 4 years ago by dimpase

without cephes, missing cpow() (shows up during building docs). And this is how things are on official FreeBSD.

Thus reverting f5a9b80. After thus, make completes.

Running tests now.


New commits:

a391ad6cephes still is needed for cpow/clog

comment:20 Changed 4 years ago by kcrisman

  • Cc stephen added

comment:21 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554 to #22687, #22643, #22554, #22692

comment:22 follow-up: Changed 4 years ago by dimpase

  • Cc fbissey jdemeyer added

Question to linking experts:

I am not sure whether letting cephes create libm.so rather than something like libm_complex.so is a good idea. At least I do not see how one can in such a configuration link an executable that needs things from the "normal" libm as well as from cephes---in particular as Sage has moved away from using things like LD_LIBRARY_PATH to using -rpath to link. E.g. consider

#include <math.h>
#include <complex.h>
#include <stdio.h>
int main()
{
        double e=exp2(2.0);
        double complex c=clog(2.0);
        printf("\n e=%f\n", e);
        printf("\n c=%f + i%f\n", creal(c), cimag(c));
        return 0;
}

With two different libm's, I don't see a way to create a working executable (with clang). However if I rename cephes' libm to libm_complex then (from SAGE_ROOT) the result of

cc -I./local/include t.c -lm -L./local/lib -lm_complex -Wl,-rpath=./local/lib

works just fine.

comment:23 in reply to: ↑ 22 ; follow-up: Changed 4 years ago by fbissey

Replying to dimpase:

Question to linking experts:

I am not sure whether letting cephes create libm.so rather than something like libm_complex.so is a good idea. At least I do not see how one can in such a configuration link an executable that needs things from the "normal" libm as well as from cephes---in particular as Sage has moved away from using things like LD_LIBRARY_PATH to using -rpath to link. E.g. consider

#include <math.h>
#include <complex.h>
#include <stdio.h>
int main()
{
        double e=exp2(2.0);
        double complex c=clog(2.0);
        printf("\n e=%f\n", e);
        printf("\n c=%f + i%f\n", creal(c), cimag(c));
        return 0;
}

With two different libm's, I don't see a way to create a working executable (with clang). However if I rename cephes' libm to libm_complex then (from SAGE_ROOT) the result of

cc -I./local/include t.c -lm -L./local/lib -lm_complex -Wl,-rpath=./local/lib

works just fine.

I am not entirely sure because I think freeBSD doesn't use sonames. But linking with two shared libraries of the same name is at best awkward. The linker can take -lm several times and several paths with -L, not a problem. The problem is at runtime. Let's see an elf shared object on linux, say openblas:

fbissey@moonloop ~ $ readelf -d /usr/lib64/libopenblas_threads_haswellp-r0.2.15.so

Dynamic section at offset 0x3e3b60 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libopenblas_threads.so.0]
 0x000000000000000c (INIT)               0x2ab38
....

The linker effectively create a table of libraries to be loaded at runtime. It also can include runtime path to search first before the normal order from ld.so.conf. But you see that if two libraries have strictly the same name the first one to be found will be the only one used.

If you link a library the linker may not complain if it cannot resolve some symbols because it cannot combine the two libm.so. Undefined symbols are legal in a library. It is not in an executable. I am not quite sure in that case if the linker can even see the two different set of symbols at the same time. In any case you will still have the problem at run time.

Apart from different name the only way out is to have one of the two being static instead of dynamic and then you pass it to the linker as an object with its full path (not just -L$path -lm but $path/libm.a).

comment:24 in reply to: ↑ 23 Changed 4 years ago by dimpase

  • Cc jpflori added

Replying to fbissey:

If you link a library the linker may not complain if it cannot resolve some symbols because it cannot combine the two libm.so. Undefined symbols are legal in a library. It is not in an executable. I am not quite sure in that case if the linker can even see the two different set of symbols at the same time. In any case you will still have the problem at run time.

There is no /lib/libm.so, only /lib/libm.so.5.

I think the idea at the time of old Sage port to FreeBSD (#9543) was that $SAGE_LOCAL/lib/libm.so serves as a replacement for system's libm. But it seems that even though the former (renamed) is linked to the latter,

$ ldd local/lib/libm_complex.so 
local/lib/libm_complex.so:
        libc.so.7 => /lib/libc.so.7 (0x800823000)
        libm.so.5 => /lib/libm.so.5 (0x80120c000)

this still leads to ugly linker errors complaining about absent exp2. More specifically, if I remove -lm from the invocation above I get

$ cc -I./local/include t.c -L./local/lib -lm_complex -Wl,-rpath=./local/lib

/usr/bin/ld: undefined reference to symbol `exp2@@FBSD_1.0' (try adding -lm)
//lib/libm.so.5: could not read symbols: Bad value

(the latter is puzzling - it knows where to look for exp2, but bails out...)

Apart from different name the only way out is to have one of the two being static instead of dynamic and then you pass it to the linker as an object with its full path (not just -L$path -lm but $path/libm.a).

this is not really better than creating a separate dynamic library, IMHO.

comment:25 follow-up: Changed 4 years ago by fbissey

Is /lib/libm.so.5 a real library? What does running file on it says?

comment:26 in reply to: ↑ 25 Changed 4 years ago by dimpase

Replying to fbissey:

Is /lib/libm.so.5 a real library? What does running file on it says?

$ file /lib/libm.so.5 
/lib/libm.so.5: ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, stripped

comment:27 follow-up: Changed 4 years ago by fbissey

OK that leaves us with the next stupid question: are you compiling in 32bits? Finally can you move -lm after -lm_complex?

comment:28 in reply to: ↑ 27 Changed 4 years ago by dimpase

Replying to fbissey:

OK that leaves us with the next stupid question: are you compiling in 32bits? Finally can you move -lm after -lm_complex?

it's 64-bit, and yes, I can build and run the result of

$ cc -I./local/include t.c -L./local/lib -lm_complex -lm -Wl,-rpath=./local/lib

just fine.

Certainly, having a new library to add, instead of providing a replacement for libm, is a pain, although a limited to few packages one.

comment:29 follow-up: Changed 4 years ago by fbissey

--as-needed is probably your default behavior which means order is important. I suspect you are using the "gold" linker instead of the "bfd" (classic) one [don't remember how to check for that]. Short of passing -Wl,--no-as-needed I don't know how to make the order irrelevant.

comment:30 follow-up: Changed 4 years ago by jpflori

Just a reminder that https://svnweb.freebsd.org/ports/head/math/sage/ used to be maintained. Maybe some stuff can be pulled from there.

comment:31 Changed 4 years ago by kcrisman

See also #12858, #14651, #14392 and in general this report. I'm sure username stephen would be happy to help but we didn't really take much stuff in terms of patches lately so perhaps that is why it was no longer maintained.

Last edited 4 years ago by kcrisman (previous) (diff)

comment:32 in reply to: ↑ 30 Changed 4 years ago by dimpase

Replying to jpflori:

Just a reminder that https://svnweb.freebsd.org/ports/head/math/sage/ used to be maintained. Maybe some stuff can be pulled from there.

they have there (as well as in #14491) an ld wrapper that uses --copy-dt-needed-entries option to recursively search libraries. And indeed the following (with the "classical" GNU buntils ld) gives working a.out.

$ cc -c -I./local/include t.c
$ /usr/local/bin/ld --eh-frame-hdr -dynamic-linker /libexec/ld-elf.so.1 -L./local/lib --copy-dt-needed-entries -rpath=./local/lib /usr/lib/crt1.o /usr/lib/crti.o t.o -lc -lm_complex /usr/lib/crtn.o

But this option has a different name with the default linker, as far as testing and manuals say.

/usr/bin/ld: unrecognized option '--copy-dt-needed-entries'

The reason is that the default linker used by clang is

$ /usr/bin/ld -v
GNU ld 2.17.50 [FreeBSD] 2007-07-03

The corresponding option is called --add-needed. (It's possible to find this by calling /usr/bin/ld --help.) So the right command (giving working a.out) to pull system's libm with the custom (renamed to libm_complex) libm is

cc -I./local/include -L./local/lib -Wl,--add-needed -Wl,-rpath=./local/lib -lm_complex t.c

comment:33 in reply to: ↑ 29 Changed 4 years ago by jdemeyer

Replying to fbissey:

--as-needed is probably your default behavior which means order is important. I suspect you are using the "gold" linker instead of the "bfd" (classic) one [don't remember how to check for that]. Short of passing -Wl,--no-as-needed I don't know how to make the order irrelevant.

Have a look at library_order_list in src/module_list.py to deal with ordering of libraries.

comment:34 Changed 4 years ago by dimpase

  • Description modified (diff)

Somewhat ironically, numpy now conflics with cephes (via <complex.h> header), as it adapted some FreeBSD code by stephen for its use in complex arithmetic. :-) Fortunately, it appears to be only the header-level conflict.

comment:35 Changed 4 years ago by dimpase

running make as follows

LDFLAGS="-L./local/lib -Wl,--add-needed -lm" $MAKE build

allows the build do finish, and pass many tests. Some of docbuilding crashes with rather puzzling (most probably related to multi-processing/threading) errors like

[dochtml] [hyperboli] loading pickled environment... not yet created
[dochtml] 
[dochtml] Internal or unrecoverable error in:
[dochtml] Got signal before environment was installed on our thread
[dochtml]   [2: No such file or directory]
[dochtml] 
[dochtml] ;;; ECL C Backtrace
[dochtml] ;;;    0 ecl_internal_error (0x89a790705) 
[dochtml] ;;;    1 init_unixint (0x89a7b6b70) 
[dochtml] ;;;    2 init_unixint (0x89a7b6912) 
[dochtml] ;;;    3 pthread_sigmask (0x80105779d) 
[dochtml] ;;;    4 pthread_getspecific (0x801056d6f) 
[dochtml] ;;;    5 unknown (0x7ffffffff193) 
[dochtml] ;;;    6 GC_push_all_stacks (0x89ab1b7aa) 
[dochtml] ;;;    7 GC_mark_some (0x89ab10a93) 
[dochtml] ;;;    8 GC_stopped_mark (0x89ab08b3a) 
[dochtml] ;;;    9 GC_try_to_collect_inner (0x89ab08a31) 
[dochtml] ;;;   10 GC_init (0x89ab14593) 
[dochtml] ;;;   11 init_alloc (0x89a7ca9f9) 
[dochtml] ;;;   12 cl_boot (0x89a694a4b) 
[dochtml] ;;;   13 initecl (0x89a218bc0) 
[dochtml] ;;;   14 initecl (0x89a20a9f5) 
[dochtml] ;;;   15 initecl (0x89a208678) 
[dochtml] ;;;   16 _PyImport_LoadDynamicModule (0x8009334fc) 
[dochtml] ;;;   17 PyImport_AppendInittab (0x800931eff) 
[dochtml] ;;;   18 PyImport_AppendInittab (0x8009319b4) 
[dochtml] ;;;   19 PyImport_ImportModuleLevel (0x800930abe) 
[dochtml] ;;;   20 _PyBuiltin_Init (0x80090b187) 
[dochtml] ;;;   21 PyObject_Call (0x800871454) 
[dochtml] ;;;   22 PyEval_EvalFrameEx (0x800915f47) 
[dochtml] ;;;   23 PyEval_EvalCodeEx (0x800910505) 
[dochtml] ;;;   24 PyEval_EvalCode (0x80090fcd6) 
[dochtml] ;;;   25 PyImport_ExecCodeModuleEx (0x80092f521) 
[dochtml] ;;;   26 PyImport_AppendInittab (0x800932598) 
[dochtml] ;;;   27 PyImport_AppendInittab (0x800931eff) 
[dochtml] ;;;   28 PyImport_AppendInittab (0x8009319b4) 
[dochtml] ;;;   29 PyImport_ImportModuleLevel (0x800930abe) 
[dochtml] ;;;   30 _PyBuiltin_Init (0x80090b187) 
[dochtml] ;;;   31 PyEval_EvalFrameEx (0x800917ab0) 

building docs with make -j1 works. Running tests now.

comment:36 Changed 4 years ago by dimpase

"interesting" error message: 10 such errors, leading to segfaults in total in testlong, all related to (un)picklings and libpng. What the meaning of "Application" in these messages? And what is the reason? Missing or wrong -I option for the compiler?

sage -t --long src/sage/coding/binary_code.pyx
    Killed due to segmentation fault
...
sage: from sage.coding.binary_code import * ## line 824 ##
sage: M = Matrix(GF(2), [[1,1,1,1]]) ## line 825 ##
sage: B = BinaryCode(M) ## line 826 ##
sage: loads(dumps(B)) == B ## line 827 ##
libpng warning: Application was compiled with png.h from libpng-1.6.27+apng
libpng warning: Application  is  running with png.c from libpng-1.2.51
GD Error: gd-png: fatal libpng error: Incompatible libpng version in application and library

The rest---for the early days like this---looks not too bad, apart from code related to elliptic curves (probably another well-localised bug, only not quite clear where):

sage/coding/binary_code.pyx  # Killed due to segmentation fault
sage/coding/linear_code.py  # Killed due to segmentation fault
sage/crypto/mq/sr.py  # Killed due to segmentation fault
sage/graphs/generic_graph.py  # Killed due to segmentation fault
sage/graphs/base/graph_backends.pyx  # Killed due to segmentation fault
sage/homology/chain_complex.py  # Killed due to illegal instruction
sage/interfaces/mwrank.py  # 5 doctests failed
sage/interfaces/singular.py  # 2 doctests failed
sage/interfaces/tests.py  # 1 doctest failed
sage/lfunctions/sympow.py  # 4 doctests failed
sage/libs/eclib/interface.py  # 34 doctests failed
sage/libs/eclib/mwrank.pyx  # 3 doctests failed
sage/matrix/matrix_gf2e_dense.pyx  # Killed due to segmentation fault
sage/matrix/matrix_mod2_dense.pyx  # Killed due to segmentation fault
sage/misc/benchmark.py  # 3 doctests failed
sage/numerical/sdp.pyx  # 1 doctest failed
sage/plot/graphics.py  # 1 doctest failed
sage/plot/plot.py  # 1 doctest failed
sage/rings/complex_double.pyx  # 2 doctests failed
sage/rings/padics/CR_template.pxi  # 2 doctests failed
sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to segmentation fault
sage/rings/polynomial/pbori.pyx  # Killed due to abort
sage/rings/polynomial/polydict.pyx  # 1 doctest failed
sage/rings/polynomial/polynomial_real_mpfr_dense.pyx  # 1 doctest failed
sage/schemes/elliptic_curves/BSD.py  # 23 doctests failed
sage/schemes/elliptic_curves/constructor.py  # 11 doctests failed
sage/schemes/elliptic_curves/ell_egros.py  # 9 doctests failed
sage/schemes/elliptic_curves/ell_generic.py  # 14 doctests failed
sage/schemes/elliptic_curves/ell_number_field.py  # 16 doctests failed
sage/schemes/elliptic_curves/ell_point.py  # 13 doctests failed
sage/schemes/elliptic_curves/ell_rational_field.py  # Timed out
sage/schemes/elliptic_curves/ell_tate_curve.py  # 4 doctests failed
sage/schemes/elliptic_curves/heegner.py  # 48 doctests failed
sage/schemes/elliptic_curves/height.py  # 2 doctests failed
sage/schemes/elliptic_curves/lseries_ell.py  # 2 doctests failed
sage/schemes/elliptic_curves/padic_lseries.py  # 2 doctests failed
sage/schemes/elliptic_curves/padics.py  # 34 doctests failed 
sage/schemes/elliptic_curves/sha_tate.py  # Killed due to segmentation fault
sage/structure/coerce.pyx  # 1 doctest failed
sage/structure/element.pyx  # 1 doctest failed
sage/structure/sage_object.pyx  # Killed due to bus error
sage/tests/benchmark.py  # 1 doctest failed
sage/tests/book_stein_ent.py  # 3 doctests failed
sage/tests/cmdline.py  # 3 doctests failed
doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
doc/en/developer/coding_basics.rst  # 1 doctest failed
doc/en/developer/coding_in_other.rst  # 1 doctest failed
doc/en/prep/Calculus.rst  # 1 doctest failed
doc/en/thematic_tutorials/explicit_methods_in_number_theory/elliptic_curves.rst  # 11 doctests failed
Last edited 4 years ago by dimpase (previous) (diff)

comment:37 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554, #22692 to #22687, #22643, #22554, #22692, #22159

libpng-related segfaults are "cured" by #22159 ; this is of course fixing the symptom, not the root cause---as #22159 happens to carry the same version of libpng as the system-wide version.

comment:38 follow-up: Changed 4 years ago by fbissey

It is strange I would have expected runpath to cover you for that. If runpath don't allow to load a library other than the one in the system default location, what's the point? "Application" looks like it is coming from inside libpng itself, so that's an helpful message from png devs.

Do you have readelf available? If so could we have a look at readelf -d local/lib/python/site-packages/sage/coding/binary_code.so when it is in a failing state?

comment:39 Changed 4 years ago by dimpase

after applying #22159 branch, I got

sage/crypto/mq/sr.py  # 7 doctests failed
sage/interfaces/mwrank.py  # 5 doctests failed
sage/interfaces/singular.py  # 2 doctests failed
sage/interfaces/tests.py  # 1 doctest failed
sage/lfunctions/sympow.py  # 4 doctests failed
sage/libs/eclib/interface.py  # 34 doctests failed
sage/libs/eclib/mwrank.pyx  # 3 doctests failed 
sage/misc/benchmark.py  # 3 doctests failed  
sage/numerical/sdp.pyx  # 1 doctest failed
sage/plot/graphics.py  # 1 doctest failed
sage/plot/plot.py  # 1 doctest failed
sage/rings/complex_double.pyx  # 2 doctests failed
sage/rings/padics/CR_template.pxi  # 2 doctests failed 
sage/rings/polynomial/multi_polynomial_sequence.py  # Killed due to segmentation fault
sage/rings/polynomial/pbori.pyx  # Killed due to abort
sage/rings/polynomial/polydict.pyx  # 1 doctest failed
sage/rings/polynomial/polynomial_real_mpfr_dense.pyx  # 1 doctest failed
sage/schemes/elliptic_curves/BSD.py  # 23 doctests failed
sage/schemes/elliptic_curves/constructor.py  # 11 doctests failed
sage/schemes/elliptic_curves/ell_egros.py  # 9 doctests failed
sage/schemes/elliptic_curves/ell_generic.py  # 14 doctests failed
sage/schemes/elliptic_curves/ell_number_field.py  # 16 doctests failed
sage/schemes/elliptic_curves/ell_point.py  # 13 doctests failed 
sage/schemes/elliptic_curves/ell_rational_field.py  # 61 doctests failed
sage/schemes/elliptic_curves/ell_tate_curve.py  # 4 doctests failed
sage/schemes/elliptic_curves/heegner.py  # 48 doctests failed
sage/schemes/elliptic_curves/height.py  # 2 doctests failed
sage/schemes/elliptic_curves/lseries_ell.py  # 2 doctests failed
sage/schemes/elliptic_curves/padic_lseries.py  # 2 doctests failed 
sage/schemes/elliptic_curves/padics.py  # 34 doctests failed
sage/schemes/elliptic_curves/sha_tate.py  # Killed due to segmentation fault
sage/structure/coerce.pyx  # 1 doctest failed
sage/structure/element.pyx  # 1 doctest failed
sage/structure/sage_object.pyx  # Killed due to abort
sage/tests/benchmark.py  # 1 doctest failed
sage/tests/book_stein_ent.py  # 3 doctests failed 
sage/tests/cmdline.py  # 3 doctests failed
doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
doc/en/developer/coding_basics.rst  # 1 doctest failed
doc/en/prep/Calculus.rst  # 1 doctest failed
doc/en/thematic_tutorials/explicit_methods_in_number_theory/elliptic_curves.rst  # 11 doctests failed

comment:40 in reply to: ↑ 38 Changed 4 years ago by dimpase

Replying to fbissey:

It is strange I would have expected runpath to cover you for that. If runpath don't allow to load a library other than the one in the system default location, what's the point? "Application" looks like it is coming from inside libpng itself, so that's an helpful message from png devs.

OK, I'll rebuild without #22159 and post the outputs of readelf and ldd. Anyway, with #22159 I get

$ ldd local/lib/python/site-packages/sage/coding/binary_code.so
local/lib/python/site-packages/sage/coding/binary_code.so:
        libm.so => /usr/home/dima/Sage/sage/local/lib/libm.so (0x801256000)
        libgmp.so.22 => /usr/home/dima/Sage/sage/local/lib/libgmp.so.22 (0x801462000)
        libpython2.7.so.1 => /usr/home/dima/Sage/sage/local/lib/libpython2.7.so.1 (0x8016e6000)
        libpari-gmp.so.5 => /usr/home/dima/Sage/sage/local/lib/libpari-gmp.so.5 (0x801c00000)
        libthr.so.3 => /lib/libthr.so.3 (0x802678000)
        libc.so.7 => /lib/libc.so.7 (0x800823000)
        libm.so.5 => /lib/libm.so.5 (0x80289f000)
        libutil.so.9 => /lib/libutil.so.9 (0x802aca000)

readelf:

0x0000000000000001 (NEEDED)             Shared library: [libm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgmp.so.22]
 0x0000000000000001 (NEEDED)             Shared library: [libpython2.7.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpari-gmp.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libthr.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]
 0x000000000000000f (RPATH)              Library rpath: [/usr/home/dima/Sage/sage/local/lib]
 0x000000000000001d (RUNPATH)            Library runpath: [/usr/home/dima/Sage/sage/local/lib]

so there is no sign of libpng anywhere; I looked further in Python's dynamically loaded modules in local/lib/python2.7/lib-dynload/, but even there I do not see libpng.

What does load it, and how? Is it a sort of dynamic loading (dlopen?) that is not recorded by the linker? Would one have to resort to analysing a dump or running under gdb? Uh-oh...

comment:41 follow-up: Changed 4 years ago by fbissey

1) note that it says GD Error: gd-png: fatal libpng error: Incompatible libpng version in application and library so we are talking about something using libgd.

2) the error may not be in that file but in another part of sage that is called from that file.

Check sage/matrix/matrix_mod2_dense.so.

comment:42 Changed 4 years ago by fbissey

Actually there may even be a gd-png executable. I have the following executables installed by gd

/usr/bin/annotate
/usr/bin/bdftogd
/usr/bin/gd2copypal
/usr/bin/gd2togif
/usr/bin/gd2topng
/usr/bin/gdcmpgif
/usr/bin/gdlib-config
/usr/bin/gdparttopng
/usr/bin/gdtopng
/usr/bin/giftogd2
/usr/bin/pngtogd
/usr/bin/pngtogd2
/usr/bin/webpng

comment:43 Changed 4 years ago by git

  • Commit changed from a391ad68175a75057647230aedd758e8a648f03b to ed47d31a65374521bdf9ee09c292adca188965a4

Branch pushed to git repo; I updated commit sha1. New commits:

7467913Upgrade to libpng-1.6.28
55fe881Merge branch 'u/jdemeyer/upgrade_to_latest_libpng' of trac.sagemath.org:sage into sqliteupdate
ed47d31numpy workarond to skip cephes' complex.h

comment:44 in reply to: ↑ 41 Changed 4 years ago by dimpase

Replying to fbissey:

1) note that it says GD Error: gd-png: fatal libpng error: Incompatible libpng version in application and library so we are talking about something using libgd.

2) the error may not be in that file but in another part of sage that is called from that file.

Check sage/matrix/matrix_mod2_dense.so.

OK, I've rebuilt with old (Sage's current) libpng:

$ ldd local/lib/python2.7/site-packages/sage/matrix/matrix_mod2_dense.so
local/lib/python2.7/site-packages/sage/matrix/matrix_mod2_dense.so:
        libm.so => /usr/home/dima/Sage/sage/local/lib/libm.so (0x801255000)
        libgmp.so.22 => /usr/home/dima/Sage/sage/local/lib/libgmp.so.22 (0x801461000)
        libm4ri-0.0.20140914.so => /usr/home/dima/Sage/sage/local/lib/libm4ri-0.0.20140914.so (0x8016e5000)
        libpng12.so.0 => /usr/home/dima/Sage/sage/local/lib/libpng12.so.0 (0x801917000)
        libz.so => /usr/home/dima/Sage/sage/local/lib/libz.so (0x801b40000)
        libpython2.7.so.1 => /usr/home/dima/Sage/sage/local/lib/libpython2.7.so.1 (0x801d57000)
        libpari-gmp.so.5 => /usr/home/dima/Sage/sage/local/lib/libpari-gmp.so.5 (0x802200000)
        libthr.so.3 => /lib/libthr.so.3 (0x802c78000)
        libc.so.7 => /lib/libc.so.7 (0x800823000)
        libm.so.5 => /lib/libm.so.5 (0x802e9f000)
        libutil.so.9 => /lib/libutil.so.9 (0x8030ca000)

And got these errors back, too:

sage -t --long src/sage/matrix/matrix_mod2_dense.pyx  # Killed due to illegal instruction
...
sage: b*a ## line 39 ##
[1 0 1]
[1 0 0]
sage: TestSuite(a).run() ## line 43 ##
libpng warning: Application was compiled with png.h from libpng-1.6.27+apng
libpng warning: Application  is  running with png.c from libpng-1.2.51
GD Error: gd-png: fatal libpng error: Incompatible libpng version in application and library

That is, if we are trusting the error message, somewhere wrong headers (and a library, I suppose) were used for compilation, isn't it?

comment:45 Changed 4 years ago by dimpase

this is how to reproduce it at the Sage prompt, with more details (pickling is the problem):

sage: from sage.misc.all import loads, dumps
sage: a=matrix(GF(2),3,range(9),sparse=False); a

[0 1 0]
[1 0 1]
[0 1 0]
sage: dumps(a)
libpng warning: Application was compiled with png.h from libpng-1.6.27+apng
...
Segmentation fault (core dumped)

digging out what dumps does gives

sage: from six.moves import cPickle
sage: # or just import cPickle # makes no difference
sage: a=matrix(GF(2),3,range(9),sparse=False)
sage: cPickle.dumps(a)
libpng warning: Application was compiled with png.h from libpng-1.6.27+apng
...

digging more gives:

sage: a=matrix(GF(2),3,range(9),sparse=False)
sage: from sage.matrix.matrix_mod2_dense import to_png
sage: to_png(a,"/tmp/blah")
libpng warning: Application was compiled with png.h from libpng-1.6.27+apng
libpng warning: Application  is  running with png.c from libpng-1.2.51
GD Error: gd-png: fatal libpng error: Incompatible libpng version in application and library

to_png calls gdImage* things from libgd, which in turn call gd_png; this uses libpng. Now one can see the cause of trouble:

it's a bug in libgd configure, which places -I/usr/local/include in front of -I$SAGE_LOCAL/include/...; more precisely, there is an old FreeBSD-specific patch in its spkg-install, which causes this thing, 4cf1a0c642a.

Removing it fixes this problem. Duh...

comment:46 Changed 4 years ago by git

  • Commit changed from ed47d31a65374521bdf9ee09c292adca188965a4 to 82de057129d6c61a8898bcd0327e3dfb400456ff

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

9b783c8numpy workarond to skip cephes' complex.h
82de057not needed on FreeBSD, and causes misconfig

comment:47 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554, #22692, #22159 to #22687, #22643, #22554, #22692

comment:48 Changed 4 years ago by git

  • Commit changed from 82de057129d6c61a8898bcd0327e3dfb400456ff to d63b82685e0faf34afe4717a16abab9e75f84363

Branch pushed to git repo; I updated commit sha1. New commits:

d63b826more C++ standard conformance

comment:49 follow-ups: Changed 4 years ago by fbissey

You should split the libgd commit to its own ticket. It really should be fixed. Is your eclib stuff in upstream already? Or in an issue on github?

comment:50 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554, #22692 to #22687, #22643, #22554, #22692, #22711

fixed few ugly c++ bugs in eclib in #22711. This makes clang much happier, many more doctests work.

comment:51 in reply to: ↑ 49 Changed 4 years ago by dimpase

Replying to fbissey:

You should split the libgd commit to its own ticket. It really should be fixed.

well, it's FreeBSD-only issue, it's OK to wait until more stuff for it is ready...

comment:52 Changed 4 years ago by dimpase

Now it's down to something relatively small---mostly numerical noise:

src/sage/rings/polynomial/multi_polynomial_sequence.py  # Timed out (with segmentation fault after interrupt)
src/sage/symbolic/series.pyx  # Killed due to kill signal
src/sage/rings/polynomial/pbori.pyx  # Killed due to abort
src/sage/plot/plot.py  # 1 doctest failed
src/sage/schemes/elliptic_curves/ell_rational_field.py  # 2 doctests failed
src/sage/plot/graphics.py  # 1 doctest failed
src/sage/tests/cmdline.py  # 1 doctest failed
src/sage/interfaces/tests.py  # Timed out
src/sage/structure/element.pyx  # 1 doctest failed
src/doc/en/prep/Calculus.rst  # 1 doctest failed
src/sage/doctest/forker.py  # 1 doctest failed
src/sage/structure/sage_object.pyx  # Timed out (with abort after interrupt)
src/sage/crypto/mq/sr.py  # 7 doctests failed
src/sage/schemes/elliptic_curves/lseries_ell.py  # 1 doctest failed
src/sage/interfaces/singular.py  # 2 doctests failed
src/sage/lfunctions/sympow.py  # 4 doctests failed
src/sage/structure/coerce.pyx  # 1 doctest failed
src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
src/sage/rings/complex_double.pyx  # 2 doctests failed
src/sage/numerical/sdp.pyx  # 1 doctest failed
src/sage/rings/polynomial/polynomial_real_mpfr_dense.pyx  # 1 doctest failed
src/sage/rings/polynomial/polydict.pyx  # 1 doctest failed

comment:53 Changed 4 years ago by git

  • Commit changed from d63b82685e0faf34afe4717a16abab9e75f84363 to 118429ba062ede124cba25ad04e69f4a81a728bb

Branch pushed to git repo; I updated commit sha1. New commits:

118429bbrial updated to commit 63f747626822f5e0fa2bf975f7801fcc988eb530

comment:54 follow-up: Changed 4 years ago by fbissey

Dima, how are you skipping the building of gcc? Or you don't like I do on OS X?

comment:55 in reply to: ↑ 54 Changed 4 years ago by dimpase

Replying to fbissey:

Dima, how are you skipping the building of gcc? Or you don't like I do on OS X?

It's easy in this case. I've installed FreeBSD's package gcc6, which comes, well, with gcc-6.3, and does not install gcc or gfortran. Then I made a symbolic link pointing gfortran to gfortran6. And building with SAGE_INSTALL_GCC=no does the trick.

PS. I'm on vacation for another week, replies might be slow :-)

comment:56 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554, #22692, #22711 to #22687, #22643, #22554, #22692, #22711, #22677

comment:57 Changed 4 years ago by git

  • Commit changed from 118429ba062ede124cba25ad04e69f4a81a728bb to 27e7ae2281fbac817e895e1ed9dcb76ad938ddab

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

ca304a8Merge branch 'u/fbissey/brial-0.8.6' of trac.sagemath.org:sage into testpng
5cd7e91delete...
6099ae4update to brial 0.8.7
4abc444Merge branch 'u/dimpase/sqliteupdate' of trac.sagemath.org:sage into develop
7677635Merge branch 'u/malb/22643-fplll-upgrade' of trac.sagemath.org:sage into develop
7c6c479merging sage 8.0.beta0 and new eclib
382b25cupdate to brial 0.8.7
cc8cb1cMerge branch 'u/dimpase/brial086' of trac.sagemath.org:sage into freebsdport
0b2b060LONG_LONG_MAX is an obsolete gnu-ism
27e7ae2Merge branch 'u/dimpase/llongfix' of trac.sagemath.org:sage into freebsdport

comment:58 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554, #22692, #22711, #22677 to #22687, #22643, #22554, #22692, #22711, #22677, #22759, #22714

comment:59 Changed 4 years ago by dimpase

  • Dependencies changed from #22687, #22643, #22554, #22692, #22711, #22677, #22759, #22714 to #21977, #22677, #22759, #22714

comment:60 Changed 4 years ago by dimpase

  • Dependencies changed from #21977, #22677, #22759, #22714 to #21977, #22677, #22759, #22714, #22784
  • Description modified (diff)

comment:61 Changed 4 years ago by dimpase

  • Dependencies changed from #21977, #22677, #22759, #22714, #22784 to #21977, #22677, #22759, #22714, #22784, #22785
  • Description modified (diff)

comment:62 in reply to: ↑ 49 Changed 4 years ago by dimpase

Replying to fbissey:

You should split the libgd commit to its own ticket. It really should be fixed.

this is #22785, ready for review.

comment:63 Changed 4 years ago by dimpase

  • Dependencies changed from #21977, #22677, #22759, #22714, #22784, #22785 to #21977, #22677, #22759, #22714, #22784, #22785, #22786
  • Description modified (diff)

comment:64 follow-up: Changed 4 years ago by dimpase

sympow tests of analytic_rank fail on curves of even rank, and works for odd rank, does not matter whether sympow is built with clang 3.8 or with gcc 6.3.

Using clang-devel, which is clang-4.0.0, makes the sympow tests pass. This is probably due to it using llvm-as rather than as, with is horribly old (version 2.17, dates back to 2007) on FreeBSD.

Looks like we're converging to the state we have on OSX w.r.t. clang (#12426).

comment:65 in reply to: ↑ 64 Changed 4 years ago by dimpase

Replying to dimpase:

Using clang-devel, which is clang-4.0.0, makes the sympow tests pass. This is probably due to it using llvm-as rather than as, with is horribly old (version 2.17, dates back to 2007) on FreeBSD.

Alternatively, using /usr/local/bin/as (version 2.28) instead, makes sympow tests pass, too.

comment:66 Changed 4 years ago by dimpase

  • Description modified (diff)

comment:67 Changed 4 years ago by dimpase

  • Dependencies changed from #21977, #22677, #22759, #22714, #22784, #22785, #22786 to #22759, #22786

comment:68 Changed 4 years ago by dimpase

ccache, gfan,lcalc,tachyon need to have their LDFLAGS sorted due to libm linking, as our special LDFLAGS must come first in the linking list of parameters. Worked around for the time being by adding LDFLAGS to CC and CXX, i.e.

$ CXX="$CXX $LDFLAGS" CC="$CC $LDFLAGS" make 

however, it seems that passing unrecognised options to the compiler has strange side effects on sympow - it creates an executable that fails tests it fails if built with old as (sic!).

Last edited 4 years ago by dimpase (previous) (diff)

comment:69 Changed 4 years ago by git

  • Commit changed from 27e7ae2281fbac817e895e1ed9dcb76ad938ddab to 73f9f7c30a016963175f0d7959e2bc938ff0cd48

Branch pushed to git repo; I updated commit sha1. New commits:

7259995Merge branch 'develop' of trac.sagemath.org:sage into develop
e6fb7b0Merge branch 'u/dimpase/freebsd_experimental' of trac.sagemath.org:sage into beta2
d2624dcworkaround to skip cephes' complex.h, bump up version
a7bac75Merge branch 'u/jdemeyer/clean_up_numpy_s_spkg_install' of trac.sagemath.org:sage into numpy_freebsd_workaround
dbdc716moved -D from CC to CFLAGS
0ab0707Merge branch 'u/dimpase/numpy_freebsd_workaround' of trac.sagemath.org:sage into beta2
e53c199Merge branch 'u/jdemeyer/factorize_result_of_algdep__' of trac.sagemath.org:sage into dcfix
fd96c0afix for #22759
8fd9de6Merge branch 'u/dimpase/t22759' of trac.sagemath.org:sage into beta2
73f9f7cdelete obsolete patch

comment:70 Changed 4 years ago by dimpase

  • Dependencies changed from #22759, #22786 to #22759, #22786, #22812, #22840

#22812 and #22840 deal with misplaced LDFLAGS.

comment:71 Changed 4 years ago by dimpase

Apart from #22799, which is responsible for majority of failing doctests, the only other real worry is the double unpickling crash after the line sage.structure.sage_object.unpickle_all() ## line 1563 ## in src/sage/structure/sage_object.pyx.

comment:72 Changed 4 years ago by dimpase

  • Dependencies changed from #22759, #22786, #22812, #22840 to #22895

comment:73 Changed 4 years ago by dimpase

TODO: gf2x needs CPP and CXXCPP to be set to "clang-devel -E" if we like to run its tuning.

(and probably a fresh autoreconf run)

Last edited 4 years ago by dimpase (previous) (diff)

comment:74 Changed 4 years ago by fbissey

clang-devel? is this a freeBSDism? No such thing on OS X. AC_PROG_CPP would usually test $CC -E (or $CXX -E depending on the language selected).

comment:75 Changed 4 years ago by dimpase

clang-devel == clang 4.0.0 on FreeBSD 11.0.

comment:76 Changed 4 years ago by dimpase

  • Dependencies changed from #22895 to #22582

comment:77 Changed 4 years ago by dimpase

  • Branch changed from u/dimpase/freebsd_experimental to u/dimpase/fbsd
  • Commit changed from 73f9f7c30a016963175f0d7959e2bc938ff0cd48 to 3debd116304b0c3c16624d57950418670e1c7ffd
  • Dependencies changed from #22582 to #12426
  • Description modified (diff)

comment:78 Changed 4 years ago by dimpase

ptest now gives

sage -t --warn-long 54.2 src/sage/structure/sage_object.pyx  # Killed due to abort
sage -t --warn-long 54.2 src/sage/interfaces/tests.py  # 1 doctest failed
sage -t --warn-long 54.2 src/sage/tests/cmdline.py  # 1 doctest failed
sage -t --warn-long 54.2 src/sage/homology/simplicial_complex.py  # 1 doctest failed
sage -t --warn-long 54.2 src/sage/interfaces/singular.py  # 2 doctests failed
sage -t --warn-long 54.2 src/sage/libs/lcalc/lcalc_Lfunction.pyx  # 1 doctest failed
sage -t --warn-long 54.2 src/doc/en/developer/coding_in_other.rst  # 1 doctest failed
sage -t --warn-long 54.2 src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
sage -t --warn-long 54.2 src/sage/numerical/sdp.pyx  # 1 doctest failed

all of this was seen before --- numerical noise or a different order of output, random weird failures, or crash at 2nd unpickle.all.

comment:79 Changed 4 years ago by git

  • Commit changed from 3debd116304b0c3c16624d57950418670e1c7ffd to 96e6482a9ff5cd637fad6f4c89414f2066115dfb

Branch pushed to git repo; I updated commit sha1. New commits:

96e6482fix linking of libflint and libarb on FreeBSD

comment:80 Changed 4 years ago by dimpase

  • Description modified (diff)

comment:81 Changed 4 years ago by dimpase

wow, now ptest gives just this:

sage -t --warn-long 50.1 src/sage/interfaces/tests.py  # 1 doctest failed
sage -t --warn-long 50.1 src/sage/tests/cmdline.py  # 1 doctest failed
sage -t --warn-long 50.1 src/sage/homology/simplicial_complex.py  # 1 doctest failed
sage -t --warn-long 50.1 src/sage/geometry/cone.py  # Timed out
sage -t --warn-long 50.1 src/sage/numerical/sdp.pyx  # 1 doctest failed

(with the timed out test running fine if repeated separately)

Apparently it's due to flint and arb properly set up at the right moment, and not at some later moment, and/or more stable clang version.

comment:82 Changed 4 years ago by git

  • Commit changed from 96e6482a9ff5cd637fad6f4c89414f2066115dfb to 057e47e5a4749597175521b9539f9b5fcad4d984

Branch pushed to git repo; I updated commit sha1. New commits:

057e47emerge of rc0

comment:83 Changed 4 years ago by dimpase

  • Milestone changed from sage-8.0 to sage-8.1

comment:84 Changed 4 years ago by dimpase

  • Dependencies changed from #12426 to #12426, #23565

New branch is u/dimpase/fbsd-support, keeping the old one for browsing purposes a.t.m.

comment:85 Changed 4 years ago by dimpase

  • Branch changed from u/dimpase/fbsd to u/dimpase/fbsd-support
  • Commit changed from 057e47e5a4749597175521b9539f9b5fcad4d984 to 94e52eaa09b4a4a3ea4759630ae08151c18e4c64

comment:86 Changed 4 years ago by git

  • Commit changed from 94e52eaa09b4a4a3ea4759630ae08151c18e4c64 to d04c9f7b792dc9c43f07fa4446241862d088ff2c

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

7c8c3e3Merge branch 'libatomic_ops' into gcupdate
277ddd7libatomic_ops is a dependency
4d5bbd0libatomic_ops 7.4.6 added as standard pkg
e53f784Merge branch 'u/dimpase/libatomic_ops' of trac.sagemath.org:sage into gcupdate
4929b58Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81
8166e21libatomic_ops 7.6.4 added as standard pkg
64da17fMerge branch 'libatomic_ops' into gcupdate
82eabaepatch for FreeBSD
931652aMerge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into gcupdate
d04c9f7Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81

comment:87 Changed 4 years ago by dimpase

  • Dependencies changed from #12426, #23565 to #12426, #23565, #23700

now MAKE="make -j1 make ptest has just 3 failures:

sage -t --warn-long 79.3 src/sage/interfaces/tests.py  # 1 doctest failed
sage -t --warn-long 79.3 src/sage/numerical/sdp.pyx  # 1 doctest failed
sage -t --warn-long 79.3 src/sage/tests/cmdline.py  # 1 doctest failed

of which the 1st is something to look at:

File "src/sage/interfaces/tests.py", line 36, in sage.interfaces.tests
Failed example:
    subprocess.call("echo syntax error | gap", **kwds)
Expected:
    0
Got:
    136

2nd is just numerical noise, the 3rd is

File "src/sage/tests/cmdline.py", line 534, in sage.tests.cmdline.test_executable
Failed example:
    out
Expected:
    '120\n'
Got:
    '#E  Could not create interval timer. Timeouts will not be supported\n120\n'

A bigger problem is with -jX for X>1; it's not really possible to build docs, one gets weird segfaults related to ECL/Maxima. It's very likely that they are of the same nature as the following weirdness (they look very similar): start Sage, type

sage: from sage.interfaces.maxima_lib import 

and hit Tab. This segfaults with

Internal or unrecoverable error in:                               
Got signal before environment was installed on our thread         
  [2: No such file or directory]                                  

;;; ECL C Backtrace
;;;    0 ecl_internal_error (0x87d790765) 
;;;    1 init_unixint (0x87d7b6bd0) 
;;;    2 init_unixint (0x87d7b6972) 
;;;    3 pthread_sigmask (0x80103779d) 
;;;    4 pthread_getspecific (0x801036d6f) 
;;;    5 unknown (0x7ffffffff193) 
;;;    6 GC_push_all_stacks (0x87db1ea2c) 
;;;    7 GC_mark_some (0x87db12eec) 
;;;    8 GC_stopped_mark (0x87db09baa) 
;;;    9 GC_try_to_collect_inner (0x87db09a75) 
;;;   10 GC_init (0x87db16f4f) 
;;;   11 init_alloc (0x87d7caa59) 
;;;   12 cl_boot (0x87d694a5b) 
;;;   13 initecl (0x87d218340) 
;;;   14 initecl (0x87d20a43f) 
;;;   15 initecl (0x87d207e28) 
;;;   16 _PyImport_LoadDynamicModule (0x800b3ed1c) 
;;;   17 PyImport_AppendInittab (0x800b3d71f) 
;;;   18 PyImport_AppendInittab (0x800b3d1a8) 
;;;   19 PyImport_ImportModuleLevel (0x800b3c2ce) 
;;;   20 _PyBuiltin_Init (0x800b162d7) 
;;;   21 PyObject_Call (0x800a7d3e3) 
;;;   22 PyEval_EvalFrameEx (0x800b2121c) 
;;;   23 PyEval_EvalCodeEx (0x800b1b5d4) 
;;;   24 PyEval_EvalCode (0x800b1ad96) 
;;;   25 PyImport_ExecCodeModuleEx (0x800b3ad11) 
;;;   26 PyImport_AppendInittab (0x800b3ddb8) 
;;;   27 PyImport_AppendInittab (0x800b3d71f) 
;;;   28 PyImport_AppendInittab (0x800b3d1a8) 
;;;   29 PyImport_ImportModuleLevel (0x800b3c2ce) 
;;;   30 _PyBuiltin_Init (0x800b162d7) 
;;;   31 PyEval_EvalFrameEx (0x800b22dd1) 
Segmentation fault (core dumped)

Last 10 new commits:

7c8c3e3Merge branch 'libatomic_ops' into gcupdate
277ddd7libatomic_ops is a dependency
4d5bbd0libatomic_ops 7.4.6 added as standard pkg
e53f784Merge branch 'u/dimpase/libatomic_ops' of trac.sagemath.org:sage into gcupdate
4929b58Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81
8166e21libatomic_ops 7.6.4 added as standard pkg
64da17fMerge branch 'libatomic_ops' into gcupdate
82eabaepatch for FreeBSD
931652aMerge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into gcupdate
d04c9f7Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81

comment:88 Changed 4 years ago by dimpase

  • Dependencies changed from #12426, #23565, #23700 to #12426, #23565
  • Description modified (diff)

Last 10 new commits:

7c8c3e3Merge branch 'libatomic_ops' into gcupdate
277ddd7libatomic_ops is a dependency
4d5bbd0libatomic_ops 7.4.6 added as standard pkg
e53f784Merge branch 'u/dimpase/libatomic_ops' of trac.sagemath.org:sage into gcupdate
4929b58Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81
8166e21libatomic_ops 7.6.4 added as standard pkg
64da17fMerge branch 'libatomic_ops' into gcupdate
82eabaepatch for FreeBSD
931652aMerge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into gcupdate
d04c9f7Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81

comment:89 Changed 4 years ago by dimpase

  • Dependencies changed from #12426, #23565 to #12426, #23565, #23700
  • Description modified (diff)

ticket description should be edited on an updated ticket page, duh...

comment:90 Changed 4 years ago by dimpase

  • Description modified (diff)

The segfault in the bottom part of comment 87 remains the same if gc and libatomic_ops are rebuilt with threads disabled.

comment:91 Changed 4 years ago by git

  • Commit changed from d04c9f7b792dc9c43f07fa4446241862d088ff2c to a7bf51fc42bf738f69336029afe139397e94b57b

Branch pushed to git repo; I updated commit sha1. New commits:

eacd3d6Merge branch 'develop' of trac.sagemath.org:sage into fbsdon81
6f17c02Merge branch 'compiler' into clang-progress
ceb3bf5conform to new packaging in place since 8.1.beta1
07bfc5aMerge branch 'compiler' into clang-progress
d8cb828Merge branch 'compiler' into clang-progress
0ec453bMerge branch 'develop' into clang-progress
a7bf51fMerge branch 'u/fbissey/clang-progress' of trac.sagemath.org:sage into fbsdon81

comment:92 Changed 4 years ago by git

  • Commit changed from a7bf51fc42bf738f69336029afe139397e94b57b to 4aaeeb491635abea35e2804eeee7c93a80af38ea

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

f9ca589Merge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into gc76
e365c32backported patch for FreeBSD support
c98c98cMerge branch 'u/dimpase/gcupdate' of trac.sagemath.org:sage into fbsdon81
83815b2Merge branch 'develop' into clang-progress
40d3214Merge branch 'develop' into clang-progress
6902166Simplify gfortran spkg-install a bit. Thanks to Jeroen Demeyer for the feedback on this.
afca2b6no need to be so verbose about gfortran.
235c09eIncluding upstream PR 29 to compile with xcode 9 until we have a new release.
48462eaMerge branch 'eclib-xcode9' into clang-progress
4aaeeb4Merge branch 'u/fbissey/clang-progress' of trac.sagemath.org:sage into fbsdon81

comment:93 Changed 4 years ago by dimpase

New code for RiemannSurface uses Voronoi from scipy. But the latter is broken on FreeBSD:

$ ./sage --python
Python 2.7.13 (default, Oct 11 2017, 17:41:18) 
[GCC 4.2.1 Compatible Clang 6.0.0 ] on freebsd11
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> points = np.array([[0, 0], [0, 1], [0, 2], [1, 0], [1, 1], [1, 2],[2, 0], [2, 1], [2, 2]])
>>> from scipy.spatial import Voronoi
>>> Voronoi(points)
Segmentation fault (core dumped)

(might be due to careless use of threading...)

Note that the previously used in Sage version of scipy 0.17.1, still works.

Last edited 3 years ago by dimpase (previous) (diff)

comment:94 Changed 4 years ago by dimpase

  • Description modified (diff)

it turns out that scipy's Voronoi works with the system's Python 2.7.14, but not with Sage's one (there are BSD-only Python patches; trying them now).

comment:95 Changed 4 years ago by dimpase

  • Dependencies changed from #12426, #23565, #23700 to #12426, #23700

comment:96 Changed 3 years ago by dimpase

  • Milestone changed from sage-8.1 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review
  • Type changed from enhancement to task

Please see #26249 for a followup, for FreeBSD version 12, where things are considerably simpler.

comment:97 Changed 3 years ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

comment:98 Changed 3 years ago by jdemeyer

  • Resolution set to wontfix
  • Status changed from positive_review to closed

comment:99 Changed 2 years ago by dimpase

  • Description modified (diff)
Note: See TracTickets for help on using tickets.