Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#27212 closed enhancement (fixed)

spkg-configure.m4 for gmp and mpir

Reported by: dimpase Owned by: vbraun
Priority: major Milestone: sage-8.8
Component: packages: optional Keywords: spkg-configure
Cc: embray Merged in:
Authors: Erik Bray, Dima Pasechnik Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: e735b4b (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by dimpase)

This should be easy, and it's a potential source of library conflicts, as gmp is often installed...

note that GMP is pulled in via ./configure --with-mp=gmp (as well as by specifying SAGE_MP_LIBRARY?)

Configure: http://users.ox.ac.uk/~coml0531/sage/configure-316.tar.gz

Attachments (2)

mp_star.patch (1.3 KB) - added by dimpase 3 years ago.
configure-314.tar.gz (106.5 KB) - added by embray 3 years ago.

Download all attachments as: .zip

Change History (126)

comment:1 Changed 3 years ago by dimpase

  • Authors set to Dima Pasechnik
  • Branch set to u/dimpase/packages/gmp_m4
  • Commit set to 1ed38e19baea5feb0d2f5d79e7989a7e80440a21
  • Status changed from new to needs_review

here we check that GMP is new enough, by verifying it has a new function. The rest is just copypaste.

To actually build with GMP, one needs

./configure --with-mp=gmp

(configure does the test regardless)


New commits:

1ed38e1spkg-configure for GMP

comment:2 Changed 3 years ago by dimpase

  • Status changed from needs_review to needs_work

I suppose due to a special treatment of GMP/MPIR, despite configure reporting GMP as not being installed, it still gets picked up for installation from source. (I tested this on a clean build...)

comment:3 follow-up: Changed 3 years ago by embray

Yes, I've already put some thought into how to handle this case, though I don't have a solid answer. The spkg-configure.m4 seems fine, but we can't just leave it at that.

The tricky thing is that with MPIR it can be compiled with or without gmp-compatibility mode. On Sage we use --with-gmpcompat so if we link to a gmp we're actually getting the one installed by MPIR. But some systems might have MPIR built without gmp-compatiblity, and yet no separate libgmp.so. So in that case we'd actually want to use libmpir (if such a system exists I'm not sure).

So if we run ./configure with no --with-mp= flag the question is what to do? Do we just try to detect any "libgmp" and use that if found? Do look for a libmpir, ever? If the user passes --with-mp=mpir or --with-mp=gmp should that imply that it should just build the Sage SPKG, or do we explicitly go searching for, say, a libmpir in the former case?

Last edited 3 years ago by embray (previous) (diff)

comment:4 Changed 3 years ago by embray

Deciding which BLAS to use from the system raises similar questions.

comment:5 Changed 3 years ago by embray

The more I think about it, the more I'm inclined toward --with-mp=mpir should mean that the MP library must by MPIR by explicit request of the user, and if not found on the system then we should build the SPKG.

If --with-mp= is not otherwise specified then we can take whatever MP lib was found on the system.

We should also add an spkg-configure.m4 for MPIR. We can impose an ordering (e.g. first check for MPIR, and then for plain GMP) by putting AC_REQURE(SAGE_SPKG_CONFIGURE_MPIR) at the beginning of the spkg-configure.m4 for gmp. I did something similar for gfortran/gcc and seems to work.

Last edited 3 years ago by embray (previous) (diff)

comment:6 Changed 3 years ago by embray

It should also be noted that we have also in the past patched other SPKGS (see e.g. #21188) to always link with -lgmp and #include <gmp.h> (and never use -lmpir and #include <mpir.h> respectively).

I'm not 100% convinced that that's necessary, though certainly one or the other should be used. So it's probably best to always ensure that there is at least a working libgmp and a gmp.h. The question then is whether or not it's possible to determine if that libgmp happened to be provided by MPIR.

Update: It seems that a libgmp.so provided by MPIR does contain a __mpir_version symbol, so that's something we can look for.

Last edited 3 years ago by embray (previous) (diff)

comment:7 Changed 3 years ago by embray

It seems MPIR is not in fact packaged for Debian (not sure about other distros): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=708391 I had just assumed it would be a question of using either MPIR or GMP (in other words, they conflict with each other, and installing either one or the other is possible, as is the case in Sage).

comment:8 in reply to: ↑ 3 Changed 3 years ago by dimpase

Replying to embray:

Yes, I've already put some thought into how to handle this case, though I don't have a solid answer. The spkg-configure.m4 seems fine, but we can't just leave it at that.

The tricky thing is that with MPIR it can be compiled with or without gmp-compatibility mode. On Sage we use --with-gmpcompat so if we link to a gmp we're actually getting the one installed by MPIR. But some systems might have MPIR built without gmp-compatiblity, and yet no separate libgmp.so. So in that case we'd actually want to use libmpir (if such a system exists I'm not sure).

FreeBSD provides GMP and MPIR, not in GMP-compatibility mode (I looked at the corresponding makefile) https://www.freshports.org/math/mpir https://www.freshports.org/math/gmp

both are in /usr/local/ there.

I would not be worried about a hypothetical situation of many libgmp's on the system (in fact, MPFR has a very long configure script dealing with detecting all sorts of info about GMP (or a GMP-compatible thing)

comment:9 Changed 3 years ago by embray

I guess the main question is what to do with the --with-mp= flag in Sage's ./configure script. I don't think it necessarily needs to continue working exactly the same way, but whatever it does it should be sensible. The way I see there are a couple possible ways to interpret it:

One way is that there are really three possible values:

  • detect
  • gmp
  • mpir

where the default value is "detect". In the "detect" case we just try to find the best working libgmp on the system and use it if found. If not found the next fallback would be to install the mpir SPKG (the current default). If passed "mpir" or "gmp" explicitly we don't bother detecting and just install the specifically requested SPKG.

Another possible interpretation is something more like:

  • gmp
  • mpir

where now default value "gmp" is similar to the first case: just try to detect any libgmp on the system to use, and failing that install an SPKG for some libgmp (where we might still use MPIR, but that fact is an implementation detail), whereas "mpir" means "detect MPIR specifically", which is to say find a libgmp that also, at a minimum, contains the __mpir_version flag. Perhaps, for some reason, a user might specifically want to use MPIR e.g. for performance reasons. If not found then we build the MPIR SPKG and use that.

Perhaps there are some other ways of interpreting/re-interpreting this flag, but those are a couple I have in mind.

comment:10 Changed 3 years ago by embray

I think I lean more toward the latter case. Most people who want to build Sage do not care what GMP implementation they use, so I think it's good enough to use the one they have, if any, and if not pick one for them. Whereas if someone is explicitly passing --with-mp=mpir then the really want MPIR specifically, and we should not give them something that isn't it.

OTOH I'd also be curious in what cases MPIR gives better performance, and if we're doing users a disservice by not at least strongly recommending that they use it. I think this is more a question for the mailing list though. The details of what to do at configure-time can be malleable.

Last edited 3 years ago by embray (previous) (diff)

comment:11 Changed 3 years ago by dimpase

Perhaps we could leave the mpir-based toolchain aside for a moment, after all mpir is an optional package, so it is a bug if it is pulled in once GMP is chosen, and concentrate on the rest of the surrounding number theory libs: mpc, mpfr, gf2x, ntl, pari.

So first we have to sort out why GMP is pulled in despite being recognised by configure as not to be installed, no?


Note that this spurious GMP installation isn't even triggered before some of spkgs dependent on GMP are installed, e.g. MPFR happily installs long before it's started on GMP. I get

$ ldd local/lib/libmpfr.so
	linux-vdso.so.1 (0x00007ffcc02e9000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f9345368000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f9345199000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9345696000)

but the beginning of MPFR install log says, wrongly,

  --with-gmp="/mnt/opt/Sage/sage-dev/local"

This is of course as expected, as it's hardwired in its spkg-install, as sdh_configure --with-gmp="$SAGE_LOCAL"...

It appears to me that instead the main ./configure should set up something like SAGE_GMP to be either SAGE_LOCAL or a location computed by GMP's spkg-configure.m4, and then this SAGE_GMP must be used by MPFR...

Last edited 3 years ago by dimpase (previous) (diff)

comment:12 Changed 3 years ago by dimpase

And MPFR's only part of trouble:

$ grep "gmp=..SAGE_LOCAL." */spkg-install  
arb/spkg-install:    --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL"
deformation/spkg-install:            --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" \
ecm/spkg-install:sdh_configure --with-gmp="$SAGE_LOCAL" $ECM_CONFIGURE
flint/spkg-install:    --with-gmp="$SAGE_LOCAL" \
gap/spkg-install:sdh_configure --with-gmp="$SAGE_LOCAL"
gcc/spkg-install:    --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" --with-mpc="$SAGE_LOCAL" \
gdb/spkg-install:    --with-gmp="$SAGE_LOCAL"
gfortran/spkg-install:    --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" --with-mpc="$SAGE_LOCAL" \
givaro/spkg-install:sdh_configure --with-gmp="$SAGE_LOCAL" --enable-shared $GIVARO_CONFIGURE
glpk/spkg-install:#       GMP/MPIR and zlib, so we should just use `--with-gmp="$SAGE_LOCAL"` and
mpc/spkg-install:sdh_configure --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" $EXTRA
mpfi/spkg-install:sdh_configure --with-mpfr="$SAGE_LOCAL" --with-gmp="$SAGE_LOCAL"
mpfrcx/spkg-install:sdh_configure --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" \
mpfr/spkg-install:        ./configure --with-gmp="$SAGE_LOCAL" $SAGE_CONF_OPTS $MPFR_CONFIGURE) &>/dev/null;
mpfr/spkg-install:    sdh_configure --with-gmp="$SAGE_LOCAL" \
pari/spkg-install:    --with-readline="$SAGE_LOCAL" --with-gmp="$SAGE_LOCAL" \
polylib/spkg-install:sdh_configure --with-libgmp=${SAGE_LOCAL}/lib
singular/spkg-install:                  --with-gmp="$SAGE_LOCAL" \
topcom/spkg-install:    --with-gmp="$SAGE_LOCAL" \

So passing SAGE_LOCAL left and right for everything is something to work on.

comment:13 Changed 3 years ago by dimpase

I don't think I understand how the .dummy target for skipping "dummy" packages in build/make/Makefile works, in particular, why it's just a single $(INST)/.dummy file for all of them.

Or perhaps it's because gmp spkg is known as well as $(MP_LIBRARY), and it's not found on the list of "dummy" packages, which duly includes gmp...

comment:14 Changed 3 years ago by dimpase

I've added spkg-configures to the whole "toolchain" bunch (see attachment) - the one for MPIR is basically just a plug, and will now try running the build to see if GMP install still surfaces...

Changed 3 years ago by dimpase

comment:15 Changed 3 years ago by dimpase

no, this does not help - and while I didn't have patience for the build to come to a halt, I still saw GMP getting built. (and none of MP*'s)

comment:16 follow-up: Changed 3 years ago by embray

The thing to do with all of these is that configure needs to output some config file based on its results which contains the correct prefixes for all these libraries (and likely others to come).

For this it might be useful to use the existing sage-env-config file, since I believe it gets sourced when building each package. It would probably be useful to have there since having the correct paths to libraries may also be useful at runtime (e.g. when running cython(), I think). To this end it would also be nice to have a sort of sage.env_config Python module that is also generated, so these values can easily be loaded by the Python library without sage-env being sourced, but this is not necessary for the SPKG build process.

Then every --with-gmp=$SAGE_LOCAL" should be replaced with --with-gmp=$SAGE_GMP_PREFIX", or something like that (the value of which is still ofc $SAGE_LOCAL when using a GMP installed from an SPKG).

What I'm less sure about is how exactly to determine the value for $SAGE_GMP_PREFIX. Macros like AC_SEARCH_LIBS only tests that linking with -l<libname> works; it has no clue about how the system searches library paths in that case. I think the best way to handle this is not unlike the code I wrote in #27230, unfortunately.

Last edited 3 years ago by embray (previous) (diff)

comment:17 follow-up: Changed 3 years ago by dimpase

I think the .dummy magic breaks for $(MP_LIBRARY). To prove this, I did

  • build/make/deps

    diff --git a/build/make/deps b/build/make/deps
    index e9008d20e6..c37b6062ee 100644
    a b toolchain: $(foreach pkgname,$(TOOLCHAIN),$(inst_$(pkgname))) 
    7777# See #14168 and #14232.
    7878#
    7979# Note: This list consists of only the *runtime* dependencies of the toolchain.
    80 TOOLCHAIN_DEPS = zlib $(MP_LIBRARY) mpfr mpc
     80TOOLCHAIN_DEPS = zlib gmp mpfr mpc
    8181TOOLCHAIN_DEP_INSTS = \
    8282       $(foreach pkgname,$(TOOLCHAIN_DEPS),$(inst_$(pkgname)))
    8383
    sagelib: \ 
    156156               $(inst_mpc) \
    157157               $(inst_mpfi) \
    158158               $(inst_mpfr) \
    159                $(MP_LIBRARY) \
    160159               $(inst_ntl) \
    161160               $(inst_numpy) \
    162161               $(inst_pari) \

(probably only the 2nd change, removal from sagelib: deps matters, not the 1st change.)

and, viola, no GMP building pops up. Needless to say, "dummies" of the form $(inst_...) do not get built.

Last edited 3 years ago by dimpase (previous) (diff)

comment:18 in reply to: ↑ 17 Changed 3 years ago by embray

Replying to dimpase:

I think the .dummy magic breaks for $(MP_LIBRARY). To prove this, I did

Sorry, I didn't realize you were being stymied by this. Yes, to me there was no mystery here, which is in part why I was focused on how best to handle the existing --with-mp flag. I should have been more clear about that.

Last edited 3 years ago by embray (previous) (diff)

comment:19 in reply to: ↑ 16 Changed 3 years ago by embray

Replying to embray:

What I'm less sure about is how exactly to determine the value for $SAGE_GMP_PREFIX. Macros like AC_SEARCH_LIBS only tests that linking with -l<libname> works; it has no clue about how the system searches library paths in that case. I think the best way to handle this is not unlike the code I wrote in #27230, unfortunately.

Another possibility of course is to just leave it empty and not pass --with-gmp flags at all, in this case they will be forced to just pick up whatever GMP they find on the default paths as well, which usually will be the same one.

comment:20 follow-up: Changed 3 years ago by dimpase

Pragmatically, I'd say that--with-mp=foo ought to always install foo from Sage's source, unless foo=detect. This is your first possibility from comment 9. (And leaving it out would result in either picking up system's "GMP", be it real GMP or MPIR in compatibility mode.)

This is reasonably flexible. Experts who need special versions of GMP/MPIR may just update Sage's packages and use them, or install them system-wide.

Most of all, it would not need fiddling with recongising "hidden" MPIR in a system's libgmp, something that would otherwise cater for non-existing systems, i.e. pure over-design for nothing and nobody...

comment:21 in reply to: ↑ 20 ; follow-up: Changed 3 years ago by embray

Replying to dimpase:

Pragmatically, I'd say that--with-mp=foo ought to always install foo from Sage's source, unless foo=detect. This is your first possibility from comment 9. (And leaving it out would result in either picking up system's "GMP", be it real GMP or MPIR in compatibility mode.)

This is reasonably flexible. Experts who need special versions of GMP/MPIR may just update Sage's packages and use them, or install them system-wide.

Most of all, it would not need fiddling with recongising "hidden" MPIR in a system's libgmp, something that would otherwise cater for non-existing systems, i.e. pure over-design for nothing and nobody...

Okay, let's go with that then, for now. That would be good because it is also the most "backwards compatible" with the current behavior.

As for the --with-gmp= question, I keep thinking about it and I think this is the best thing to do:

  • in the gmp/spkg-configure.m4 add support for our own --with-gmp= flag. Most people won't need this, but it does provide some desirable flexibility. This would just set the GMP prefix explicitly (and be saved to $SAGE_GMP_PREFIX) to be passed on to other packages that look for a GMP
  • when passed an explicit --with-gmp flag that should also imply that we want to use the system package, and if it can't be found this should be an error
  • if a usable system GMP is detected (with or without --with-gmp) just go with it; if --with-gmp was not specified then we set $SAGE_GMP_PREFIX to blank, and in that case explicitly do not pass --with-gmp= to other packages: let them just pick up the system GMP (note: I checked, and passing --with-gmp= with a blank argument is buggy in at least a few of these packages, so better not to use the flag at all in that case)
  • if a usable system GMP is not detected we just build the SPKG as normal, with whatever option was selected by --with-mp, with MPIR by default and set SAGE_GMP_PREFIX=$SAGE_LOCAL.

comment:22 Changed 3 years ago by embray

I'm working on a prototype for the above scheme. Which is not to say I'm convinced is the best way, but it seems worth a try.

Last edited 3 years ago by embray (previous) (diff)

comment:23 Changed 3 years ago by dimpase

OK, great. By the way, on Gentoo, with these extra spkg-configures in Attachment (all of them used to actually use system's libraries) , I let make ptests run, and all the tests passed.

I actually had one more, for gf2x

SAGE_SPKG_CONFIGURE([gf2x], [
    AC_CHECK_HEADER(gf2x.h, [], [sage_spkg_install_gf2x=yes])
dnl gf2x_mul_r appeared in version 1.2 of GF2X 
    AC_SEARCH_LIBS([gf2x_mul_r], [gf2x], [break], [sage_spkg_install_gf2x=yes])
])

I guess it should go on another ticket.

Last edited 3 years ago by dimpase (previous) (diff)

comment:24 in reply to: ↑ 21 Changed 3 years ago by embray

Replying to embray:

  • in the gmp/spkg-configure.m4 add support for our own --with-gmp= flag. Most people won't need this, but it does provide some desirable flexibility. This would just set the GMP prefix explicitly (and be saved to $SAGE_GMP_PREFIX) to be passed on to other packages that look for a GMP
  • when passed an explicit --with-gmp flag that should also imply that we want to use the system package, and if it can't be found this should be an error

It turns out doing this right is likely more trouble than it's worth, so I'm going to say "YAGNI" to this, until and unless proven otherwise. It turns out even most of the packages that have a --with-gmp flag in their own configures don't really implement it well. As long as Sage is using the system GMP, this flag shouldn't really be needed anyways for compiling other packages against the system GMP. So better to just leave it off entirely, at least when using the system package.

comment:25 Changed 3 years ago by dimpase

  • Dependencies set to #27254

comment:26 Changed 3 years ago by embray

I updated a bunch of spkg-installs to not use --with-gmp when using the system package. The ./configure happily detected my system's GMP 6.1.2 (the most recent). Doing a full rebuild and make ptestlong against that to test.

comment:27 Changed 3 years ago by embray

Getting weird segfaults and other errors from cypari when building with my system's GMP. Not sure if there's any relation though, something else could be wrong, at this point, with this build...

comment:28 Changed 3 years ago by embray

Okay, I don't know where I got the idea that this system has GMP 6.1.2. I appear to have gotten mixed up between machines. This machine has GMP 5.1.3. Nevertheless that does seem to be a problem because this spkg-configure.m4 allowed me to use it. But apparently PARI does not support this old a version.

I get

[pari-2.11.1.p0] linux-x86_64-gmp30818: error while loading shared libraries: libgmp.so.23: cannot open shared object file: No such file or directory
[pari-2.11.1.p0] ### Your version of GMP is too old for PARI. Please upgrade

I don't know why it's looking for some "libgmp.so.23" though. I don't have anything like that on my system. So this could be a bug in the PARI build system. I will see if I can convince PARI to use this GMP and if it will work. Nevertheless it's not encouraging.

Last edited 3 years ago by embray (previous) (diff)

comment:30 Changed 3 years ago by embray

Okay, I think pari does not actually need this version of GMP (which is not to say some other package might not need it, but I want to see how far I can get).

What was messing me up was that I did not prevent mpir from being installed in this build, so it was partially picking up the MPIR in $SAGE_LOCAL but in an incomplete and broken manner. After applying part of my fix to prevent MPIR from being installed, PARI built okay.

comment:31 Changed 3 years ago by embray

Final update on this before I really go to sleep: pynac uses mpq_cmp_z which is definitely only available as of GMP 6.1.0. pynac could probably manage to not use it if it's not available, but I don't think it's worth bothering with. I'm perfectly happy to just require GMP>=6.1.0 in that case, which is still rather old at this point.

The symbol to check for in this case is __gmpq_cmp_z. I confirmed in does not exist in GMP 6.0.x or below, and has the benefit of being a function that is actually used in one of the dependents of GMP.

comment:32 Changed 3 years ago by dimpase

  • Keywords spkg-configure added

comment:33 Changed 3 years ago by embray

  • Branch changed from u/dimpase/packages/gmp_m4 to u/public/packages/gmp_m4
  • Commit 1ed38e19baea5feb0d2f5d79e7989a7e80440a21 deleted
  • Status changed from needs_work to needs_review

Here's my attempt at this:

Moved all the --with-mp logic into the spkg-configure.m4 for MPIR so that it is local to where it is used.

Also, most of the logic is in mpir/spkg-configure.m4 rather than gmp/spkg-configure.m4 just because MPIR is the implementation we install by default if it can't be used from the sytem. It could go either way, but this way seemed a bit more logical for that reason.

Then, the gmp/spkg-configure.m4 is mostly just a stub.

Finally, uses the symbol __gmpq_cmp_z to check for GMP>=6.1.0 (or whatever the equivalent MPIR is).

So to summarize, by default --with-mp=system and it tries to use the libgmp from the system. If that check fails it falls back to installing the mpir SPKG. If --with-mp=mpir it just doesn't even check the system and installs the mpir SPKG; similarly for --with-mp=gmp.

comment:34 Changed 3 years ago by embray

  • Branch changed from u/public/packages/gmp_m4 to public/packages/gmp_m4
  • Commit set to dd39b8453c2eda2d4b9bc83f0bc146a42743c14a

Oops, typo on branch name.


New commits:

1ed38e1spkg-configure for GMP
dd39b84Move all the --with-mp logic into the spkg-configure.m4 for MPIR so that it

comment:35 Changed 3 years ago by dimpase

  • Summary changed from spkg-configure.m4 for gmp to spkg-configure.m4 for gmp and mpir

comment:36 Changed 3 years ago by embray

  • Status changed from needs_review to needs_work

I accidentally left off a bunch of changes related to the --with-gmp flags in other packages.

comment:37 Changed 3 years ago by git

  • Commit changed from dd39b8453c2eda2d4b9bc83f0bc146a42743c14a to 27b469252871005abd57fe3bc9d8bbf4e451a7f1

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

356621acorrectly setup sagelib dependencies-trac #27254
da291d8spkg-configure for GMP
f7698adMove all the --with-mp logic into the spkg-configure.m4 for MPIR so that it
27b4692Replace various --with-gmp=$SAGE_LOCAL flags in spkg-installs with a

comment:38 Changed 3 years ago by embray

  • Status changed from needs_work to needs_review

Also rebased on #27254.

comment:39 Changed 3 years ago by embray

  • Status changed from needs_review to needs_work

I just realized there is another problem with this, which didn't appear until I rebased. Now if SAGE_MP_LIBRARY=system things break because there is no target "system".

comment:40 Changed 3 years ago by dimpase

the ECM bit looks fishy:

--- a/build/pkgs/ecm/spkg-install
+++ b/build/pkgs/ecm/spkg-install
@@ -210,7 +210,10 @@ else
 fi
 echo "  --prefix=\"$SAGE_LOCAL\""
 echo "  --libdir=\"$SAGE_LOCAL/lib\""
-echo "  --with-gmp=\"$SAGE_LOCAL\""
+if [ -n "$SAGE_GMP_CONFIGURE" ]; then
+    echo "  --with-gmp=\"$SAGE_LOCAL\""
+fi
+

comment:41 Changed 3 years ago by embray

It is a typo. It should be $SAGE_CONFIGURE_GMP. It's not really important though, I was just retaining the existing behavior in that code.

comment:42 Changed 3 years ago by git

  • Commit changed from 27b469252871005abd57fe3bc9d8bbf4e451a7f1 to b979d7b8d93ce6da2967317766735de7a7a0c6a4

Branch pushed to git repo; I updated commit sha1. New commits:

566bd21Reworked this a bit more
a4ab468fix typo
b979d7badded a bit more explanation

comment:43 Changed 3 years ago by embray

  • Status changed from needs_work to needs_review

I think this is fixed up now, though the commits should be squashed before setting final positive_review.

comment:44 Changed 3 years ago by dimpase

I see no chaining in mpir/mpfr macros - if I intentionally break mpir recognition I still get mpfr not being installed, but picked up from the system... ---if I just run ./configire without parameters.

(I don't know whether this breaks builds or not, but I guess we would like to see in such a case, if mpfr is not available, mpfr being built as well)

Last edited 3 years ago by dimpase (previous) (diff)

comment:45 follow-up: Changed 3 years ago by dimpase

  • Dependencies changed from #27254 to #27254, #27272

With this branch, I've run into a problem with IML package, see #27272

comment:46 in reply to: ↑ 45 Changed 3 years ago by dimpase

Replying to dimpase:

With this branch, I've run into a problem with IML package, see #27272

this is platform-specific - e.g. on Fedora 26 I don't get this problem.

comment:47 follow-up: Changed 3 years ago by git

  • Commit changed from b979d7b8d93ce6da2967317766735de7a7a0c6a4 to 1e48b301165b7756b203e5a2935fb91375b1da7b

Branch pushed to git repo; I updated commit sha1. New commits:

1e48b30correct logic for SAGE_GMP_PREFIX etc

comment:48 Changed 3 years ago by dimpase

  • Authors changed from Dima Pasechnik to Erik Bray, Dima Pasechnik
  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

comment:49 Changed 3 years ago by isuruf

There are some sage dependencies using libgmpxx and if gmpxx was compiled with a different C++ standard library, you'll get link failures. It'll be good check for this here.

comment:50 follow-up: Changed 3 years ago by isuruf

Here's an example file you can use, https://github.com/symengine/symengine/blob/master/cmake/checkgmpxx.cpp. This check fails when gmpxx is compiled using g++ and the dependent library is compiled using clang++ and vice-versa on OSX.

comment:51 in reply to: ↑ 50 Changed 3 years ago by dimpase

Replying to isuruf:

Here's an example file you can use, https://github.com/symengine/symengine/blob/master/cmake/checkgmpxx.cpp. This check fails when gmpxx is compiled using g++ and the dependent library is compiled using clang++ and vice-versa on OSX.

I suppose that there is no compatibility even across different versions of g++, and perhaps not even when the same compiler is used with different -std=.

comment:52 follow-up: Changed 3 years ago by isuruf

https://stackoverflow.com/questions/46746878/is-it-safe-to-link-c17-c14-and-c11-objects should answer your general question. For gmpxx though, I don't think it matters a lot.

comment:53 in reply to: ↑ 52 Changed 3 years ago by dimpase

Replying to isuruf:

https://stackoverflow.com/questions/46746878/is-it-safe-to-link-c17-c14-and-c11-objects should answer your general question. For gmpxx though, I don't think it matters a lot.

Thanks. It seems that this link basically says that as far as linking issues are concerned, you might be unable to chase down all the corner cases if one used different versions of g++ (which is unsurprising to me...).

comment:54 in reply to: ↑ 47 Changed 3 years ago by dimpase

  • Status changed from positive_review to needs_work

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

1e48b30correct logic for SAGE_GMP_PREFIX etc

setting of SAGE_GMP_PREFIX does not quite work there, it's always ''.

comment:55 follow-up: Changed 3 years ago by git

  • Commit changed from 1e48b301165b7756b203e5a2935fb91375b1da7b to 50c3482dd8700d4b62d229edf8beb3edc6fc7f5e

Branch pushed to git repo; I updated commit sha1. New commits:

50c3482remove iffy if, do setting SAGE_GMP_PREFIX in cases

comment:56 in reply to: ↑ 55 Changed 3 years ago by embray

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

50c3482remove iffy if, do setting SAGE_GMP_PREFIX in cases

This is still wrong because in that case statement, in the system) case you can still wind up installing the Sage SPKG if any of the checks fail (hence setting sage_spkg_install_mpir=yes). That's why there is a separate if/then clause later.

I think the problem here is is with quoting. You can see it in the configure log when running the previous version (and also fixing the SAGE_MP_LIBRARY = mpir line):

./configure: line 8899: test: too many arguments

I believe it's possible for $sage_spkg_install_gmp to not be defined here, so we need the x quoting.

comment:57 Changed 3 years ago by git

  • Commit changed from 50c3482dd8700d4b62d229edf8beb3edc6fc7f5e to b364988a362ba2c4872a002b0477858b9cac073d

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b364988correct logic for SAGE_GMP_PREFIX etc

comment:58 Changed 3 years ago by embray

This should fix it I think. I also fixed the AC_MSG_RESULT calls to report the correct result when --with-mp=system but the system lib cannot be used (and hence we fall back on installing the MPIR SPKG).

I think what might be continuing to confuse you is that the case for --with-mp=system can still result in the MPIR SPKG being selected instead.

comment:59 Changed 3 years ago by embray

Building a fresh build from this branch on Ubuntu 16.04 I also ran into the same problem Dima was telling me about with iml, reported also at #27272. I just wanted to confirm that that was the case, so now I can also review that ticket with some intelligence.

comment:60 Changed 3 years ago by git

  • Commit changed from b364988a362ba2c4872a002b0477858b9cac073d to cb6f12448719f9fec4ec56d24b8e1275a168f09f

Branch pushed to git repo; I updated commit sha1. New commits:

cb6f124add the AX_ABSOLUTE_HEADER macro

comment:61 Changed 3 years ago by git

  • Commit changed from cb6f12448719f9fec4ec56d24b8e1275a168f09f to cf359b1df8fe4bf2009401888ac2e32600ec1ce0

Branch pushed to git repo; I updated commit sha1. New commits:

cf359b1iml in particular is very picky about being given an absolute path to the

comment:62 Changed 3 years ago by embray

Added a fix to the spkg-install for iml for it to use the system GMP as specified by SAGE_GMP_INCLUDE.

I am a bit iffy on the hack for setting SAGE_GMP_LIB and am open to other ideas. But I did confirm that this works when using the system GMP on Ubuntu 16.04. With the above two fixes I was able to build Sage just fine and make ptestlong passes.

comment:63 Changed 3 years ago by embray

  • Status changed from needs_work to needs_review

comment:64 Changed 3 years ago by embray

  • Dependencies changed from #27254, #27272 to #27254

I think it will make more sense to keep this ticket as a dependency of #27272 (or close #27272) since this is now covering both.

comment:65 Changed 3 years ago by dimpase

  • Status changed from needs_review to positive_review

It's good to go, I think.

comment:66 Changed 3 years ago by dimpase

Probably this one needs an updated configure package - unless all the bots nowadays run ./bootstrap (I don't recall the current state of this).

comment:67 Changed 3 years ago by vbraun

  • Status changed from positive_review to needs_work

Merge conflict

comment:68 Changed 3 years ago by dimpase

it's gcc and gfortran spkg's that were touched in the latest beta. Erik, could you rebase these, it's certainly your code there...

comment:69 Changed 3 years ago by git

  • Commit changed from cf359b1df8fe4bf2009401888ac2e32600ec1ce0 to 101537b93c3343a3e7a8554d8e82d8da36be5691

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

c4e42e6spkg-configure for GMP
b070088Move all the --with-mp logic into the spkg-configure.m4 for MPIR so that it
3b6eebdReplace various --with-gmp=$SAGE_LOCAL flags in spkg-installs with a
b80bf72Reworked this a bit more
0ca3f56fix typo
06798caadded a bit more explanation
b592d77correct logic for SAGE_GMP_PREFIX etc
5057680add the AX_ABSOLUTE_HEADER macro
101537biml in particular is very picky about being given an absolute path to the

comment:70 Changed 3 years ago by dimpase

ok, it was trivial to rebase. now testing before setting it to positive again.

comment:71 Changed 3 years ago by dimpase

  • Status changed from needs_work to positive_review

comment:72 Changed 3 years ago by dimpase

  • Dependencies changed from #27254 to #27254, 27367

The present mpfrcx does not work with this branch, but it does with the new version. Thus the new dependence.

comment:73 Changed 3 years ago by dimpase

  • Dependencies changed from #27254, 27367 to #27254, #27367

comment:74 Changed 3 years ago by dimpase

  • Dependencies changed from #27254, #27367 to #27254, #27367, #27373

$SAGE_LOCAL/bin/sage-env-config must get updated after, say, sage -i mpir (or any other dummy package that sets anything in the build environment). Currently this does not happen. This is a bug that leads to sage -i mpir breaking stuff.

See #27373 to deal with this. (Hopefully everything that is on this ticket can stay as it is, so I don't flip it to "needs work")

Last edited 3 years ago by dimpase (previous) (diff)

comment:75 Changed 3 years ago by dimpase

We need to decide how to move forward here. I'm fine with a no-fix for #27373, i.e. "re-run configure with the parameters you need, do not use ./sage -i for un-dummifying packages".

comment:76 Changed 3 years ago by embray

I guess I don't really see how #27373 is relevant here. Or at least, I don't see that it's an issue particular to this ticket or that should be a dependency of it.

comment:77 Changed 3 years ago by jdemeyer

  • Status changed from positive_review to needs_work

Merge conflict

comment:78 Changed 3 years ago by git

  • Commit changed from 101537b93c3343a3e7a8554d8e82d8da36be5691 to 862ca6af87ac256bf726b0bd7ef7439e2d9b0080

Branch pushed to git repo; I updated commit sha1. New commits:

862ca6aMerge remote-tracking branch 'trac/develop' into HEAD

comment:79 Changed 3 years ago by dimpase

  • Status changed from needs_work to needs_review

comment:80 Changed 3 years ago by dimpase

  • Status changed from needs_review to positive_review

the only manual part of the merge was here:

  • build/pkgs/arb/spkg-install

    a b cd src 
    44# be removed in arb >= 2.8 when it is released
    55export EXTRA_SHARED_FLAGS=$LDFLAGS
    66
    7 ./configure --disable-static --prefix="$SAGE_LOCAL" --with-flint="$SAGE_LOCAL" \
    8     --with-gmp="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" || \
     7./configure --disable-static --prefix="$SAGE_LOCAL" $SAGE_CONFIGURE_GMP \
     8            --with-flint="$SAGE_LOCAL" --with-mpfr="$SAGE_LOCAL" || \
    99    sdh_die "Error configuring arb."
    1010
    1111sdh_make verbose

comment:81 Changed 3 years ago by dimpase

  • Dependencies changed from #27254, #27367, #27373 to #27254, #27367, #27373, #27215

comment:82 Changed 3 years ago by git

  • Commit changed from 862ca6af87ac256bf726b0bd7ef7439e2d9b0080 to 03dc987c6d475279b0008fa8f5ba37a0045bb75e
  • Status changed from positive_review to needs_review

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

7e2b100Remove broken SAGE_BUILD_TOOLCHAIN support
03dc987Merged trac #27215 in

comment:83 Changed 3 years ago by dimpase

  • Milestone changed from sage-8.7 to sage-8.8
  • Status changed from needs_review to positive_review

comment:84 Changed 3 years ago by dimpase

  • Dependencies changed from #27254, #27367, #27373, #27215 to #27373

comment:85 follow-up: Changed 3 years ago by dimpase

I wonder whether for gcc and gfortran --with-gmp="$SAGE_LOCAL" is left there on purpose, or it's simply a typo/omission?

This might have "interesting" interactions with what is to come on #27373.

comment:86 Changed 3 years ago by dimpase

Isn't the nesting of AC_CHECK_HEADER's and AC_SEARCH_LIBS needed here? Otherwise we may end up with system's GMP library, but without headers available.

Last edited 3 years ago by dimpase (previous) (diff)

comment:87 in reply to: ↑ 85 Changed 3 years ago by embray

  • Status changed from positive_review to needs_work

Replying to dimpase:

I wonder whether for gcc and gfortran --with-gmp="$SAGE_LOCAL" is left there on purpose, or it's simply a typo/omission?

This might have "interesting" interactions with what is to come on #27373.

I was just about to ask you about this. I'm working on rebasing this on top of current develop (mostly straightforward but not trivial). In the current upstream version of this branch, which you were the last to touch, gcc/gfortran was omitted in this commit. However, in my local copy of the branch it still includes them.

I was going to ask if that was intentional but now it's sounding like maybe it wasn't. So I'll put it back in for now.

comment:88 Changed 3 years ago by git

  • Commit changed from 03dc987c6d475279b0008fa8f5ba37a0045bb75e to eea06854a43e37c893c9fa9e26866d7e4cce1da5

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8c8285fspkg-configure for GMP
956e382Move all the --with-mp logic into the spkg-configure.m4 for MPIR so that it
687cdf0Replace various --with-gmp=$SAGE_LOCAL flags in spkg-installs with a
6de961dReworked this a bit more
f76d6d5fix typo
5f90ff7added a bit more explanation
3dee97ccorrect logic for SAGE_GMP_PREFIX etc
0308142add the AX_ABSOLUTE_HEADER macro
eea0685iml in particular is very picky about being given an absolute path to the

comment:89 Changed 3 years ago by embray

  • Status changed from needs_work to needs_review

Rebased on 8.8.beta0. Now to test again; this is all toward making progress on #27373...

comment:90 Changed 3 years ago by dimpase

Once again, about nesting. Shouldn't e.g.

            AC_CHECK_HEADER(gmp.h, [], [sage_spkg_install_mpir=yes])
            AC_CHECK_HEADER(gmpxx.h, [], [sage_spkg_install_mpir=yes])
            dnl mpq_cmp_z appeared in GMP 6.1.0 and is used by pynac
            AC_SEARCH_LIBS([__gmpq_cmp_z], [gmp], [break],
                [sage_spkg_install_mpir=yes])

be written as

            AC_CHECK_HEADER(gmp.h, [
              AC_CHECK_HEADER(gmpxx.h, [
                dnl mpq_cmp_z appeared in GMP 6.1.0 and is used by pynac
                AC_SEARCH_LIBS([__gmpq_cmp_z], [gmp], [break],
                   [sage_spkg_install_mpir=yes])
              ], [sage_spkg_install_mpir=yes])
            ], [sage_spkg_install_mpir=yes])            

And, by the way, what's that break doing there? I gather it's to interrupt loops, but there is no loop here...

comment:91 Changed 3 years ago by embray

I don't think the nesting is necessary. If finding the headers fails then it will set sage_spkg_install_mpir=yes; there's no advantage to then skipping the library check because it will still keep sage_spkg_install_mpir=yes reagardless. Nesting it just makes the code harder to read IMO.

And, by the way, what's that break doing there? I gather it's to interrupt loops, but there is no loop here...

Remember, most of these autoconf macros expand to shell code. So that's where the loop is, and you can see what a break would do there by looking at the expanded macro in the actual configure script.

However, I think it's superfluous in this case. It would break out of the loop if AC_SEARCH_LIBS finds that you can link the test program without explicitly passing -lgmp, but I don't think that would ever be the case (it is the case sometimes with other libraries, such as libm, which may be linked automatically). So I can just remove it if it's clearer (it's also harmless though).

Last edited 3 years ago by embray (previous) (diff)

comment:92 Changed 3 years ago by dimpase

  • Status changed from needs_review to positive_review

OK, it's fine then.

comment:93 Changed 3 years ago by embray

Thanks--I still think it would be ok to remove #27373 as a dependency. Because although it is a known bug, it's not entirely new after all as I explained in comment:73:ticket:27373. In short, there are other cases where we should require re-running configure after installing new packages, but don't, resulting in an incorrect configuration.

It should still be fixed ASAP but I think for now it's good enough to say "don't do that".

Last edited 3 years ago by embray (previous) (diff)

comment:94 Changed 3 years ago by dimpase

  • Dependencies #27373 deleted

OK, let's get it merged then!

comment:95 follow-up: Changed 3 years ago by vbraun

  • Status changed from positive_review to needs_work

Singular doesn't build for me...

comment:96 Changed 3 years ago by dimpase

Logs please.

Do you have gmp installed in a weird place?

You might need to start from make distclean, to be on the safe side.

comment:97 Changed 3 years ago by dimpase

and, needless to say, this needs a run of ./bootstrap.

what we might have forgotten is to bump up versions of gmp and mpir, to force re-building of appropriate packages, our poor man dependency resolution. I never understood why spkg-install and spkg-configure.m4 are not listed as package dependencies in automatically generated targets for packages.

comment:98 in reply to: ↑ 95 Changed 3 years ago by embray

  • Owner changed from (none) to vbraun

Replying to vbraun:

Singular doesn't build for me...

This that building from scratch, or what? Note that the configure script should be regenerated or lots of things won'g build properly due to the changes to sage-env-config.in.

comment:99 Changed 3 years ago by embray

  • Status changed from needs_work to needs_info

comment:100 Changed 3 years ago by vbraun

If thats the case then a new confball is needed on the branch...

Changed 3 years ago by embray

comment:101 Changed 3 years ago by embray

  • Description modified (diff)
  • Status changed from needs_info to positive_review

New configure tarball attached and linked in the description.

Nice number on that one =)

comment:102 Changed 3 years ago by dimpase

This is a sub-optimal workflow, as ticket authors have little control over the moment the ticket comes up for merging, and thus it's not unlikely that the configure tarball would be out of sync, if there is more than one stream of tickets that touch configure.

And we do have quite a few of them on #27330.

comment:103 Changed 3 years ago by embray

Indeed; I've complained about this as well. A better solution isn't immediately obvious to me other than to make sure that all of the buildbot workers have up-to-date autotools installed and run ./bootstrap as part of the normal build procedure. I think that should be done but it's not always trivial I guess (e.g. on OSX?).

In the meantime I'm happy to follow the workflow as it is; it could just be confusing for all the tickets attached to #27330. Maybe we can do something like bundle multiple of those tickets to be merged together once several of them have been otherwise positively reviewed individually.

comment:104 Changed 3 years ago by dimpase

it is trivial to install autotools on OSX using Homebrew.

comment:105 follow-up: Changed 3 years ago by vbraun

The one and only OSX buildbot worker is for testing that Sage installs on a as-vanilla-as-possible OSX install...

comment:106 Changed 3 years ago by vbraun

  • Status changed from positive_review to needs_work

configure-314.tar.gz is for 8.8.beta2, as always you are supposed to bump the version if you change the file.

comment:107 Changed 3 years ago by dimpase

I have no idea where this 314 can be found. If you update from source then upstream/configure-xyz.tar.gz might not correspond to anything at all, may not even exist.

E.g. my different sage builds, all based on 8.8.beta2, have different from 314 xyz.

I am tired of doing manually things that must have been automated 10 years ago. Why is that xyz there at all?! The build is run by make, which is intelligent enough to know if a file has been updated, you know...

comment:108 Changed 3 years ago by vbraun

Actually, 8.8.beta2 has already configure-315.tar.gz. The version number is in build/pkgs/configure/package-version.txt, like all tarballs.

To make a new confball:

  • merge in the latest beta (so you have the most up-to-date build/pkgs/configure/package-version.txt)
  • run ./bootstrap -s -i
  • Commit the new version and checksum, and upload the confball

comment:109 Changed 3 years ago by git

  • Commit changed from eea06854a43e37c893c9fa9e26866d7e4cce1da5 to e735b4bd95072234d72385ee78634b9000f7e420

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

a50fe85spkg-configure for GMP
8a16493Move all the --with-mp logic into the spkg-configure.m4 for MPIR so that it
299e367Replace various --with-gmp=$SAGE_LOCAL flags in spkg-installs with a
058d515Reworked this a bit more
7beadaafix typo
b930874added a bit more explanation
6d2b496correct logic for SAGE_GMP_PREFIX etc
43be546add the AX_ABSOLUTE_HEADER macro
e343418iml in particular is very picky about being given an absolute path to the
e735b4bconfigure version bump

comment:110 Changed 3 years ago by dimpase

  • Description modified (diff)
  • Status changed from needs_work to positive_review

New commits:

a50fe85spkg-configure for GMP
8a16493Move all the --with-mp logic into the spkg-configure.m4 for MPIR so that it
299e367Replace various --with-gmp=$SAGE_LOCAL flags in spkg-installs with a
058d515Reworked this a bit more
7beadaafix typo
b930874added a bit more explanation
6d2b496correct logic for SAGE_GMP_PREFIX etc
43be546add the AX_ABSOLUTE_HEADER macro
e343418iml in particular is very picky about being given an absolute path to the
e735b4bconfigure version bump

comment:111 Changed 3 years ago by dimpase

rebased over 8.8.beta2, new configure tarball

comment:112 Changed 3 years ago by dimpase

  • Status changed from positive_review to needs_work

and now this fails on OSX 10.14 with external gmp, specifically, flint does not build:

          make build/interfaces/NTL-interface.lo; \
          clang++ -std=gnu++11 -fno-common -pedantic -Wall -O2 -funroll-loops -g -mpopcnt   -shared -install_name /Users/administrator/
py/sage/local/lib/libflint-13.5.2.dylib -compatibility_version 13.5 -current_version 13.5.2 -Wl,-rpath,/usr/local/lib -Wl,-rpath,/Users
/administrator/py/sage/local/lib build/interfaces/NTL-interface.lo  build/printf.lo  build/fprintf.lo  build/sprintf.lo  build/scanf.lo
  build/fscanf.lo  build/sscanf.lo  build/clz_tab.lo  build/memory_manager.lo  build/version.lo  build/profiler.lo  build/thread_support.lo  build/ulong_extras.lo  build/long_extras.lo  build/perm.lo  build/fmpz.lo  build/fmpz_vec.lo  build/fmpz_poly.lo  build/fmpq_poly.lo  build/fmpz_mat.lo  build/fmpz_lll.lo  build/mpfr_vec.lo  build/mpfr_mat.lo  build/mpf_vec.lo  build/mpf_mat.lo  build/nmod_vec.lo  build/nmod_poly.lo  build/nmod_poly_factor.lo  build/arith.lo  build/mpn_extras.lo  build/nmod_mat.lo  build/fmpq.lo  build/fmpq_vec.lo  build/fmpq_mat.lo  build/padic.lo  build/fmpz_poly_q.lo  build/fmpz_poly_mat.lo  build/nmod_poly_mat.lo  build/fmpz_mod_poly.lo  build/fmpz_mod_poly_factor.lo  build/fmpz_factor.lo  build/fmpz_poly_factor.lo  build/fft.lo  build/qsieve.lo  build/double_extras.lo  build/d_vec.lo  build/d_mat.lo  build/padic_poly.lo  build/padic_mat.lo  build/qadic.lo  build/fq.lo  build/fq_vec.lo  build/fq_mat.lo  build/fq_poly.lo  build/fq_poly_factor.lo  build/fq_nmod.lo  build/fq_nmod_vec.lo  build/fq_nmod_mat.lo  build/fq_nmod_poly.lo  build/fq_nmod_poly_factor.lo  build/fq_zech.lo  build/fq_zech_vec.lo  build/fq_zech_mat.lo  build/fq_zech_poly.lo  build/fq_zech_poly_factor.lo  -o libflint-13.5.2.dylib -L/Users/administrator/py/sage/local/lib -Wl,-rpath,/Users/administrator/py/sage/local/lib  -L/Users/administrator/py/sage/local/var/tmp/sage/build/flint-2.5.2.p4/src -L/usr/local/lib -L/Users/administrator/py/sage/local/lib -L/Users/administrator/py/sage/local/lib -lmpfr -lgmp -lm -lntl -lpthread ; \
        fi
clang++ -std=gnu++11 -fPIC -fno-common -pedantic -Wall -O2 -funroll-loops -g -mpopcnt  -I/Users/administrator/py/sage/local/var/tmp/sage/build/flint-2.5.2.p4/src -I/usr/local/include -I/Users/administrator/py/sage/local/include -I/Users/administrator/py/sage/local/include -c interfaces/NTL-interface.cpp -o build/interfaces/NTL-interface.lo
ld: illegal thread local variable reference to regular symbol __ZN3NTL8ZZ_pInfoE for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[4]: *** [libflint-13.5.2.dylib] Error 1

note that mpfr and ntl, its dependencies build just fine with SAGE_CHECK=yes, so that's something in flint C++ interface that is broken...

comment:113 Changed 3 years ago by dimpase

I suppose this is caused by the very recent (Apr 6) NTL update.

comment:114 follow-up: Changed 3 years ago by dimpase

No, it must be either lack of testing on OSX 10.14 (I tested on 10.13 before), or new clang in Xcode:

$ clang++ -v
Apple LLVM version 10.0.1 (clang-1001.0.46.3)
Target: x86_64-apple-darwin18.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

comment:115 in reply to: ↑ 105 ; follow-up: Changed 3 years ago by dimpase

Replying to vbraun:

The one and only OSX buildbot worker is for testing that Sage installs on a as-vanilla-as-possible OSX install...

Well, if you like I can install autotools on your OSX bot just from sources. (I recall doing that on OSX 10.12, it should still be possible on 10.14). This would be vanilla enough then, still...

comment:116 in reply to: ↑ 114 Changed 3 years ago by dimpase

Replying to dimpase:

No, it must be either lack of testing on OSX 10.14 (I tested on 10.13 before), or new clang in Xcode:

$ clang++ -v
Apple LLVM version 10.0.1 (clang-1001.0.46.3)
Target: x86_64-apple-darwin18.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

The problem seems to be caused by -std=gnu++11 option, see https://trac.sagemath.org/ticket/25532#comment:88

comment:117 follow-up: Changed 3 years ago by dimpase

  • Status changed from needs_work to positive_review

facepalm.... That was just a library clash - I did have libntl installed in /usr/local, and this seems to be deadly on OSX...

comment:118 in reply to: ↑ 115 ; follow-up: Changed 3 years ago by vbraun

Replying to dimpase:

Well, if you like I can install autotools on your OSX bot just from sources. (I recall doing that on OSX 10.12, it should still be possible on 10.14).

Again, unless we make autotools a prerequisite we need a way to verify that Sage builds on OSX without autotools installed...

comment:119 in reply to: ↑ 118 Changed 3 years ago by dimpase

Replying to vbraun:

Replying to dimpase:

Well, if you like I can install autotools on your OSX bot just from sources. (I recall doing that on OSX 10.12, it should still be possible on 10.14).

Again, unless we make autotools a prerequisite we need a way to verify that Sage builds on OSX without autotools installed...

Do we have Linux buildbots without autotools installed?

Anyhow, one can have autotools installed in a non-standard location only buildbot knows, so that it can run ./bootstrap, and then it's as if there are not present.

It's hard for me to understand the problem without knowing how exactly a release is made. (if be any chance you are in Berlin, you can show me - I'm here until 21st Apr).

comment:120 Changed 3 years ago by vbraun

  • Branch changed from public/packages/gmp_m4 to e735b4bd95072234d72385ee78634b9000f7e420
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:121 follow-up: Changed 3 years ago by vbraun

  • Commit e735b4bd95072234d72385ee78634b9000f7e420 deleted

No matter how you slice it, either the buildbot has access to autotools OR it tests that you can build without autotools. The only safe way to avoid generating a confball by hand is to have some automatic system, e.g. whenever I submit a new buildbot job. That requires some work to avoid version clashes when I back out commits. Truth be told, changes to the configure in Sage are pretty rare so this issue does not come up very often.

I'm in Berlin, hit me up when you have some time. I'm located in the East (Elsenstrasse).

comment:122 in reply to: ↑ 117 Changed 3 years ago by embray

Replying to dimpase:

facepalm.... That was just a library clash - I did have libntl installed in /usr/local, and this seems to be deadly on OSX...

I have had similar problems--not necessarily with NTL--but with other packages where having a copy of the development package already installed on my system created difficult to avoid clashes when trying to build some Sage SPKGs. See e.g. #22768. In that case it's more due to bugginess with Python's module build system, but I can envision similar problems in other build systems that make too many assumptions about your paths in some context.

comment:123 in reply to: ↑ 121 Changed 3 years ago by dimpase

Replying to vbraun:

Truth be told, changes to the configure in Sage are pretty rare

I have a dozen tickets open that do change configure, see #27330. For the purpose of testing the configure spkg versioning, today I attached configure tarballs to 3 of them: #27258, #27265, #27186. No matter in which order they would be merged, I'd need to re-do these tarballs for all but one, you know. This is not exactly entertaining...

comment:124 Changed 3 years ago by jdemeyer

Breakage: #27751

Note: See TracTickets for help on using tickets.