spkg-configure.m4 for linbox

[gh-thierry-FreeBSD]

This ticket adds an spkg-configure.m4 for linbox, in order to use it from a system package if possible (see #27330).

Note for packagers: by default, linbox produces a buggy linbox.pc, and @LINBOXSAGE_LIBS@ must be removed from their linbox.pc.in (see #27205).

No more problem after that.

[gh-thierry-FreeBSD]

To be put under build/pkgs/linbox

[mjo]

We also need iml, ntl, flint, fplll, and mpfr in the DEPCHECK. Linbox can be built without some of those libraries with, for example, --without-ntl, so we may need to check for the parts that are actually used in sage. The sage-on-gentoo ebuild passes --without-fplll, so that dependency is probably superfluous, and maybe others are as well? In any case, to check which libraries linbox actually supports... after the PKG_CHECK_MODULES call, I guess we could look for things like -lntl in LINBOX_LIBS?

I don't see any distros that have a 1.6.x version other than 1.6.3, so it would probably be pointless to lower the version to 1.6.0. The real question is whether or not we can use the v1.5.2 that e.g. older Debian/Fedora? ship.

[gh-thierry-FreeBSD]

pkgconf linbox --libs

sets LINBOX_LIB, and we could grep that the required libs are there?

[mjo]

Hmm, looking at the spkg-install for linbox, we already pass --without-fplll:

# Disable fplll as version 5.x is not supported by linbox <= 1.5.0.
# This is harmless as no functionality using fplll is exposed in Sage.
# See trac ticket #21221.
LINBOX_CONFIGURE="--without-fplll $LINBOX_CONFIGURE"

So, the existing dependency on fplll (in build/pkgs/linbox/dependencies) is wrong. I'm going to build a system copy of linbox with all of the other stuff disabled and see what goes wrong. Maybe we don't care if the other stuff is enabled, either.

[mjo]

For posterity, this also appears to be obsolete,

# Need to use 'bash' for configure, see
# https://trac.sagemath.org/ticket/23451
if [ -z "$CONFIG_SHELL" ]; then
    export CONFIG_SHELL=`command -v bash`
fi

as I've been compiling linbox-1.6.3 this whole time with /bin/sh -> /bin/dash.

[mjo]

Even if you pass --without-foo for all of the optional linbox dependencies, flint still gets pulled in automagically, so it's going to be required either way. I'm doing a full sage build / ptestlong now with the rest (ntl, fplll, mpfr, iml) disabled to see if any of them are actually important.

[mjo]

Everything is fine if we disable ntl, iml, and mpfr in linbox, so I created a branch with those dependencies eliminated. That means that we only need to add "flint" to the DEPCHECK.

---

New commits:

​72291a0	Trac #29631: don't force SHELL=bash when installing linbox.
​787b5df	Trac #29631: drop fplll from linbox's dependencies.
​e4a14c0	Trac #29631: disable additional unused features of the linbox SPKG.

[mjo]

This page has some information on how to checkout and push new branches to our trac server:

​http://doc.sagemath.org/html/en/developer/manual_git.html

You should be able to checkout my branch, add a commit on top of it, and then push a new branch of your own (which you would then stick in the "branch" field of this ticket). That's the best way to collaborate since it will keep your commit info and author information in-tact. If you don't feel like dealing with it, though, I can just add the spkg-configure.m4 onto my branch and re-push.

After adding flint to the DEPCHECK, I think only two questions remains:

• Do we need to check for flint support in the spkg-configure check? Given that flint support is "automagically" enabled, I don't think it's necessary on source-based distros, who should have flint support enabled unconditionally to avoid problems. If none of the binary distros ship a flint-less linbox, then we can skip it.
• Should we try to set the version bound to 1.5.2? Someone would need to test that.

[mkoeppe]

These questions can be resolved by adding system package information to build/pkgs/linbox/distros and then testing...

[mkoeppe]

The duplicate ticket #29484 reminds us to "(attempt to) lower the bound on fflas-ffpack in its spkg-configure.m4. If we can accept an older system linbox that itself uses an older system fflas-ffpack, that would be nice."

[mkoeppe]

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[slelievre]

Replying to mkoeppe:

These questions can be resolved by adding system package information to build/pkgs/linbox/distros and then testing...

Here we go.

[slelievre]

New commits:

​0e7917e	Trac #29631: don't force SHELL=bash when installing linbox.
​c238f84	Trac #29631: drop fplll from linbox's dependencies.
​bb670d8	Trac #29631: disable additional unused features of the linbox SPKG.
​6864ae5	29361: Add Fedora and Gentoo linbox package info

[mkoeppe]

The branch does not do what ticket title and description promise

[mkoeppe]

Setting a new milestone for this ticket based on a cursory review.

[mjo]

let's unforget about this

---

New commits:

​19b93eb	Trac #29631: don't force SHELL=bash when installing linbox.
​6568cfb	Trac #29631: drop fplll from linbox's dependencies.
​e5c12ca	Trac #29631: disable additional unused features of the linbox SPKG.
​83c89a4	29361: Add Fedora and Gentoo linbox package info
​756f9ee	Trac #29631: new spkg-configure.m4 for linbox.

[mkoeppe]

+# Disable all additional features of linbox that aren't needed by sage.
+# This helps keep the dependency list small, and makes it more likely
+# that we can use the system copy of linbox (which may be missing some
+# of these features).

Note that the purpose of the SPKG_DEPCHECK logic is as follows: The system package MIGHT link to an optional dependency like fplll. Therefore, if we build our own fplll, we need to avoid using system linbox.

It does not help at all to remove features from our build of the linbox spkg.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​c5d523b

Trac #29631: new spkg-configure.m4 for linbox.

[mjo]

Replying to mkoeppe:

+# Disable all additional features of linbox that aren't needed by sage.
+# This helps keep the dependency list small, and makes it more likely
+# that we can use the system copy of linbox (which may be missing some
+# of these features).

Note that the purpose of the SPKG_DEPCHECK logic is as follows: The system package MIGHT link to an optional dependency like fplll. Therefore, if we build our own fplll, we need to avoid using system linbox.

Ideally we would detect that optional linkage using AC_SEARCH_LIBS, so that we only reject a system linbox if it actually links to something on the system that is also installed (and linked into sage) as an SPKG. But DEPCHECK is much easier than digging through the source looking for public functions hidden behind e.g. HAVE_IML, so I've just gone ahead and used DEPCHECK for now.

It does not help at all to remove features from our build of the linbox spkg.

Not having those features in the SPKG means that we don't have to insist that they be present in the system copy.

[mkoeppe]

Replying to mjo:

It does not help at all to remove features from our build of the linbox spkg.

Not having those features in the SPKG means that we don't have to insist that they be present in the system copy.

We don't have to insist on it either way. No need to remove the feature.

[mjo]

Replying to mkoeppe:

Not having those features in the SPKG means that we don't have to insist that they be present in the system copy.

We don't have to insist on it either way. No need to remove the feature.

You want the linbox SPKG to have mpfr, ntl, and iml support... but also accept a system linbox that doesn't? And will crash the moment anyone tries to use those optional features?

[mkoeppe]

Replying to mjo:

You want the linbox SPKG to have mpfr, ntl, and iml support... but also accept a system linbox that doesn't? And will crash the moment anyone tries to use those optional features?

The same is also true when a user uses a system linbox that provides the optional features.

[mjo]

Replying to mkoeppe:

Replying to mjo:

You want the linbox SPKG to have mpfr, ntl, and iml support... but also accept a system linbox that doesn't? And will crash the moment anyone tries to use those optional features?

The same is also true when a user uses a system linbox that provides the optional features.

A user can write his own code that depends on his own special system copy of linbox, but he won't get it committed to sage, which is the risk we run retaining the optional features in the SPKG. And the general trend is from-SPKG-to-system-package -- not the other way around -- minimizing the risk that someone will shoot himself in the foot that way. Finally, we're not doing "two wrong make a right." Avoiding the one potential problem is a win. Sage doesn't need those features enabled. Why slow down the build, bloat the install, and invite future bug reports?

[mkoeppe]

Removing the features is neither an improvement, nor a prerequisite for the goal of the ticket. So it's not appropriate for this ticket.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​85b19db	29361: Add Fedora and Gentoo linbox package info
​7072119	Trac #29631: new spkg-configure.m4 for linbox.

[mjo]

As you wish.

[mkoeppe]

Thanks. The branch still has this change though:

--- a/build/pkgs/linbox/dependencies
+++ b/build/pkgs/linbox/dependencies
@@ -1,4 +1,4 @@
-$(MP_LIBRARY) ntl givaro mpfr fplll iml flint fflas_ffpack
+$(MP_LIBRARY) ntl givaro mpfr iml flint fflas_ffpack

[mjo]

Replying to mkoeppe:

Thanks. The branch still has this change though:

--- a/build/pkgs/linbox/dependencies
+++ b/build/pkgs/linbox/dependencies
@@ -1,4 +1,4 @@
-$(MP_LIBRARY) ntl givaro mpfr fplll iml flint fflas_ffpack
+$(MP_LIBRARY) ntl givaro mpfr iml flint fflas_ffpack

Its spkg-install explicitly disables fplll.

[mkoeppe]

Ah, thanks.

[mkoeppe]

Looking good then, but this will need platform testing on GH Actions

[tornaria]

After fixing the system linbox package (as in #27205) this works for me on void linux.

[dimpase]

rebased over the latest beta

---

New commits:

​4758635	Trac #29631: don't force SHELL=bash when installing linbox.
​3becab5	Trac #29631: drop fplll from linbox's dependencies.
​b51dba4	29361: Add Fedora and Gentoo linbox package info
​794713e	Trac #29631: new spkg-configure.m4 for linbox.

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​2240cc1	build/pkgs/pyzmq: Update to 22.3.0
​fc11612	build/pkgs/zeromq: Update to 4.3.4, add upstream_url
​a8fe9da	build/pkgs/zeromq/patches/getrandom.patch: Remove
​078e37b	build/pkgs/setuptools: Update to 59.1.0, remove patch
​ee4f6b4	Merge tag '9.5.beta7' into t/32828/pyzmq__build_errors
​b430bcf	build/pkgs/setuptools: Update to 59.2.0
​c986098	Merge branch 'u/mkoeppe/pyzmq__build_errors' of trac.sagemath.org:sage into u/mjo/ticket/29631
​53cfadf	update cysignals to 1.11.0
​5ed4717	update cysignals to 1.11.1
​f1fdff2	Merge branch 'public/32576' of trac.sagemath.org:sage into u/mjo/ticket/29631

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​3795599	Trac #8450: update the output from a few plotting doctests.
​518349e	Trac #8450: pass imaginary tolerance down through plot.options.
​5b46a44	Trac #8450: remove imag_tol default from FastCallablePlotWrapper().
​b47adf4	Trac #8450: use CDF for symbolic expression fast-callables.
​8b43bb8	Trac #8450: catch more cases in setup_for_eval_on_grid.
​77ed41c	Trac #8450: fix complex type error in real_nth_root().
​5a70ea6	Trac #8450: add an example using "imaginary_tolerance" in plot().
​7899e73	Merge remote-tracking branch 'trac/develop' into u/mjo/ticket/8450
​2c086e2	add missing imports
​00a28d5	Merge branch 'u/dimpase/rebased/mjo/ticket/8450' of trac.sagemath.org:sage into u/mjo/ticket/29631

[dimpase]

tests on ​https://github.com/sagemath/sagetrac-mirror/actions/runs/1538684506

[dimpase]

currently the branch includes some positively reviewed tickets - most of them to increase chances of more systems passing.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7e16ea9

correct the package name: `-`->`_`

[dimpase]

the last commit should not be forgotten while cutting the final branch here...

[dimpase]

also need

build/pkgs/m4ri/distros/debian.txt

@@ +1 @@
+libm4ri-dev

[dimpase]

and

new file uild/pkgs/fplll/distros/debian.txt

@@ +1 @@
+libfplll-dev

[slelievre]

New commits:

​60e69ed

29631: add fplll and libm4ri debian distro info

[slelievre]

Added the changes proposed in @dimpase's last two comments: comment:43, comment:44.

[mkoeppe]

Replying to dimpase:

currently the branch includes some positively reviewed tickets - most of them to increase chances of more systems passing.

These should be listed in the Dependencies field

[mkoeppe]

In light of #33042, let's also set an upper bound for the linbox version

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​56aba2f

29631: upper bound for linbox version

[slelievre]

Like that?

[mjo]

Replying to slelievre:

Like that?

LGTM. The linbox team doesn't really do maintenance releases, so even if we set the bound to < 1.7.0, it would have the same effect. sage_spkg_install_linbox=no is the default so it's not really necessary but it won't hurt anything.

[slelievre]

I mimicked build/pkgsfplll/spkg-configure.m4.

Is anything else needed here?

[mjo]

Replying to slelievre:

Is anything else needed here?

I think we need #8450 to be merged to clear up the merge conflict but otherwise the commits for this ticket are good to go.

[mjo]

I don't think #8450 is going to make it into v9.5, maybe time to rebase this with just the linbox commits?

[slelievre]

#8450 made it into Sage 9.6.beta0.

[mjo]

Replying to slelievre:

#8450 made it into Sage 9.6.beta0.

Merge failed though D:

[mjo]

New commits:

​ab58ae4	Trac #29631: don't force SHELL=bash when installing linbox.
​94d7717	Trac #29631: drop fplll from linbox's dependencies.
​7afd361	29361: Add Fedora and Gentoo linbox package info
​8128f66	Trac #29631: new spkg-configure.m4 for linbox.
​48eab64	add missing imports
​91aee3b	correct the package name: `-`->`_`
​6ee6942	29631: add fplll and libm4ri debian distro info
​fc083c2	29631: upper bound for linbox version

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​9a8bf64	Trac #29631: don't force SHELL=bash when installing linbox.
​9f788f5	Trac #29631: drop fplll from linbox's dependencies.
​796b0c3	29361: Add Fedora and Gentoo linbox package info
​9b7d1ec	Trac #29631: new spkg-configure.m4 for linbox.
​a5d8c72	correct the package name: `-`->`_`
​4d41356	29631: add fplll and libm4ri debian distro info
​bc7a6f8	29631: upper bound for linbox version

[mjo]

Ping, this one should be easy now:

• Very few lines changed
• We require the one exact version of linbox that sage itself ships
• It already went through Github CI

[dimpase]

lgtm

[mjo]

Thanks! Now I guess I have to test the maxima ticket on github.

[mkoeppe]

This ticket makes no sense because we have no spkg-configure.m4 for the dependency fflas_ffpack

[mkoeppe]

Sorry for the noise - I was wrong about that (misled by a typo that I made)

[dimpase]

I've missed that build/pkgs/linbox/distros/fedora.txt also needs linbox-devel