Opened 2 years ago

Closed 19 months ago

#31069 closed enhancement (fixed)

Upgrade: Flint 2.7.1

Reported by: Samuel Lelièvre Owned by:
Priority: critical Milestone: sage-9.4
Component: packages: standard Keywords: upgrade, flint
Cc: Antonio Rojas, Dima Pasechnik, François Bissey, Timo Kaufmann, Isuru Fernando, Michael Orlitzky, Matthias Köppe, Samuel Lelièvre, Julian Rüth, Vincent Delecroix, gh-kliem Merged in:
Authors: Vincent Delecroix, Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 8c71a1c (Commits, GitHub, GitLab) Commit: 8c71a1c694075d021008794ef3a6fd461ec4dc0e
Dependencies: #25993 Stopgaps:

Status badges

Description (last modified by Vincent Delecroix)

This is to upgrade to Flint 2.7.1.

Our last upgrade was to Flint 2.6.3 in #29719.

Changes in FLINT 2.7.0 (from NEWS file)

2.7.0 -- 11 Dec 2020

  • Multivariate factorisation
  • Square root and square testing for finite fields
  • Square root and square testing for multivariates
  • Zassenhaus factoring speedups (incl. degree pruning)
  • Fast factorisation of cubic univariate polynomials
  • Add context objects to fmpz_mod_poly functions
  • Use BLAS for matrix multiplication over Z/nZ (small n)
  • Linear solving for non-square/singular matrices (can_solve)
  • Speed up factorisation over Z/nZ (for multiprecision n)

From the packaging point of view the main changes will be

  • configuring BLAS
  • pthread support
  • context object in fmpz_mod_poly function (API change)

Related:

Change History (60)

comment:1 Changed 2 years ago by Samuel Lelièvre

Cc: Antonio Rojas Dima Pasechnik François Bissey Timo Kaufmann Isuru Fernando Michael Orlitzky Matthias Köppe Samuel Lelièvre added

comment:2 Changed 2 years ago by François Bissey

Description: modified (diff)

comment:3 Changed 2 years ago by François Bissey

Description: modified (diff)

comment:4 Changed 2 years ago by Samuel Lelièvre

Flint 2.7.0 was released today (2020-12-18).

comment:5 Changed 2 years ago by Antonio Rojas

Cc: Julian Rüth Vincent Delecroix added

singular and e-antic fail to build due to the API changes in fmpz_mod_poly

comment:6 Changed 2 years ago by Antonio Rojas

Actually, Singular is fixed in the (just released) 4.2.0 version. Unfortunately doesn't help much since the Singular upgrade has been stuck for years in #25993

comment:7 Changed 2 years ago by Vincent Delecroix

Description: modified (diff)

comment:8 Changed 2 years ago by Antonio Rojas

sagelib itself builds fine with flint 2.7 and passes doctests, modulo the singular upgrade.

comment:9 Changed 2 years ago by Matthias Köppe

Milestone: sage-9.3sage-9.4

comment:10 Changed 23 months ago by Kiran Kedlaya

For the record, we are now up to Flint 2.7.1. As the timing and numbering suggest, this is mainly a bugfix release.

comment:11 Changed 23 months ago by Thierry Thomas

e-antic has been fixed (in head, to be 1.0.0).

comment:12 Changed 22 months ago by Samuel Lelièvre

Description: modified (diff)
Summary: Upgrade: Flint 2.7.0Upgrade: Flint 2.7.1

comment:13 Changed 22 months ago by Vincent Delecroix

Description: modified (diff)

comment:14 Changed 22 months ago by Vincent Delecroix

Description: modified (diff)

comment:15 Changed 22 months ago by Vincent Delecroix

Authors: Vincent Delecroix
Branch: public/31069
Commit: 29c0477d4aeb9394df2c80cc1628f244647ce40d

New commits:

73bd1a731069: flint version upgrade
29c047731069: flint declarations

comment:16 Changed 22 months ago by Vincent Delecroix

I did not have any trouble to build singular. What was the issue?

comment:17 in reply to:  16 ; Changed 22 months ago by Antonio Rojas

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

comment:18 in reply to:  17 ; Changed 22 months ago by Vincent Delecroix

Replying to arojas:

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

4.1.1p2.p0 shipped with sage 9.3.beta7

comment:19 in reply to:  18 Changed 22 months ago by Antonio Rojas

Replying to vdelecroix:

Replying to arojas:

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

4.1.1p2.p0 shipped with sage 9.3.beta7

It builds because it's not using flint

checking for FLINT >= 2.4... problem
Sorry, your FLINT version is too old. Disabling.
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)

comment:20 Changed 22 months ago by Matthias Köppe

Milestone: sage-9.4sage-9.3
Priority: majorcritical

comment:21 Changed 22 months ago by Vincent Delecroix

Dependencies: #25993

comment:22 Changed 22 months ago by Matthias Köppe

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

comment:23 in reply to:  22 Changed 22 months ago by Vincent Delecroix

Replying to mkoeppe:

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

That would be an alternative. But because of the API change I believe it is not enough. Let me give it a try.

comment:24 in reply to:  22 Changed 22 months ago by Antonio Rojas

Replying to mkoeppe:

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

Then it will fail to build with real compilation errors, unless you also backport the flint 2.7 support patches for singular

comment:25 Changed 22 months ago by Matthias Köppe

perhaps just Singular commit 2e7e3dc89e04ca1ef4175816cd23db6780ad5a38?

comment:26 Changed 22 months ago by Matthias Köppe

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

comment:27 in reply to:  26 ; Changed 22 months ago by Antonio Rojas

Replying to mkoeppe:

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch

Needs massive ifdef'ing if you still want to support flint <2.7

Last edited 22 months ago by Antonio Rojas (previous) (diff)

comment:28 in reply to:  27 Changed 22 months ago by Matthias Köppe

Replying to arojas:

Replying to mkoeppe:

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch

Needs massive ifdef'ing if you still want to support flint <2.7

Thanks! I've prepared this in https://github.com/mkoeppe/e-antic/tree/flint-2.7, testing currently at https://github.com/mkoeppe/e-antic/actions/runs/593267164

Last edited 22 months ago by Matthias Köppe (previous) (diff)

comment:29 Changed 22 months ago by Matthias Köppe

Opened #31430 for e-antic patch (or ideally new 0.1.x release)

comment:30 Changed 22 months ago by Matthias Köppe

Dependencies: #25993#25993, #31430

comment:31 Changed 21 months ago by Matthias Köppe

Dependencies: #25993, #31430#25993

comment:32 Changed 21 months ago by git

Commit: 29c0477d4aeb9394df2c80cc1628f244647ce40ddbc64778ddf69fe39218fc406656e559aaddd096

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

871fdbd31069: flint version upgrade
dbc647731069: flint declarations

comment:33 Changed 21 months ago by git

Commit: dbc64778ddf69fe39218fc406656e559aaddd0968e3040c2a22fbd256f088ca58619442ae13af362

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

797bacdsrc/sage/rings/polynomial/multi_polynomial_ideal.py: Use sorted() to make doctests more stable
31fe1ebbuild/pkgs/singular/spkg-install.in: Build the documentation if the Singular tarball does not contain it
6c01591Merge #31439
f3119b1Merge tag '9.3.beta9' into t/25993/public/singular4-1-3
49de2ffbuild/pkgs/singular: Use 4.2.0p1+2021-03-13+sage
f816f07build/pkgs/singular/spkg-install.in: Use --disable-static on all platforms
b0d7748build/pkgs/singular/spkg-install.in: Build subdirs in parallel
c33ca2bsrc/sage/rings/polynomial/{laurent,multi}_polynomial_ideal: Use sorted(..., key=str) to make doctests stable
ec471e0build/pkgs/singular/SPKG.rst: Add link to branch
8e3040cMerge #25993

comment:34 Changed 21 months ago by Matthias Köppe

Authors: Vincent DelecroixVincent Delecroix, Matthias Koeppe
Status: newneeds_review

comment:35 Changed 21 months ago by Matthias Köppe

Reviewers: https://github.com/mkoeppe/sage/actions/runs/666698487

comment:36 Changed 21 months ago by Matthias Köppe

Cc: gh-kliem added
Status: needs_reviewneeds_work

There are still problems with e_antic (0.1.9):

ubuntu-trusty-maximal (https://github.com/mkoeppe/sage/runs/2145415921 - uses FLINT from SPKG):

  [e_antic-0.1.9]   checking for gmp.h... yes
  [e_antic-0.1.9]   checking for __gmpz_add in -lgmp... yes
  [e_antic-0.1.9]   checking for flint/flint.h... yes
  [e_antic-0.1.9]   checking for flint/fmpz.h... no
  [e_antic-0.1.9]   configure: error: flint header not found

cygwin-standard (https://github.com/mkoeppe/sage/runs/2145195263 - uses system FLINT 2.7)

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -fopenmp -O2 -g -march=native -c poly_extra/fmpz_nextprime.c -o poly_extra/fmpz_nextprime.o >/dev/null 2>&1
poly_extra/fmpz_randprime.c:17:6: error: conflicting types for 'fmpz_randprime'
   17 | void fmpz_randprime(fmpz_t f, flint_rand_t state, mp_bitcnt_t bits, int proved)
      |      ^~~~~~~~~~~~~~
In file included from /usr/include/flint/fmpz_poly.h:33,
                 from ./e-antic/poly_extra.h:17,
                 from poly_extra/fmpz_randprime.c:13:
/usr/include/flint/fmpz.h:188:16: note: previous declaration of 'fmpz_randprime' was here
  188 | FLINT_DLL void fmpz_randprime(fmpz_t f, flint_rand_t state,
      |                ^~~~~~~~~~~~~~
make[4]: *** [Makefile:2699: poly_extra/fmpz_randprime.lo] Error 1
Last edited 21 months ago by Matthias Köppe (previous) (diff)

comment:37 Changed 21 months ago by Matthias Köppe

OTOH, on ubuntu-xenial-maximal (https://github.com/mkoeppe/sage/runs/2145415962)and ubuntu-bionic-maximal (https://github.com/mkoeppe/sage/runs/2145416018), e_antic installs OK. We can just write off ubuntu-trusty - Normaliz does not compile with the ancient compiler there either.

Last edited 21 months ago by Matthias Köppe (previous) (diff)

comment:38 Changed 21 months ago by Matthias Köppe

Status: needs_workneeds_review

comment:39 Changed 21 months ago by Matthias Köppe

Status: needs_reviewneeds_work

A serious issue: On ubuntu-focal-standard (https://github.com/mkoeppe/sage/runs/2145416039)

  [sagelib-9.3.beta9]   In file included from build/cythonized/sage/data_structures/bounded_integer_sequences.c:3535:
  [sagelib-9.3.beta9]   sage/libs/flint/flint_wrap.h: At top level:
  [sagelib-9.3.beta9]   sage/libs/flint/flint_wrap.h:39:10: fatal error: flint/fmpz_mod.h: No such file or directory
  [sagelib-9.3.beta9]      39 | #include <flint/fmpz_mod.h>
  [sagelib-9.3.beta9]         |          ^~~~~~~~~~~~~~~~~~
  [sagelib-9.3.beta9]   compilation terminated.

This is using system flint 2.5.2 (see https://repology.org/project/flint/versions)

comment:40 Changed 21 months ago by Matthias Köppe

(fmpz_mod.h was introduced in 2.6.0 according to https://github.com/wbhart/flint2/blob/trunk/NEWS)

Last edited 21 months ago by Matthias Köppe (previous) (diff)

comment:41 Changed 21 months ago by Matthias Köppe

I think dropping support for system FLINT 2.5.x is too disruptive for Sage 9.3, given that the new features are not used yet by anything.

So I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.

comment:42 Changed 21 months ago by git

Commit: 8e3040c2a22fbd256f088ca58619442ae13af3628c71a1c694075d021008794ef3a6fd461ec4dc0e

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8c71a1c31069: flint version upgrade

comment:43 in reply to:  41 Changed 21 months ago by Matthias Köppe

Replying to mkoeppe:

I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.

This is now #31525.

comment:44 Changed 21 months ago by Matthias Köppe

Status: needs_workneeds_review

comment:45 Changed 21 months ago by Michael Orlitzky

I think this is OK. The main issues were in other packages that have already been fixed. With system flint-2.7.1, I see only one doctest failure due to a timeout in pari that probably cannot be blamed on flint.

I see "unrelated" failures in the CI runs as well. If you are convinced of flint's innocence, we can set this to positive review.

comment:46 Changed 21 months ago by Dima Pasechnik

Reviewers: https://github.com/mkoeppe/sage/actions/runs/666698487Dima Pasechnik
Status: needs_reviewpositive_review

lgtm too

comment:47 Changed 21 months ago by Matthias Köppe

Thanks!

comment:48 Changed 21 months ago by Matthias Köppe

Priority: criticalblocker

Setting priority to blocker to bring this ticket to the attention of the release bot.

comment:49 Changed 21 months ago by David Roe

After merging this ticket into #31548 (since I want to use new functionality from FLINT 2.7.1 there) I'm getting errors starting Sage. I don't usually update spkgs outside of beta releases, so I could be doing something wrong (I'm running OS X 10.15.2).

$ git trac fetch 31069 && git merge
$ make build
...
Sage build/upgrade complete!
$ sage
ImportError Python 3.9.1: /Users/roed/sage/sage-9.3.beta7/local/bin/python3
...
ImportError: dlopen(/Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so, 2): Library not loaded: /Users/roed/sage/sage-9.3.beta7/local/lib/libSingular-4.1.1.dylib
  Referenced from: /Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so
  Reason: image not found

Full install.log available here.

comment:50 Changed 21 months ago by François Bissey

I believe you need singular 4.2.0p1 (9.3.beta8 if I am not mistaken) with that. Singular uses flint so does need rebuilding and 4.2.0 is the first one that is compatible with flint-2.7.1.

comment:51 Changed 21 months ago by David Roe

Despite the folder name, this is based on sage-9.3.beta9, and when I did make build it appeared to be building singular-4.2.0p1+2021-03-13+sage.

comment:52 Changed 21 months ago by David Roe

I feel like I'm missing something basic here: when I try running sage -f flint it gives me lots of errors saying that flint-2.7.1 isn't found:

[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors
[flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz
[flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
[flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz'
[flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz
...
[flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network
[flint-2.7.1] ************************************************************************
[flint-2.7.1] ************************************************************************
[flint-2.7.1] Error downloading flint-2.7.1.tar.gz

Do I need to download a tarball manually from somewhere?

comment:53 in reply to:  51 Changed 21 months ago by François Bissey

Replying to roed:

Despite the folder name, this is based on sage-9.3.beta9, and when I did make build it appeared to be building singular-4.2.0p1+2021-03-13+sage.

OK, then it appears that somehow sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so has not be rebuilt because it is still looking for libSingular-4.1.1 when it should look for libSingular-4.2.0.

comment:54 in reply to:  52 Changed 21 months ago by François Bissey

Replying to roed:

I feel like I'm missing something basic here: when I try running sage -f flint it gives me lots of errors saying that flint-2.7.1 isn't found:

[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors
[flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz
[flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
[flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz'
[flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz
...
[flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network
[flint-2.7.1] ************************************************************************
[flint-2.7.1] ************************************************************************
[flint-2.7.1] Error downloading flint-2.7.1.tar.gz

Do I need to download a tarball manually from somewhere?

This ticket is not merged, tarball are not usually mirrored before Volker attempt to merge the tickets. You'll have to fetch the source a different way. Matthias created a handy way of using a string called upstream_url in the file checksum.ini to do the fetching from upstream but I cannot remember how it works.

comment:55 Changed 21 months ago by Samuel Lelièvre

Do this:

$ ./bootstrap -q
$ ./configure --enable-download-from-upstream-url -q
$ make -s V=0

comment:56 Changed 21 months ago by David Roe

Thanks! That seems to have worked.

comment:57 Changed 20 months ago by David Roe

For anyone else looking at this, I also had to do sage -f flint after the commands Samuel gave.

comment:58 Changed 20 months ago by Samuel Lelièvre

This should do everything at once:

$ ./bootstrap -q
$ ./configure --enable-download-from-upstream-url -q
$ make -s V=0 flint-clean
$ make -s V=0

comment:59 Changed 20 months ago by Matthias Köppe

Milestone: sage-9.3sage-9.4
Priority: blockercritical

Looks unrealistic to squeeze this into 9.3

comment:60 Changed 19 months ago by Volker Braun

Branch: public/310698c71a1c694075d021008794ef3a6fd461ec4dc0e
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.