Opened 2 years ago

Closed 21 months ago

#31069 closed enhancement (fixed)

Upgrade: Flint 2.7.1

Reported by: slelievre Owned by:
Priority: critical Milestone: sage-9.4
Component: packages: standard Keywords: upgrade, flint
Cc: arojas, dimpase, fbissey, gh-timokau, isuruf, mjo, mkoeppe, slelievre, saraedum, vdelecroix, gh-kliem Merged in:
Authors: Vincent Delecroix, Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 8c71a1c (Commits, GitHub, GitLab) Commit: 8c71a1c694075d021008794ef3a6fd461ec4dc0e
Dependencies: #25993 Stopgaps:

Status badges

Description (last modified by vdelecroix)

This is to upgrade to Flint 2.7.1.

Our last upgrade was to Flint 2.6.3 in #29719.

Changes in FLINT 2.7.0 (from NEWS file)

2.7.0 -- 11 Dec 2020

  • Multivariate factorisation
  • Square root and square testing for finite fields
  • Square root and square testing for multivariates
  • Zassenhaus factoring speedups (incl. degree pruning)
  • Fast factorisation of cubic univariate polynomials
  • Add context objects to fmpz_mod_poly functions
  • Use BLAS for matrix multiplication over Z/nZ (small n)
  • Linear solving for non-square/singular matrices (can_solve)
  • Speed up factorisation over Z/nZ (for multiprecision n)

From the packaging point of view the main changes will be

  • configuring BLAS
  • pthread support
  • context object in fmpz_mod_poly function (API change)

Related:

Change History (60)

comment:1 Changed 2 years ago by slelievre

Cc: arojas dimpase fbissey gh-timokau isuruf mjo mkoeppe slelievre added

comment:2 Changed 2 years ago by fbissey

Description: modified (diff)

comment:3 Changed 2 years ago by fbissey

Description: modified (diff)

comment:4 Changed 2 years ago by slelievre

Flint 2.7.0 was released today (2020-12-18).

comment:5 Changed 2 years ago by arojas

Cc: saraedum vdelecroix added

singular and e-antic fail to build due to the API changes in fmpz_mod_poly

comment:6 Changed 2 years ago by arojas

Actually, Singular is fixed in the (just released) 4.2.0 version. Unfortunately doesn't help much since the Singular upgrade has been stuck for years in #25993

comment:7 Changed 2 years ago by vdelecroix

Description: modified (diff)

comment:8 Changed 2 years ago by arojas

sagelib itself builds fine with flint 2.7 and passes doctests, modulo the singular upgrade.

comment:9 Changed 2 years ago by mkoeppe

Milestone: sage-9.3sage-9.4

comment:10 Changed 2 years ago by kedlaya

For the record, we are now up to Flint 2.7.1. As the timing and numbering suggest, this is mainly a bugfix release.

comment:11 Changed 2 years ago by gh-thierry-FreeBSD

e-antic has been fixed (in head, to be 1.0.0).

comment:12 Changed 2 years ago by slelievre

Description: modified (diff)
Summary: Upgrade: Flint 2.7.0Upgrade: Flint 2.7.1

comment:13 Changed 2 years ago by vdelecroix

Description: modified (diff)

comment:14 Changed 2 years ago by vdelecroix

Description: modified (diff)

comment:15 Changed 2 years ago by vdelecroix

Authors: Vincent Delecroix
Branch: public/31069
Commit: 29c0477d4aeb9394df2c80cc1628f244647ce40d

New commits:

73bd1a731069: flint version upgrade
29c047731069: flint declarations

comment:16 Changed 2 years ago by vdelecroix

I did not have any trouble to build singular. What was the issue?

comment:17 in reply to:  16 ; Changed 2 years ago by arojas

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

comment:18 in reply to:  17 ; Changed 2 years ago by vdelecroix

Replying to arojas:

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

4.1.1p2.p0 shipped with sage 9.3.beta7

comment:19 in reply to:  18 Changed 2 years ago by arojas

Replying to vdelecroix:

Replying to arojas:

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

4.1.1p2.p0 shipped with sage 9.3.beta7

It builds because it's not using flint

checking for FLINT >= 2.4... problem
Sorry, your FLINT version is too old. Disabling.
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)

comment:20 Changed 2 years ago by mkoeppe

Milestone: sage-9.4sage-9.3
Priority: majorcritical

comment:21 Changed 2 years ago by vdelecroix

Dependencies: #25993

comment:22 Changed 2 years ago by mkoeppe

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

comment:23 in reply to:  22 Changed 2 years ago by vdelecroix

Replying to mkoeppe:

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

That would be an alternative. But because of the API change I believe it is not enough. Let me give it a try.

comment:24 in reply to:  22 Changed 2 years ago by arojas

Replying to mkoeppe:

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

Then it will fail to build with real compilation errors, unless you also backport the flint 2.7 support patches for singular

comment:25 Changed 2 years ago by mkoeppe

perhaps just Singular commit 2e7e3dc89e04ca1ef4175816cd23db6780ad5a38?

comment:26 Changed 2 years ago by mkoeppe

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

comment:27 in reply to:  26 ; Changed 2 years ago by arojas

Replying to mkoeppe:

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch

Needs massive ifdef'ing if you still want to support flint <2.7

Last edited 2 years ago by arojas (previous) (diff)

comment:28 in reply to:  27 Changed 2 years ago by mkoeppe

Replying to arojas:

Replying to mkoeppe:

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch

Needs massive ifdef'ing if you still want to support flint <2.7

Thanks! I've prepared this in https://github.com/mkoeppe/e-antic/tree/flint-2.7, testing currently at https://github.com/mkoeppe/e-antic/actions/runs/593267164

Last edited 2 years ago by mkoeppe (previous) (diff)

comment:29 Changed 2 years ago by mkoeppe

Opened #31430 for e-antic patch (or ideally new 0.1.x release)

comment:30 Changed 2 years ago by mkoeppe

Dependencies: #25993#25993, #31430

comment:31 Changed 23 months ago by mkoeppe

Dependencies: #25993, #31430#25993

comment:32 Changed 23 months ago by git

Commit: 29c0477d4aeb9394df2c80cc1628f244647ce40ddbc64778ddf69fe39218fc406656e559aaddd096

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

871fdbd31069: flint version upgrade
dbc647731069: flint declarations

comment:33 Changed 23 months ago by git

Commit: dbc64778ddf69fe39218fc406656e559aaddd0968e3040c2a22fbd256f088ca58619442ae13af362

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

797bacdsrc/sage/rings/polynomial/multi_polynomial_ideal.py: Use sorted() to make doctests more stable
31fe1ebbuild/pkgs/singular/spkg-install.in: Build the documentation if the Singular tarball does not contain it
6c01591Merge #31439
f3119b1Merge tag '9.3.beta9' into t/25993/public/singular4-1-3
49de2ffbuild/pkgs/singular: Use 4.2.0p1+2021-03-13+sage
f816f07build/pkgs/singular/spkg-install.in: Use --disable-static on all platforms
b0d7748build/pkgs/singular/spkg-install.in: Build subdirs in parallel
c33ca2bsrc/sage/rings/polynomial/{laurent,multi}_polynomial_ideal: Use sorted(..., key=str) to make doctests stable
ec471e0build/pkgs/singular/SPKG.rst: Add link to branch
8e3040cMerge #25993

comment:34 Changed 23 months ago by mkoeppe

Authors: Vincent DelecroixVincent Delecroix, Matthias Koeppe
Status: newneeds_review

comment:35 Changed 23 months ago by mkoeppe

Reviewers: https://github.com/mkoeppe/sage/actions/runs/666698487

comment:36 Changed 23 months ago by mkoeppe

Cc: gh-kliem added
Status: needs_reviewneeds_work

There are still problems with e_antic (0.1.9) - uses FLINT from SPKG:

ubuntu-trusty-maximal (https://github.com/mkoeppe/sage/runs/2145415921):

  [e_antic-0.1.9]   checking for gmp.h... yes
  [e_antic-0.1.9]   checking for __gmpz_add in -lgmp... yes
  [e_antic-0.1.9]   checking for flint/flint.h... yes
  [e_antic-0.1.9]   checking for flint/fmpz.h... no
  [e_antic-0.1.9]   configure: error: flint header not found

cygwin-standard (https://github.com/mkoeppe/sage/runs/2145195263 - uses system FLINT 2.7)

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -fopenmp -O2 -g -march=native -c poly_extra/fmpz_nextprime.c -o poly_extra/fmpz_nextprime.o >/dev/null 2>&1
poly_extra/fmpz_randprime.c:17:6: error: conflicting types for 'fmpz_randprime'
   17 | void fmpz_randprime(fmpz_t f, flint_rand_t state, mp_bitcnt_t bits, int proved)
      |      ^~~~~~~~~~~~~~
In file included from /usr/include/flint/fmpz_poly.h:33,
                 from ./e-antic/poly_extra.h:17,
                 from poly_extra/fmpz_randprime.c:13:
/usr/include/flint/fmpz.h:188:16: note: previous declaration of 'fmpz_randprime' was here
  188 | FLINT_DLL void fmpz_randprime(fmpz_t f, flint_rand_t state,
      |                ^~~~~~~~~~~~~~
make[4]: *** [Makefile:2699: poly_extra/fmpz_randprime.lo] Error 1
Version 0, edited 23 months ago by mkoeppe (next)

comment:37 Changed 23 months ago by mkoeppe

OTOH, on ubuntu-xenial-maximal (https://github.com/mkoeppe/sage/runs/2145415962)and ubuntu-bionic-maximal (https://github.com/mkoeppe/sage/runs/2145416018), e_antic installs OK. We can just write off ubuntu-trusty - Normaliz does not compile with the ancient compiler there either.

Last edited 23 months ago by mkoeppe (previous) (diff)

comment:38 Changed 23 months ago by mkoeppe

Status: needs_workneeds_review

comment:39 Changed 23 months ago by mkoeppe

Status: needs_reviewneeds_work

A serious issue: On ubuntu-focal-standard (https://github.com/mkoeppe/sage/runs/2145416039)

  [sagelib-9.3.beta9]   In file included from build/cythonized/sage/data_structures/bounded_integer_sequences.c:3535:
  [sagelib-9.3.beta9]   sage/libs/flint/flint_wrap.h: At top level:
  [sagelib-9.3.beta9]   sage/libs/flint/flint_wrap.h:39:10: fatal error: flint/fmpz_mod.h: No such file or directory
  [sagelib-9.3.beta9]      39 | #include <flint/fmpz_mod.h>
  [sagelib-9.3.beta9]         |          ^~~~~~~~~~~~~~~~~~
  [sagelib-9.3.beta9]   compilation terminated.

This is using system flint 2.5.2 (see https://repology.org/project/flint/versions)

comment:40 Changed 23 months ago by mkoeppe

(fmpz_mod.h was introduced in 2.6.0 according to https://github.com/wbhart/flint2/blob/trunk/NEWS)

Last edited 23 months ago by mkoeppe (previous) (diff)

comment:41 Changed 23 months ago by mkoeppe

I think dropping support for system FLINT 2.5.x is too disruptive for Sage 9.3, given that the new features are not used yet by anything.

So I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.

comment:42 Changed 23 months ago by git

Commit: 8e3040c2a22fbd256f088ca58619442ae13af3628c71a1c694075d021008794ef3a6fd461ec4dc0e

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8c71a1c31069: flint version upgrade

comment:43 in reply to:  41 Changed 23 months ago by mkoeppe

Replying to mkoeppe:

I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.

This is now #31525.

comment:44 Changed 23 months ago by mkoeppe

Status: needs_workneeds_review

comment:45 Changed 23 months ago by mjo

I think this is OK. The main issues were in other packages that have already been fixed. With system flint-2.7.1, I see only one doctest failure due to a timeout in pari that probably cannot be blamed on flint.

I see "unrelated" failures in the CI runs as well. If you are convinced of flint's innocence, we can set this to positive review.

comment:46 Changed 23 months ago by dimpase

Reviewers: https://github.com/mkoeppe/sage/actions/runs/666698487Dima Pasechnik
Status: needs_reviewpositive_review

lgtm too

comment:47 Changed 23 months ago by mkoeppe

Thanks!

comment:48 Changed 23 months ago by mkoeppe

Priority: criticalblocker

Setting priority to blocker to bring this ticket to the attention of the release bot.

comment:49 Changed 23 months ago by roed

After merging this ticket into #31548 (since I want to use new functionality from FLINT 2.7.1 there) I'm getting errors starting Sage. I don't usually update spkgs outside of beta releases, so I could be doing something wrong (I'm running OS X 10.15.2).

$ git trac fetch 31069 && git merge
$ make build
...
Sage build/upgrade complete!
$ sage
ImportError Python 3.9.1: /Users/roed/sage/sage-9.3.beta7/local/bin/python3
...
ImportError: dlopen(/Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so, 2): Library not loaded: /Users/roed/sage/sage-9.3.beta7/local/lib/libSingular-4.1.1.dylib
  Referenced from: /Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so
  Reason: image not found

Full install.log available here.

comment:50 Changed 23 months ago by fbissey

I believe you need singular 4.2.0p1 (9.3.beta8 if I am not mistaken) with that. Singular uses flint so does need rebuilding and 4.2.0 is the first one that is compatible with flint-2.7.1.

comment:51 Changed 23 months ago by roed

Despite the folder name, this is based on sage-9.3.beta9, and when I did make build it appeared to be building singular-4.2.0p1+2021-03-13+sage.

comment:52 Changed 23 months ago by roed

I feel like I'm missing something basic here: when I try running sage -f flint it gives me lots of errors saying that flint-2.7.1 isn't found:

[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors
[flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz
[flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
[flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz'
[flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz
...
[flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network
[flint-2.7.1] ************************************************************************
[flint-2.7.1] ************************************************************************
[flint-2.7.1] Error downloading flint-2.7.1.tar.gz

Do I need to download a tarball manually from somewhere?

comment:53 in reply to:  51 Changed 23 months ago by fbissey

Replying to roed:

Despite the folder name, this is based on sage-9.3.beta9, and when I did make build it appeared to be building singular-4.2.0p1+2021-03-13+sage.

OK, then it appears that somehow sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so has not be rebuilt because it is still looking for libSingular-4.1.1 when it should look for libSingular-4.2.0.

comment:54 in reply to:  52 Changed 23 months ago by fbissey

Replying to roed:

I feel like I'm missing something basic here: when I try running sage -f flint it gives me lots of errors saying that flint-2.7.1 isn't found:

[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors
[flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz
[flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
[flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz'
[flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz
...
[flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network
[flint-2.7.1] ************************************************************************
[flint-2.7.1] ************************************************************************
[flint-2.7.1] Error downloading flint-2.7.1.tar.gz

Do I need to download a tarball manually from somewhere?

This ticket is not merged, tarball are not usually mirrored before Volker attempt to merge the tickets. You'll have to fetch the source a different way. Matthias created a handy way of using a string called upstream_url in the file checksum.ini to do the fetching from upstream but I cannot remember how it works.

comment:55 Changed 23 months ago by slelievre

Do this:

$ ./bootstrap -q
$ ./configure --enable-download-from-upstream-url -q
$ make -s V=0

comment:56 Changed 23 months ago by roed

Thanks! That seems to have worked.

comment:57 Changed 23 months ago by roed

For anyone else looking at this, I also had to do sage -f flint after the commands Samuel gave.

comment:58 Changed 23 months ago by slelievre

This should do everything at once:

$ ./bootstrap -q
$ ./configure --enable-download-from-upstream-url -q
$ make -s V=0 flint-clean
$ make -s V=0

comment:59 Changed 22 months ago by mkoeppe

Milestone: sage-9.3sage-9.4
Priority: blockercritical

Looks unrealistic to squeeze this into 9.3

comment:60 Changed 21 months ago by vbraun

Branch: public/310698c71a1c694075d021008794ef3a6fd461ec4dc0e
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.