#31069 closed enhancement (fixed)

Upgrade: Flint 2.7.1

Reported by: slelievre Owned by:
Priority: critical Milestone: sage-9.4
Component: packages: standard Keywords: upgrade, flint
Cc: arojas, dimpase, fbissey, gh-timokau, isuruf, mjo, mkoeppe, slelievre, saraedum, vdelecroix, gh-kliem Merged in:
Authors: Vincent Delecroix, Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 8c71a1c (Commits, GitHub, GitLab) Commit: 8c71a1c694075d021008794ef3a6fd461ec4dc0e
Dependencies: #25993 Stopgaps:

Status badges

Description (last modified by vdelecroix)

This is to upgrade to Flint 2.7.1.

Our last upgrade was to Flint 2.6.3 in #29719.

Changes in FLINT 2.7.0 (from NEWS file)

2.7.0 -- 11 Dec 2020

  • Multivariate factorisation
  • Square root and square testing for finite fields
  • Square root and square testing for multivariates
  • Zassenhaus factoring speedups (incl. degree pruning)
  • Fast factorisation of cubic univariate polynomials
  • Add context objects to fmpz_mod_poly functions
  • Use BLAS for matrix multiplication over Z/nZ (small n)
  • Linear solving for non-square/singular matrices (can_solve)
  • Speed up factorisation over Z/nZ (for multiprecision n)

From the packaging point of view the main changes will be

  • configuring BLAS
  • pthread support
  • context object in fmpz_mod_poly function (API change)

Related:

Change History (60)

comment:1 Changed 19 months ago by slelievre

  • Cc arojas dimpase fbissey gh-timokau isuruf mjo mkoeppe slelievre added

comment:2 Changed 19 months ago by fbissey

  • Description modified (diff)

comment:3 Changed 19 months ago by fbissey

  • Description modified (diff)

comment:4 Changed 19 months ago by slelievre

Flint 2.7.0 was released today (2020-12-18).

comment:5 Changed 19 months ago by arojas

  • Cc saraedum vdelecroix added

singular and e-antic fail to build due to the API changes in fmpz_mod_poly

comment:6 Changed 19 months ago by arojas

Actually, Singular is fixed in the (just released) 4.2.0 version. Unfortunately doesn't help much since the Singular upgrade has been stuck for years in #25993

comment:7 Changed 19 months ago by vdelecroix

  • Description modified (diff)

comment:8 Changed 19 months ago by arojas

sagelib itself builds fine with flint 2.7 and passes doctests, modulo the singular upgrade.

comment:9 Changed 18 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

comment:10 Changed 17 months ago by kedlaya

For the record, we are now up to Flint 2.7.1. As the timing and numbering suggest, this is mainly a bugfix release.

comment:11 Changed 17 months ago by gh-thierry-FreeBSD

e-antic has been fixed (in head, to be 1.0.0).

comment:12 Changed 16 months ago by slelievre

  • Description modified (diff)
  • Summary changed from Upgrade: Flint 2.7.0 to Upgrade: Flint 2.7.1

comment:13 Changed 16 months ago by vdelecroix

  • Description modified (diff)

comment:14 Changed 16 months ago by vdelecroix

  • Description modified (diff)

comment:15 Changed 16 months ago by vdelecroix

  • Authors set to Vincent Delecroix
  • Branch set to public/31069
  • Commit set to 29c0477d4aeb9394df2c80cc1628f244647ce40d

New commits:

73bd1a731069: flint version upgrade
29c047731069: flint declarations

comment:16 follow-up: Changed 16 months ago by vdelecroix

I did not have any trouble to build singular. What was the issue?

comment:17 in reply to: ↑ 16 ; follow-up: Changed 16 months ago by arojas

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

comment:18 in reply to: ↑ 17 ; follow-up: Changed 16 months ago by vdelecroix

Replying to arojas:

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

4.1.1p2.p0 shipped with sage 9.3.beta7

comment:19 in reply to: ↑ 18 Changed 16 months ago by arojas

Replying to vdelecroix:

Replying to arojas:

Replying to vdelecroix:

I did not have any trouble to build singular. What was the issue?

Which version of singular?

4.1.1p2.p0 shipped with sage 9.3.beta7

It builds because it's not using flint

checking for FLINT >= 2.4... problem
Sorry, your FLINT version is too old. Disabling.
configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)

comment:20 Changed 16 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.3
  • Priority changed from major to critical

comment:21 Changed 16 months ago by vdelecroix

  • Dependencies set to #25993

comment:22 follow-ups: Changed 16 months ago by mkoeppe

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

comment:23 in reply to: ↑ 22 Changed 16 months ago by vdelecroix

Replying to mkoeppe:

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

That would be an alternative. But because of the API change I believe it is not enough. Let me give it a try.

comment:24 in reply to: ↑ 22 Changed 16 months ago by arojas

Replying to mkoeppe:

Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?

Then it will fail to build with real compilation errors, unless you also backport the flint 2.7 support patches for singular

comment:25 Changed 16 months ago by mkoeppe

perhaps just Singular commit 2e7e3dc89e04ca1ef4175816cd23db6780ad5a38?

comment:26 follow-up: Changed 16 months ago by mkoeppe

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

comment:27 in reply to: ↑ 26 ; follow-up: Changed 16 months ago by arojas

Replying to mkoeppe:

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch

Needs massive ifdef'ing if you still want to support flint <2.7

Last edited 16 months ago by arojas (previous) (diff)

comment:28 in reply to: ↑ 27 Changed 16 months ago by mkoeppe

Replying to arojas:

Replying to mkoeppe:

Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?

Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch

Needs massive ifdef'ing if you still want to support flint <2.7

Thanks! I've prepared this in https://github.com/mkoeppe/e-antic/tree/flint-2.7, testing currently at https://github.com/mkoeppe/e-antic/actions/runs/593267164

Last edited 16 months ago by mkoeppe (previous) (diff)

comment:29 Changed 16 months ago by mkoeppe

Opened #31430 for e-antic patch (or ideally new 0.1.x release)

comment:30 Changed 16 months ago by mkoeppe

  • Dependencies changed from #25993 to #25993, #31430

comment:31 Changed 16 months ago by mkoeppe

  • Dependencies changed from #25993, #31430 to #25993

comment:32 Changed 16 months ago by git

  • Commit changed from 29c0477d4aeb9394df2c80cc1628f244647ce40d to dbc64778ddf69fe39218fc406656e559aaddd096

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

871fdbd31069: flint version upgrade
dbc647731069: flint declarations

comment:33 Changed 16 months ago by git

  • Commit changed from dbc64778ddf69fe39218fc406656e559aaddd096 to 8e3040c2a22fbd256f088ca58619442ae13af362

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

797bacdsrc/sage/rings/polynomial/multi_polynomial_ideal.py: Use sorted() to make doctests more stable
31fe1ebbuild/pkgs/singular/spkg-install.in: Build the documentation if the Singular tarball does not contain it
6c01591Merge #31439
f3119b1Merge tag '9.3.beta9' into t/25993/public/singular4-1-3
49de2ffbuild/pkgs/singular: Use 4.2.0p1+2021-03-13+sage
f816f07build/pkgs/singular/spkg-install.in: Use --disable-static on all platforms
b0d7748build/pkgs/singular/spkg-install.in: Build subdirs in parallel
c33ca2bsrc/sage/rings/polynomial/{laurent,multi}_polynomial_ideal: Use sorted(..., key=str) to make doctests stable
ec471e0build/pkgs/singular/SPKG.rst: Add link to branch
8e3040cMerge #25993

comment:34 Changed 16 months ago by mkoeppe

  • Authors changed from Vincent Delecroix to Vincent Delecroix, Matthias Koeppe
  • Status changed from new to needs_review

comment:35 Changed 16 months ago by mkoeppe

  • Reviewers set to https://github.com/mkoeppe/sage/actions/runs/666698487

comment:36 Changed 15 months ago by mkoeppe

  • Cc gh-kliem added
  • Status changed from needs_review to needs_work

There are still problems with e_antic (0.1.9):

ubuntu-trusty-maximal (https://github.com/mkoeppe/sage/runs/2145415921 - uses FLINT from SPKG):

  [e_antic-0.1.9]   checking for gmp.h... yes
  [e_antic-0.1.9]   checking for __gmpz_add in -lgmp... yes
  [e_antic-0.1.9]   checking for flint/flint.h... yes
  [e_antic-0.1.9]   checking for flint/fmpz.h... no
  [e_antic-0.1.9]   configure: error: flint header not found

cygwin-standard (https://github.com/mkoeppe/sage/runs/2145195263 - uses system FLINT 2.7)

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -fopenmp -O2 -g -march=native -c poly_extra/fmpz_nextprime.c -o poly_extra/fmpz_nextprime.o >/dev/null 2>&1
poly_extra/fmpz_randprime.c:17:6: error: conflicting types for 'fmpz_randprime'
   17 | void fmpz_randprime(fmpz_t f, flint_rand_t state, mp_bitcnt_t bits, int proved)
      |      ^~~~~~~~~~~~~~
In file included from /usr/include/flint/fmpz_poly.h:33,
                 from ./e-antic/poly_extra.h:17,
                 from poly_extra/fmpz_randprime.c:13:
/usr/include/flint/fmpz.h:188:16: note: previous declaration of 'fmpz_randprime' was here
  188 | FLINT_DLL void fmpz_randprime(fmpz_t f, flint_rand_t state,
      |                ^~~~~~~~~~~~~~
make[4]: *** [Makefile:2699: poly_extra/fmpz_randprime.lo] Error 1
Last edited 15 months ago by mkoeppe (previous) (diff)

comment:37 Changed 15 months ago by mkoeppe

OTOH, on ubuntu-xenial-maximal (https://github.com/mkoeppe/sage/runs/2145415962)and ubuntu-bionic-maximal (https://github.com/mkoeppe/sage/runs/2145416018), e_antic installs OK. We can just write off ubuntu-trusty - Normaliz does not compile with the ancient compiler there either.

Last edited 15 months ago by mkoeppe (previous) (diff)

comment:38 Changed 15 months ago by mkoeppe

  • Status changed from needs_work to needs_review

comment:39 Changed 15 months ago by mkoeppe

  • Status changed from needs_review to needs_work

A serious issue: On ubuntu-focal-standard (https://github.com/mkoeppe/sage/runs/2145416039)

  [sagelib-9.3.beta9]   In file included from build/cythonized/sage/data_structures/bounded_integer_sequences.c:3535:
  [sagelib-9.3.beta9]   sage/libs/flint/flint_wrap.h: At top level:
  [sagelib-9.3.beta9]   sage/libs/flint/flint_wrap.h:39:10: fatal error: flint/fmpz_mod.h: No such file or directory
  [sagelib-9.3.beta9]      39 | #include <flint/fmpz_mod.h>
  [sagelib-9.3.beta9]         |          ^~~~~~~~~~~~~~~~~~
  [sagelib-9.3.beta9]   compilation terminated.

This is using system flint 2.5.2 (see https://repology.org/project/flint/versions)

comment:40 Changed 15 months ago by mkoeppe

(fmpz_mod.h was introduced in 2.6.0 according to https://github.com/wbhart/flint2/blob/trunk/NEWS)

Last edited 15 months ago by mkoeppe (previous) (diff)

comment:41 follow-up: Changed 15 months ago by mkoeppe

I think dropping support for system FLINT 2.5.x is too disruptive for Sage 9.3, given that the new features are not used yet by anything.

So I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.

comment:42 Changed 15 months ago by git

  • Commit changed from 8e3040c2a22fbd256f088ca58619442ae13af362 to 8c71a1c694075d021008794ef3a6fd461ec4dc0e

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8c71a1c31069: flint version upgrade

comment:43 in reply to: ↑ 41 Changed 15 months ago by mkoeppe

Replying to mkoeppe:

I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.

This is now #31525.

comment:44 Changed 15 months ago by mkoeppe

  • Status changed from needs_work to needs_review

comment:45 Changed 15 months ago by mjo

I think this is OK. The main issues were in other packages that have already been fixed. With system flint-2.7.1, I see only one doctest failure due to a timeout in pari that probably cannot be blamed on flint.

I see "unrelated" failures in the CI runs as well. If you are convinced of flint's innocence, we can set this to positive review.

comment:46 Changed 15 months ago by dimpase

  • Reviewers changed from https://github.com/mkoeppe/sage/actions/runs/666698487 to Dima Pasechnik
  • Status changed from needs_review to positive_review

lgtm too

comment:47 Changed 15 months ago by mkoeppe

Thanks!

comment:48 Changed 15 months ago by mkoeppe

  • Priority changed from critical to blocker

Setting priority to blocker to bring this ticket to the attention of the release bot.

comment:49 Changed 15 months ago by roed

After merging this ticket into #31548 (since I want to use new functionality from FLINT 2.7.1 there) I'm getting errors starting Sage. I don't usually update spkgs outside of beta releases, so I could be doing something wrong (I'm running OS X 10.15.2).

$ git trac fetch 31069 && git merge
$ make build
...
Sage build/upgrade complete!
$ sage
ImportError Python 3.9.1: /Users/roed/sage/sage-9.3.beta7/local/bin/python3
...
ImportError: dlopen(/Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so, 2): Library not loaded: /Users/roed/sage/sage-9.3.beta7/local/lib/libSingular-4.1.1.dylib
  Referenced from: /Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so
  Reason: image not found

Full install.log available here.

comment:50 Changed 15 months ago by fbissey

I believe you need singular 4.2.0p1 (9.3.beta8 if I am not mistaken) with that. Singular uses flint so does need rebuilding and 4.2.0 is the first one that is compatible with flint-2.7.1.

comment:51 follow-up: Changed 15 months ago by roed

Despite the folder name, this is based on sage-9.3.beta9, and when I did make build it appeared to be building singular-4.2.0p1+2021-03-13+sage.

comment:52 follow-up: Changed 15 months ago by roed

I feel like I'm missing something basic here: when I try running sage -f flint it gives me lots of errors saying that flint-2.7.1 isn't found:

[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors
[flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz
[flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
[flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz'
[flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz
...
[flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network
[flint-2.7.1] ************************************************************************
[flint-2.7.1] ************************************************************************
[flint-2.7.1] Error downloading flint-2.7.1.tar.gz

Do I need to download a tarball manually from somewhere?

comment:53 in reply to: ↑ 51 Changed 15 months ago by fbissey

Replying to roed:

Despite the folder name, this is based on sage-9.3.beta9, and when I did make build it appeared to be building singular-4.2.0p1+2021-03-13+sage.

OK, then it appears that somehow sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so has not be rebuilt because it is still looking for libSingular-4.1.1 when it should look for libSingular-4.2.0.

comment:54 in reply to: ↑ 52 Changed 15 months ago by fbissey

Replying to roed:

I feel like I'm missing something basic here: when I try running sage -f flint it gives me lots of errors saying that flint-2.7.1 isn't found:

[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors
[flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz
[flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]
[flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz'
[flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz
...
[flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network
[flint-2.7.1] ************************************************************************
[flint-2.7.1] ************************************************************************
[flint-2.7.1] Error downloading flint-2.7.1.tar.gz

Do I need to download a tarball manually from somewhere?

This ticket is not merged, tarball are not usually mirrored before Volker attempt to merge the tickets. You'll have to fetch the source a different way. Matthias created a handy way of using a string called upstream_url in the file checksum.ini to do the fetching from upstream but I cannot remember how it works.

comment:55 Changed 15 months ago by slelievre

Do this:

$ ./bootstrap -q
$ ./configure --enable-download-from-upstream-url -q
$ make -s V=0

comment:56 Changed 15 months ago by roed

Thanks! That seems to have worked.

comment:57 Changed 15 months ago by roed

For anyone else looking at this, I also had to do sage -f flint after the commands Samuel gave.

comment:58 Changed 15 months ago by slelievre

This should do everything at once:

$ ./bootstrap -q
$ ./configure --enable-download-from-upstream-url -q
$ make -s V=0 flint-clean
$ make -s V=0

comment:59 Changed 15 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4
  • Priority changed from blocker to critical

Looks unrealistic to squeeze this into 9.3

comment:60 Changed 13 months ago by vbraun

  • Branch changed from public/31069 to 8c71a1c694075d021008794ef3a6fd461ec4dc0e
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.