Opened 2 years ago
Closed 21 months ago
#31069 closed enhancement (fixed)
Upgrade: Flint 2.7.1
Reported by:  slelievre  Owned by:  

Priority:  critical  Milestone:  sage9.4 
Component:  packages: standard  Keywords:  upgrade, flint 
Cc:  arojas, dimpase, fbissey, ghtimokau, isuruf, mjo, mkoeppe, slelievre, saraedum, vdelecroix, ghkliem  Merged in:  
Authors:  Vincent Delecroix, Matthias Koeppe  Reviewers:  Dima Pasechnik 
Report Upstream:  N/A  Work issues:  
Branch:  8c71a1c (Commits, GitHub, GitLab)  Commit:  8c71a1c694075d021008794ef3a6fd461ec4dc0e 
Dependencies:  #25993  Stopgaps: 
Description (last modified by )
This is to upgrade to Flint 2.7.1.
Our last upgrade was to Flint 2.6.3 in #29719.
Changes in FLINT 2.7.0 (from NEWS file)
2.7.0  11 Dec 2020
 Multivariate factorisation
 Square root and square testing for finite fields
 Square root and square testing for multivariates
 Zassenhaus factoring speedups (incl. degree pruning)
 Fast factorisation of cubic univariate polynomials
 Add context objects to fmpz_mod_poly functions
 Use BLAS for matrix multiplication over Z/nZ (small n)
 Linear solving for nonsquare/singular matrices (can_solve)
 Speed up factorisation over Z/nZ (for multiprecision n)
From the packaging point of view the main changes will be
 configuring BLAS
 pthread support
 context object in
fmpz_mod_poly
function (API change)
Related:
 upstream eantic issue https://github.com/flatsurf/eantic/issues/154
 #31408: Metaticket: Use new features from FLINT 2.x
Change History (60)
comment:1 Changed 2 years ago by
Cc:  arojas dimpase fbissey ghtimokau isuruf mjo mkoeppe slelievre added 

comment:2 Changed 2 years ago by
Description:  modified (diff) 

comment:3 Changed 2 years ago by
Description:  modified (diff) 

comment:5 Changed 2 years ago by
Cc:  saraedum vdelecroix added 

singular and eantic fail to build due to the API changes in fmpz_mod_poly
comment:6 Changed 2 years ago by
Actually, Singular is fixed in the (just released) 4.2.0 version. Unfortunately doesn't help much since the Singular upgrade has been stuck for years in #25993
comment:7 Changed 2 years ago by
Description:  modified (diff) 

comment:8 Changed 2 years ago by
sagelib itself builds fine with flint 2.7 and passes doctests, modulo the singular upgrade.
comment:9 Changed 2 years ago by
Milestone:  sage9.3 → sage9.4 

comment:10 Changed 2 years ago by
For the record, we are now up to Flint 2.7.1. As the timing and numbering suggest, this is mainly a bugfix release.
comment:12 Changed 2 years ago by
Description:  modified (diff) 

Summary:  Upgrade: Flint 2.7.0 → Upgrade: Flint 2.7.1 
comment:13 Changed 2 years ago by
Description:  modified (diff) 

comment:14 Changed 2 years ago by
Description:  modified (diff) 

comment:15 Changed 2 years ago by
Authors:  → Vincent Delecroix 

Branch:  → public/31069 
Commit:  → 29c0477d4aeb9394df2c80cc1628f244647ce40d 
comment:16 followup: 17 Changed 2 years ago by
I did not have any trouble to build singular. What was the issue?
comment:17 followup: 18 Changed 2 years ago by
Replying to vdelecroix:
I did not have any trouble to build singular. What was the issue?
Which version of singular?
comment:18 followup: 19 Changed 2 years ago by
Replying to arojas:
Replying to vdelecroix:
I did not have any trouble to build singular. What was the issue?
Which version of singular?
4.1.1p2.p0 shipped with sage 9.3.beta7
comment:19 Changed 2 years ago by
Replying to vdelecroix:
Replying to arojas:
Replying to vdelecroix:
I did not have any trouble to build singular. What was the issue?
Which version of singular?
4.1.1p2.p0 shipped with sage 9.3.beta7
It builds because it's not using flint
checking for FLINT >= 2.4... problem Sorry, your FLINT version is too old. Disabling. configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use withflint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure help if you do not understand what we are talking about)
comment:20 Changed 2 years ago by
Milestone:  sage9.4 → sage9.3 

Priority:  major → critical 
comment:21 Changed 2 years ago by
Dependencies:  → #25993 

comment:22 followups: 23 24 Changed 2 years ago by
Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?
comment:23 Changed 2 years ago by
comment:24 Changed 2 years ago by
comment:25 Changed 2 years ago by
perhaps just Singular commit 2e7e3dc89e04ca1ef4175816cd23db6780ad5a38?
comment:26 followup: 27 Changed 2 years ago by
Does anyone already have eantic 0.1.x patches to work with FLINT 2.7?
comment:27 followup: 28 Changed 2 years ago by
Replying to mkoeppe:
Does anyone already have eantic 0.1.x patches to work with FLINT 2.7?
Yes, https://github.com/archlinux/svntogitcommunity/blob/packages/eantic/trunk/eanticflint2.7.patch
Needs massive ifdef'ing if you still want to support flint <2.7
comment:28 Changed 2 years ago by
Replying to arojas:
Replying to mkoeppe:
Does anyone already have eantic 0.1.x patches to work with FLINT 2.7?
Yes, https://github.com/archlinux/svntogitcommunity/blob/packages/eantic/trunk/eanticflint2.7.patch
Needs massive ifdef'ing if you still want to support flint <2.7
Thanks! I've prepared this in https://github.com/mkoeppe/eantic/tree/flint2.7, testing currently at https://github.com/mkoeppe/eantic/actions/runs/593227357
comment:30 Changed 2 years ago by
Dependencies:  #25993 → #25993, #31430 

comment:31 Changed 23 months ago by
Dependencies:  #25993, #31430 → #25993 

comment:32 Changed 23 months ago by
Commit:  29c0477d4aeb9394df2c80cc1628f244647ce40d → dbc64778ddf69fe39218fc406656e559aaddd096 

comment:33 Changed 23 months ago by
Commit:  dbc64778ddf69fe39218fc406656e559aaddd096 → 8e3040c2a22fbd256f088ca58619442ae13af362 

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
797bacd  src/sage/rings/polynomial/multi_polynomial_ideal.py: Use sorted() to make doctests more stable

31fe1eb  build/pkgs/singular/spkginstall.in: Build the documentation if the Singular tarball does not contain it

6c01591  Merge #31439

f3119b1  Merge tag '9.3.beta9' into t/25993/public/singular413

49de2ff  build/pkgs/singular: Use 4.2.0p1+20210313+sage

f816f07  build/pkgs/singular/spkginstall.in: Use disablestatic on all platforms

b0d7748  build/pkgs/singular/spkginstall.in: Build subdirs in parallel

c33ca2b  src/sage/rings/polynomial/{laurent,multi}_polynomial_ideal: Use sorted(..., key=str) to make doctests stable

ec471e0  build/pkgs/singular/SPKG.rst: Add link to branch

8e3040c  Merge #25993

comment:34 Changed 23 months ago by
Authors:  Vincent Delecroix → Vincent Delecroix, Matthias Koeppe 

Status:  new → needs_review 
comment:35 Changed 23 months ago by
Reviewers:  → https://github.com/mkoeppe/sage/actions/runs/666698487 

comment:36 Changed 23 months ago by
Cc:  ghkliem added 

Status:  needs_review → needs_work 
There are still problems with e_antic
(0.1.9):
ubuntutrustymaximal
(https://github.com/mkoeppe/sage/runs/2145415921  uses FLINT from SPKG):
[e_antic0.1.9] checking for gmp.h... yes [e_antic0.1.9] checking for __gmpz_add in lgmp... yes [e_antic0.1.9] checking for flint/flint.h... yes [e_antic0.1.9] checking for flint/fmpz.h... no [e_antic0.1.9] configure: error: flint header not found
cygwinstandard
(https://github.com/mkoeppe/sage/runs/2145195263  uses system FLINT 2.7)
libtool: compile: gcc DHAVE_CONFIG_H I. fopenmp O2 g march=native c poly_extra/fmpz_nextprime.c o poly_extra/fmpz_nextprime.o >/dev/null 2>&1 poly_extra/fmpz_randprime.c:17:6: error: conflicting types for 'fmpz_randprime' 17  void fmpz_randprime(fmpz_t f, flint_rand_t state, mp_bitcnt_t bits, int proved)  ^~~~~~~~~~~~~~ In file included from /usr/include/flint/fmpz_poly.h:33, from ./eantic/poly_extra.h:17, from poly_extra/fmpz_randprime.c:13: /usr/include/flint/fmpz.h:188:16: note: previous declaration of 'fmpz_randprime' was here 188  FLINT_DLL void fmpz_randprime(fmpz_t f, flint_rand_t state,  ^~~~~~~~~~~~~~ make[4]: *** [Makefile:2699: poly_extra/fmpz_randprime.lo] Error 1
comment:37 Changed 23 months ago by
OTOH, on ubuntuxenialmaximal
(https://github.com/mkoeppe/sage/runs/2145415962)and ubuntubionicmaximal
(https://github.com/mkoeppe/sage/runs/2145416018), e_antic
installs OK.
We can just write off ubuntutrusty
 Normaliz does not compile with the ancient compiler there either.
comment:38 Changed 23 months ago by
Status:  needs_work → needs_review 

comment:39 Changed 23 months ago by
Status:  needs_review → needs_work 

A serious issue: On ubuntufocalstandard
(https://github.com/mkoeppe/sage/runs/2145416039)
[sagelib9.3.beta9] In file included from build/cythonized/sage/data_structures/bounded_integer_sequences.c:3535: [sagelib9.3.beta9] sage/libs/flint/flint_wrap.h: At top level: [sagelib9.3.beta9] sage/libs/flint/flint_wrap.h:39:10: fatal error: flint/fmpz_mod.h: No such file or directory [sagelib9.3.beta9] 39  #include <flint/fmpz_mod.h> [sagelib9.3.beta9]  ^~~~~~~~~~~~~~~~~~ [sagelib9.3.beta9] compilation terminated.
This is using system flint
2.5.2 (see https://repology.org/project/flint/versions)
comment:40 Changed 23 months ago by
(fmpz_mod.h
was introduced in 2.6.0 according to https://github.com/wbhart/flint2/blob/trunk/NEWS)
comment:41 followup: 43 Changed 23 months ago by
I think dropping support for system FLINT 2.5.x is too disruptive for Sage 9.3, given that the new features are not used yet by anything.
So I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a followup ticket for Sage 9.4.
comment:42 Changed 23 months ago by
Commit:  8e3040c2a22fbd256f088ca58619442ae13af362 → 8c71a1c694075d021008794ef3a6fd461ec4dc0e 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
8c71a1c  31069: flint version upgrade

comment:43 Changed 23 months ago by
comment:44 Changed 23 months ago by
Status:  needs_work → needs_review 

comment:45 Changed 23 months ago by
I think this is OK. The main issues were in other packages that have already been fixed. With system flint2.7.1, I see only one doctest failure due to a timeout in pari that probably cannot be blamed on flint.
I see "unrelated" failures in the CI runs as well. If you are convinced of flint's innocence, we can set this to positive review.
comment:46 Changed 23 months ago by
Reviewers:  https://github.com/mkoeppe/sage/actions/runs/666698487 → Dima Pasechnik 

Status:  needs_review → positive_review 
lgtm too
comment:48 Changed 23 months ago by
Priority:  critical → blocker 

Setting priority to blocker to bring this ticket to the attention of the release bot.
comment:49 Changed 23 months ago by
After merging this ticket into #31548 (since I want to use new functionality from FLINT 2.7.1 there) I'm getting errors starting Sage. I don't usually update spkgs outside of beta releases, so I could be doing something wrong (I'm running OS X 10.15.2).
$ git trac fetch 31069 && git merge $ make build ... Sage build/upgrade complete! $ sage ImportError Python 3.9.1: /Users/roed/sage/sage9.3.beta7/local/bin/python3 ... ImportError: dlopen(/Users/roed/sage/sage9.3.beta7/local/lib/python3.9/sitepackages/sage/matrix/matrix_mpolynomial_dense.cpython39darwin.so, 2): Library not loaded: /Users/roed/sage/sage9.3.beta7/local/lib/libSingular4.1.1.dylib Referenced from: /Users/roed/sage/sage9.3.beta7/local/lib/python3.9/sitepackages/sage/matrix/matrix_mpolynomial_dense.cpython39darwin.so Reason: image not found
Full install.log available here.
comment:50 Changed 23 months ago by
I believe you need singular 4.2.0p1 (9.3.beta8 if I am not mistaken) with that. Singular uses flint so does need rebuilding and 4.2.0 is the first one that is compatible with flint2.7.1.
comment:51 followup: 53 Changed 23 months ago by
Despite the folder name, this is based on sage9.3.beta9, and when I did make build
it appeared to be building singular4.2.0p1+20210313+sage
.
comment:52 followup: 54 Changed 23 months ago by
I feel like I'm missing something basic here: when I try running sage f flint
it gives me lots of errors saying that flint2.7.1
isn't found:
[flint2.7.1] Attempting to download package flint2.7.1.tar.gz from mirrors [flint2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint2.7.1.tar.gz [flint2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx] [flint2.7.1] ERROR [transferrun:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint2.7.1.tar.gz' [flint2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint2.7.1.tar.gz ... [flint2.7.1] FileNotMirroredError: tarball does not exist on mirror network [flint2.7.1] ************************************************************************ [flint2.7.1] ************************************************************************ [flint2.7.1] Error downloading flint2.7.1.tar.gz
Do I need to download a tarball manually from somewhere?
comment:53 Changed 23 months ago by
Replying to roed:
Despite the folder name, this is based on sage9.3.beta9, and when I did
make build
it appeared to be buildingsingular4.2.0p1+20210313+sage
.
OK, then it appears that somehow sage/matrix/matrix_mpolynomial_dense.cpython39darwin.so
has not be rebuilt because it is still looking for libSingular4.1.1 when it should look for libSingular4.2.0.
comment:54 Changed 23 months ago by
Replying to roed:
I feel like I'm missing something basic here: when I try running
sage f flint
it gives me lots of errors saying thatflint2.7.1
isn't found:[flint2.7.1] Attempting to download package flint2.7.1.tar.gz from mirrors [flint2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint2.7.1.tar.gz [flint2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx] [flint2.7.1] ERROR [transferrun:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint2.7.1.tar.gz' [flint2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint2.7.1.tar.gz ... [flint2.7.1] FileNotMirroredError: tarball does not exist on mirror network [flint2.7.1] ************************************************************************ [flint2.7.1] ************************************************************************ [flint2.7.1] Error downloading flint2.7.1.tar.gzDo I need to download a tarball manually from somewhere?
This ticket is not merged, tarball are not usually mirrored before Volker attempt to merge the tickets. You'll have to fetch the source a different way. Matthias created a handy way of using a string called upstream_url
in the file checksum.ini to do the fetching from upstream but I cannot remember how it works.
comment:55 Changed 23 months ago by
Do this:
$ ./bootstrap q $ ./configure enabledownloadfromupstreamurl q $ make s V=0
comment:57 Changed 23 months ago by
For anyone else looking at this, I also had to do sage f flint
after the commands Samuel gave.
comment:58 Changed 23 months ago by
This should do everything at once:
$ ./bootstrap q $ ./configure enabledownloadfromupstreamurl q $ make s V=0 flintclean $ make s V=0
comment:59 Changed 22 months ago by
Milestone:  sage9.3 → sage9.4 

Priority:  blocker → critical 
Looks unrealistic to squeeze this into 9.3
comment:60 Changed 21 months ago by
Branch:  public/31069 → 8c71a1c694075d021008794ef3a6fd461ec4dc0e 

Resolution:  → fixed 
Status:  positive_review → closed 
Flint 2.7rc2 is out. Can someone test with Sage?