Opened 2 years ago
Closed 21 months ago
#31069 closed enhancement (fixed)
Upgrade: Flint 2.7.1
Reported by: | slelievre | Owned by: | |
---|---|---|---|
Priority: | critical | Milestone: | sage-9.4 |
Component: | packages: standard | Keywords: | upgrade, flint |
Cc: | arojas, dimpase, fbissey, gh-timokau, isuruf, mjo, mkoeppe, slelievre, saraedum, vdelecroix, gh-kliem | Merged in: | |
Authors: | Vincent Delecroix, Matthias Koeppe | Reviewers: | Dima Pasechnik |
Report Upstream: | N/A | Work issues: | |
Branch: | 8c71a1c (Commits, GitHub, GitLab) | Commit: | 8c71a1c694075d021008794ef3a6fd461ec4dc0e |
Dependencies: | #25993 | Stopgaps: |
Description (last modified by )
This is to upgrade to Flint 2.7.1.
Our last upgrade was to Flint 2.6.3 in #29719.
Changes in FLINT 2.7.0 (from NEWS file)
2.7.0 -- 11 Dec 2020
- Multivariate factorisation
- Square root and square testing for finite fields
- Square root and square testing for multivariates
- Zassenhaus factoring speedups (incl. degree pruning)
- Fast factorisation of cubic univariate polynomials
- Add context objects to fmpz_mod_poly functions
- Use BLAS for matrix multiplication over Z/nZ (small n)
- Linear solving for non-square/singular matrices (can_solve)
- Speed up factorisation over Z/nZ (for multiprecision n)
From the packaging point of view the main changes will be
- configuring BLAS
- pthread support
- context object in
fmpz_mod_poly
function (API change)
Related:
- upstream e-antic issue https://github.com/flatsurf/e-antic/issues/154
- #31408: Meta-ticket: Use new features from FLINT 2.x
Change History (60)
comment:1 Changed 2 years ago by
Cc: | arojas dimpase fbissey gh-timokau isuruf mjo mkoeppe slelievre added |
---|
comment:2 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:3 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:5 Changed 2 years ago by
Cc: | saraedum vdelecroix added |
---|
singular and e-antic fail to build due to the API changes in fmpz_mod_poly
comment:6 Changed 2 years ago by
Actually, Singular is fixed in the (just released) 4.2.0 version. Unfortunately doesn't help much since the Singular upgrade has been stuck for years in #25993
comment:7 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:8 Changed 2 years ago by
sagelib itself builds fine with flint 2.7 and passes doctests, modulo the singular upgrade.
comment:9 Changed 2 years ago by
Milestone: | sage-9.3 → sage-9.4 |
---|
comment:10 Changed 2 years ago by
For the record, we are now up to Flint 2.7.1. As the timing and numbering suggest, this is mainly a bugfix release.
comment:12 Changed 2 years ago by
Description: | modified (diff) |
---|---|
Summary: | Upgrade: Flint 2.7.0 → Upgrade: Flint 2.7.1 |
comment:13 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:14 Changed 2 years ago by
Description: | modified (diff) |
---|
comment:15 Changed 2 years ago by
Authors: | → Vincent Delecroix |
---|---|
Branch: | → public/31069 |
Commit: | → 29c0477d4aeb9394df2c80cc1628f244647ce40d |
comment:16 follow-up: 17 Changed 2 years ago by
I did not have any trouble to build singular. What was the issue?
comment:17 follow-up: 18 Changed 2 years ago by
Replying to vdelecroix:
I did not have any trouble to build singular. What was the issue?
Which version of singular?
comment:18 follow-up: 19 Changed 2 years ago by
Replying to arojas:
Replying to vdelecroix:
I did not have any trouble to build singular. What was the issue?
Which version of singular?
4.1.1p2.p0 shipped with sage 9.3.beta7
comment:19 Changed 2 years ago by
Replying to vdelecroix:
Replying to arojas:
Replying to vdelecroix:
I did not have any trouble to build singular. What was the issue?
Which version of singular?
4.1.1p2.p0 shipped with sage 9.3.beta7
It builds because it's not using flint
checking for FLINT >= 2.4... problem Sorry, your FLINT version is too old. Disabling. configure: WARNING: Unable to find FLINT (which is strongly recommended) on your machine: please use --with-flint=PATH_TO_DIR_CONTAINING_LIB_AND_INCLUDE (see also ./configure --help if you do not understand what we are talking about)
comment:20 Changed 2 years ago by
Milestone: | sage-9.4 → sage-9.3 |
---|---|
Priority: | major → critical |
comment:21 Changed 2 years ago by
Dependencies: | → #25993 |
---|
comment:22 follow-ups: 23 24 Changed 2 years ago by
Can't we patch up the failing configure check in Singular to avoid the dependency on #25993?
comment:23 Changed 2 years ago by
comment:24 Changed 2 years ago by
comment:25 Changed 2 years ago by
perhaps just Singular commit 2e7e3dc89e04ca1ef4175816cd23db6780ad5a38?
comment:26 follow-up: 27 Changed 2 years ago by
Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?
comment:27 follow-up: 28 Changed 2 years ago by
Replying to mkoeppe:
Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?
Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch
Needs massive ifdef'ing if you still want to support flint <2.7
comment:28 Changed 2 years ago by
Replying to arojas:
Replying to mkoeppe:
Does anyone already have e-antic 0.1.x patches to work with FLINT 2.7?
Yes, https://github.com/archlinux/svntogit-community/blob/packages/e-antic/trunk/e-antic-flint-2.7.patch
Needs massive ifdef'ing if you still want to support flint <2.7
Thanks! I've prepared this in https://github.com/mkoeppe/e-antic/tree/flint-2.7, testing currently at https://github.com/mkoeppe/e-antic/actions/runs/593227357
comment:30 Changed 2 years ago by
Dependencies: | #25993 → #25993, #31430 |
---|
comment:31 Changed 23 months ago by
Dependencies: | #25993, #31430 → #25993 |
---|
comment:32 Changed 23 months ago by
Commit: | 29c0477d4aeb9394df2c80cc1628f244647ce40d → dbc64778ddf69fe39218fc406656e559aaddd096 |
---|
comment:33 Changed 23 months ago by
Commit: | dbc64778ddf69fe39218fc406656e559aaddd096 → 8e3040c2a22fbd256f088ca58619442ae13af362 |
---|
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
797bacd | src/sage/rings/polynomial/multi_polynomial_ideal.py: Use sorted() to make doctests more stable
|
31fe1eb | build/pkgs/singular/spkg-install.in: Build the documentation if the Singular tarball does not contain it
|
6c01591 | Merge #31439
|
f3119b1 | Merge tag '9.3.beta9' into t/25993/public/singular4-1-3
|
49de2ff | build/pkgs/singular: Use 4.2.0p1+2021-03-13+sage
|
f816f07 | build/pkgs/singular/spkg-install.in: Use --disable-static on all platforms
|
b0d7748 | build/pkgs/singular/spkg-install.in: Build subdirs in parallel
|
c33ca2b | src/sage/rings/polynomial/{laurent,multi}_polynomial_ideal: Use sorted(..., key=str) to make doctests stable
|
ec471e0 | build/pkgs/singular/SPKG.rst: Add link to branch
|
8e3040c | Merge #25993
|
comment:34 Changed 23 months ago by
Authors: | Vincent Delecroix → Vincent Delecroix, Matthias Koeppe |
---|---|
Status: | new → needs_review |
comment:35 Changed 23 months ago by
Reviewers: | → https://github.com/mkoeppe/sage/actions/runs/666698487 |
---|
comment:36 Changed 23 months ago by
Cc: | gh-kliem added |
---|---|
Status: | needs_review → needs_work |
There are still problems with e_antic
(0.1.9):
ubuntu-trusty-maximal
(https://github.com/mkoeppe/sage/runs/2145415921 - uses FLINT from SPKG):
[e_antic-0.1.9] checking for gmp.h... yes [e_antic-0.1.9] checking for __gmpz_add in -lgmp... yes [e_antic-0.1.9] checking for flint/flint.h... yes [e_antic-0.1.9] checking for flint/fmpz.h... no [e_antic-0.1.9] configure: error: flint header not found
cygwin-standard
(https://github.com/mkoeppe/sage/runs/2145195263 - uses system FLINT 2.7)
libtool: compile: gcc -DHAVE_CONFIG_H -I. -fopenmp -O2 -g -march=native -c poly_extra/fmpz_nextprime.c -o poly_extra/fmpz_nextprime.o >/dev/null 2>&1 poly_extra/fmpz_randprime.c:17:6: error: conflicting types for 'fmpz_randprime' 17 | void fmpz_randprime(fmpz_t f, flint_rand_t state, mp_bitcnt_t bits, int proved) | ^~~~~~~~~~~~~~ In file included from /usr/include/flint/fmpz_poly.h:33, from ./e-antic/poly_extra.h:17, from poly_extra/fmpz_randprime.c:13: /usr/include/flint/fmpz.h:188:16: note: previous declaration of 'fmpz_randprime' was here 188 | FLINT_DLL void fmpz_randprime(fmpz_t f, flint_rand_t state, | ^~~~~~~~~~~~~~ make[4]: *** [Makefile:2699: poly_extra/fmpz_randprime.lo] Error 1
comment:37 Changed 23 months ago by
OTOH, on ubuntu-xenial-maximal
(https://github.com/mkoeppe/sage/runs/2145415962)and ubuntu-bionic-maximal
(https://github.com/mkoeppe/sage/runs/2145416018), e_antic
installs OK.
We can just write off ubuntu-trusty
- Normaliz does not compile with the ancient compiler there either.
comment:38 Changed 23 months ago by
Status: | needs_work → needs_review |
---|
comment:39 Changed 23 months ago by
Status: | needs_review → needs_work |
---|
A serious issue: On ubuntu-focal-standard
(https://github.com/mkoeppe/sage/runs/2145416039)
[sagelib-9.3.beta9] In file included from build/cythonized/sage/data_structures/bounded_integer_sequences.c:3535: [sagelib-9.3.beta9] sage/libs/flint/flint_wrap.h: At top level: [sagelib-9.3.beta9] sage/libs/flint/flint_wrap.h:39:10: fatal error: flint/fmpz_mod.h: No such file or directory [sagelib-9.3.beta9] 39 | #include <flint/fmpz_mod.h> [sagelib-9.3.beta9] | ^~~~~~~~~~~~~~~~~~ [sagelib-9.3.beta9] compilation terminated.
This is using system flint
2.5.2 (see https://repology.org/project/flint/versions)
comment:40 Changed 23 months ago by
(fmpz_mod.h
was introduced in 2.6.0 according to https://github.com/wbhart/flint2/blob/trunk/NEWS)
comment:41 follow-up: 43 Changed 23 months ago by
I think dropping support for system FLINT 2.5.x is too disruptive for Sage 9.3, given that the new features are not used yet by anything.
So I propose to split the ticket into just the upgrade and put the Cython interface to the new FLINT functions on a follow-up ticket for Sage 9.4.
comment:42 Changed 23 months ago by
Commit: | 8e3040c2a22fbd256f088ca58619442ae13af362 → 8c71a1c694075d021008794ef3a6fd461ec4dc0e |
---|
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
8c71a1c | 31069: flint version upgrade
|
comment:43 Changed 23 months ago by
comment:44 Changed 23 months ago by
Status: | needs_work → needs_review |
---|
comment:45 Changed 23 months ago by
I think this is OK. The main issues were in other packages that have already been fixed. With system flint-2.7.1, I see only one doctest failure due to a timeout in pari that probably cannot be blamed on flint.
I see "unrelated" failures in the CI runs as well. If you are convinced of flint's innocence, we can set this to positive review.
comment:46 Changed 23 months ago by
Reviewers: | https://github.com/mkoeppe/sage/actions/runs/666698487 → Dima Pasechnik |
---|---|
Status: | needs_review → positive_review |
lgtm too
comment:48 Changed 23 months ago by
Priority: | critical → blocker |
---|
Setting priority to blocker to bring this ticket to the attention of the release bot.
comment:49 Changed 23 months ago by
After merging this ticket into #31548 (since I want to use new functionality from FLINT 2.7.1 there) I'm getting errors starting Sage. I don't usually update spkgs outside of beta releases, so I could be doing something wrong (I'm running OS X 10.15.2).
$ git trac fetch 31069 && git merge $ make build ... Sage build/upgrade complete! $ sage ImportError Python 3.9.1: /Users/roed/sage/sage-9.3.beta7/local/bin/python3 ... ImportError: dlopen(/Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so, 2): Library not loaded: /Users/roed/sage/sage-9.3.beta7/local/lib/libSingular-4.1.1.dylib Referenced from: /Users/roed/sage/sage-9.3.beta7/local/lib/python3.9/site-packages/sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so Reason: image not found
Full install.log available here.
comment:50 Changed 23 months ago by
I believe you need singular 4.2.0p1 (9.3.beta8 if I am not mistaken) with that. Singular uses flint so does need rebuilding and 4.2.0 is the first one that is compatible with flint-2.7.1.
comment:51 follow-up: 53 Changed 23 months ago by
Despite the folder name, this is based on sage-9.3.beta9, and when I did make build
it appeared to be building singular-4.2.0p1+2021-03-13+sage
.
comment:52 follow-up: 54 Changed 23 months ago by
I feel like I'm missing something basic here: when I try running sage -f flint
it gives me lots of errors saying that flint-2.7.1
isn't found:
[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors [flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz [flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx] [flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz' [flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz ... [flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network [flint-2.7.1] ************************************************************************ [flint-2.7.1] ************************************************************************ [flint-2.7.1] Error downloading flint-2.7.1.tar.gz
Do I need to download a tarball manually from somewhere?
comment:53 Changed 23 months ago by
Replying to roed:
Despite the folder name, this is based on sage-9.3.beta9, and when I did
make build
it appeared to be buildingsingular-4.2.0p1+2021-03-13+sage
.
OK, then it appears that somehow sage/matrix/matrix_mpolynomial_dense.cpython-39-darwin.so
has not be rebuilt because it is still looking for libSingular-4.1.1 when it should look for libSingular-4.2.0.
comment:54 Changed 23 months ago by
Replying to roed:
I feel like I'm missing something basic here: when I try running
sage -f flint
it gives me lots of errors saying thatflint-2.7.1
isn't found:[flint-2.7.1] Attempting to download package flint-2.7.1.tar.gz from mirrors [flint-2.7.1] http://mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz [flint-2.7.1] [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx] [flint-2.7.1] ERROR [transfer|run:135]: [Errno 404] Not Found: '//mirrors.mit.edu/sage/spkg/upstream/flint/flint-2.7.1.tar.gz' [flint-2.7.1] http://www.mirrorservice.org/sites/www.sagemath.org/spkg/upstream/flint/flint-2.7.1.tar.gz ... [flint-2.7.1] FileNotMirroredError: tarball does not exist on mirror network [flint-2.7.1] ************************************************************************ [flint-2.7.1] ************************************************************************ [flint-2.7.1] Error downloading flint-2.7.1.tar.gzDo I need to download a tarball manually from somewhere?
This ticket is not merged, tarball are not usually mirrored before Volker attempt to merge the tickets. You'll have to fetch the source a different way. Matthias created a handy way of using a string called upstream_url
in the file checksum.ini to do the fetching from upstream but I cannot remember how it works.
comment:55 Changed 23 months ago by
Do this:
$ ./bootstrap -q $ ./configure --enable-download-from-upstream-url -q $ make -s V=0
comment:57 Changed 23 months ago by
For anyone else looking at this, I also had to do sage -f flint
after the commands Samuel gave.
comment:58 Changed 23 months ago by
This should do everything at once:
$ ./bootstrap -q $ ./configure --enable-download-from-upstream-url -q $ make -s V=0 flint-clean $ make -s V=0
comment:59 Changed 22 months ago by
Milestone: | sage-9.3 → sage-9.4 |
---|---|
Priority: | blocker → critical |
Looks unrealistic to squeeze this into 9.3
comment:60 Changed 21 months ago by
Branch: | public/31069 → 8c71a1c694075d021008794ef3a6fd461ec4dc0e |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
Flint 2.7-rc2 is out. Can someone test with Sage?