Opened 2 years ago

Closed 18 months ago

Last modified 17 months ago

#30801 closed enhancement (fixed)

Upgrade: pari 2.13

Reported by: Samuel Lelièvre Owned by:
Priority: major Milestone: sage-9.4
Component: packages: standard Keywords: upgrade, pari
Cc: Dima Pasechnik, Michael Orlitzky, Matthias Köppe, Samuel Lelièvre, Antonio Rojas, François Bissey, Vincent Delecroix, Tobias Hansen, Mauricio Collares, Isuru Fernando, gh-dkwo, Xavier Caruso, David Loeffler, gh-kliem Merged in:
Authors: Vincent Delecroix, Antonio Rojas, Gonzalo Tornaría Reviewers: Dima Pasechnik, David Loeffler
Report Upstream: Fixed upstream, but not in a stable release. Work issues:
Branch: c78b147 (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by Vincent Delecroix)

This is to upgrade to PARI 2.13.x. https://repology.org/project/pari/versions

This new PARI release brings a lot of bug fixes, new functionality, and speedups.

Changes in the PARI library that needs adaptation in SageMath code

 34- bnfissunit is obsolete, use bnfisunit
 35- bnfsunit is mostly obsolete, use bnfunits
 36- bnfisunit and bnfissunit: torsion unit exponent is now a t_INT (used
     to be a t_INTMOD)
 1- Removed member functions .futu and .tufu [deprecated since 2.2], used in simon two descent scripts.
 56- zetamultall: add flag

Last upgrade:

  • #29313 (Upgrade: pari 2.11.4), merged in Sage 9.2.beta7 (released 2020-08-03)

Follow-up ticket: #31754

Change History (207)

comment:1 Changed 2 years ago by Antonio Rojas

Cc: Antonio Rojas added

comment:2 Changed 2 years ago by Antonio Rojas

This needs first a new cypari release including https://github.com/sagemath/cypari2/commit/1a4be24489c2e457727d76bf6dd71daa5c45aee4, otherwise it crashes immediately. After that, things don't look good:

sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/quadratic_forms/genera/genus.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/libs/pari/tests.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/libs/pari/__init__.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/structure/factorization.py  # 4 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/lfunctions/pari.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/matrix/matrix2.pyx  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/coding/linear_code.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/arith/misc.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/groups/fqf_orthogonal.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/groups/abelian_gps/abelian_group.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/groups/additive_abelian/additive_abelian_group.py  # 10 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modules/torsion_quadratic_module.py  # 3 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modules/fg_pid/fgp_module.py  # 30 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modules/fg_pid/fgp_morphism.py  # 5 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modules/fg_pid/fgp_element.py  # 8 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/tests/parigp.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/tests/books/judson-abstract-algebra/galois-sage.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/tests/books/computational-mathematics-with-sagemath/linalg_doctest.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/integer.pyx  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/complex_number.pyx  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/qqbar.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/polynomial/polynomial_element.pyx  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/polynomial/multi_polynomial_element.py  # 3 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/polynomial/multi_polynomial_ideal.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/polynomial/plural.pyx  # 3 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/polynomial/polynomial_quotient_ring.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/function_field/function_field.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/finite_rings/residue_field.pyx  # 7 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/finite_rings/finite_field_constructor.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/finite_rings/integer_mod_ring.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/maps.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/number_field_rel.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/unit_group.py  # 17 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/S_unit_solver.py  # 8 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/number_field_element.pyx  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/number_field_ideal_rel.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/splitting_field.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/order.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/number_field_ideal.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/number_field.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/rings/number_field/class_group.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/gal_reps.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/ell_generic.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/ell_finite_field.py  # 3 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/height.py  # 6 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/ell_number_field.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/ell_torsion.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/period_lattice.py  # 3 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/gp_simon.py  # 4 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/ell_rational_field.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/kraus.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/ell_curve_isogeny.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/heegner.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/gal_reps_number_field.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/elliptic_curves/isogeny_small_degree.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/toric/homset.py  # 3 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/toric/points.py  # 9 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/toric/chow_group.py  # 4 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/schemes/plane_conics/con_rational_function_field.py  # Timed out
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/geometry/cone.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modular/multiple_zeta.py  # 403 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modular/local_comp/smoothchar.py  # 2 doctests failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modular/local_comp/liftings.py  # 1 doctest failed
sage -t --long --random-seed=0 /usr/lib/python3.8/site-packages/sage/modular/modsym/p1list_nf.py  # 1 doctest failed

comment:3 Changed 2 years ago by Antonio Rojas

The timeouts are hangs when calling nffactor. A minimal test case:

sage: K=NumberField(x^12 + 5, 't')
sage: P.<x>=K[]
sage: f=x^2 - 1
sage: f.factor()

I've bisected it to upstream commit [1], but running the same code directly in gp works fine.

[1] https://pari.math.u-bordeaux.fr/cgi-bin/gitweb.cgi?p=pari.git;a=commitdiff;h=473226c477c674838de87f4aaa67939eaa2fb034

comment:4 Changed 2 years ago by François Bissey

Cc: François Bissey added

comment:5 Changed 2 years ago by Antonio Rojas

So, the actual issue seems to be:

sage: pari("default")
...
  primelimit = 0

If I set primelimit to 0 in gp I can reproduce the hangs. Reported to cypari https://github.com/sagemath/cypari2/issues/96

comment:6 Changed 2 years ago by Thierry Thomas

Another issue is the return of:

$ echo "bnf = bnfinit(y^4-y-1); bnfisunit(bnf,-y^3+2*y^2-1)" | gp -qf

returns:

[0, 2, 0]~

but configure expects [[0, 2, Mod(0, 2)]]~ (from build/pkgs/pari/spkg-configure.m4)

comment:7 Changed 2 years ago by Dima Pasechnik

ok, they changed the output format. This is certainly easy to take care of.

comment:8 Changed 2 years ago by Vincent Delecroix

Cc: Vincent Delecroix added
Description: modified (diff)

comment:9 Changed 2 years ago by Tobias Hansen

Cc: Tobias Hansen added

comment:10 Changed 2 years ago by Vincent Delecroix

Description: modified (diff)

comment:11 Changed 2 years ago by Vincent Delecroix

Authors: Vincent Delecroix
Branch: u/vdelecroix/pari-2.13.0-cypari-2.1.2
Commit: 390d8709cda46fe6e0b77c60c333b1d66d00439c

New commits:

3df29d3upgrade to pari 2.13.0
e16c581upgrade cypari2 to 2.1.2b1
68214e4adapt libs/pari to 2.13.0
ec9fbb5doctests in libs/pari
3af9483fix some doctests and discard .pari_nf() in number_field.py
ace4395doctests in unit_group.py
5358fc2poldegree is now a GEN
390d870fix cygwin patch

comment:12 Changed 2 years ago by Vincent Delecroix

Description: modified (diff)

comment:13 Changed 2 years ago by Antonio Rojas

Many test failures are caused by pari having changed the normal Smith form algorithm, which now gives different transition matrices. Most of them are still correct, but it also has some unfortunate side effects:

sage: V=ZZ^2
sage: W=span([[1,0],[0,6]],ZZ)
sage: M=V/W
sage: a=M[1]; a
(1)
sage: a.lift()
(0, -1)
Last edited 2 years ago by Antonio Rojas (previous) (diff)

comment:14 Changed 2 years ago by Antonio Rojas

Description: modified (diff)

comment:15 Changed 2 years ago by Vincent Delecroix

On Tuesday (November 3) from *14h to 17h* (UTC+2) there is a virtual PARI/GP workshop. I intend to discuss with Bill the cypari2 release and the adaptation in SageMath. Anybody interested is more than welcome. Just send me a private e-mail so that I can forward the announcement.

comment:16 Changed 2 years ago by Antonio Rojas

The update to cypari 2.1.2b causes some additional issues (not present in 2.1.1, both tested with pari 2.13)

File "/usr/lib/python3.8/site-packages/sage/rings/rational.pyx", line 1450, in sage.rings.rational.Rational.is_norm
Failed example:
    7.is_norm(K)
Expected:
    Traceback (most recent call last):
    ...
    NotImplementedError: is_norm is not implemented unconditionally for norms from non-Galois number fields
Got:
    <BLANKLINE>
    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/sage/doctest/forker.py", line 720, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.8/site-packages/sage/doctest/forker.py", line 1145, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.rings.rational.Rational.is_norm[13]>", line 1, in <module>
        Integer(7).is_norm(K)
      File "sage/rings/integer.pyx", line 5443, in sage.rings.integer.Integer.is_norm (build/cythonized/sage/rings/integer.c:34562)
        return QQ(self).is_norm(K, element=element, proof=proof)
      File "sage/rings/rational.pyx", line 1464, in sage.rings.rational.Rational.is_norm (build/cythonized/sage/rings/rational.c:13741)
        return self.is_norm(L, element=True, proof=proof)[0]
      File "sage/rings/rational.pyx", line 1484, in sage.rings.rational.Rational.is_norm (build/cythonized/sage/rings/rational.c:14464)
        M = L.galois_closure('a')
      File "/usr/lib/python3.8/site-packages/sage/rings/number_field/number_field.py", line 8606, in galois_closure
        L, self_into_L = self._galois_closure_and_embedding(names)
      File "/usr/lib/python3.8/site-packages/sage/rings/number_field/number_field.py", line 8541, in _galois_closure_and_embedding
        L, self_into_L = self.defining_polynomial().change_ring(self).splitting_field(names, map=True, degree_multiple=deg)
      File "sage/rings/polynomial/polynomial_element.pyx", line 4635, in sage.rings.polynomial.polynomial_element.Polynomial.splitting_field (build/cythonized/sage/rings/polynomial/polynomial_element.c:39244)
        return splitting_field(f, name, map, **kwds)
      File "/usr/lib/python3.8/site-packages/sage/rings/number_field/splitting_field.py", line 454, in splitting_field
        rel_degree_divisor = rel_degree_divisor.lcm(q.poldegree())
      File "sage/structure/element.pyx", line 3917, in sage.structure.element.PrincipalIdealDomainElement.lcm (build/cythonized/sage/structure/element.c:24871)
        return coercion_model.bin_op(self, right, lcm)
      File "sage/structure/coerce.pyx", line 1248, in sage.structure.coerce.CoercionModel.bin_op (build/cythonized/sage/structure/coerce.c:11304)
        raise bin_op_exception(op, x, y)
    TypeError: unsupported operand parent(s) for lcm: 'Integer Ring' and '<class 'cypari2.gen.Gen'>'

comment:17 Changed 2 years ago by Vincent Delecroix

This is the problematic line

      File "/usr/lib/python3.8/site-packages/sage/rings/number_field/splitting_field.py", line 454, in splitting_field
        rel_degree_divisor = rel_degree_divisor.lcm(q.poldegree())

poldgree used to return an Python integer. But now that it is auto-generated in cypari it returns a Gen which is not accepted by lcm

sage: 4.lcm(pari(5))
Traceback (most recent call last):
...
TypeError: unsupported operand parent(s) for lcm: 'Integer Ring' and '<class 'cypari2.gen.Gen'>'

Some of these poldegree are "corrected" in commit 5358fc2.

I see several solutions

  • manually add a q.poldegree().sage() to all the problematic calls
  • make sage gcd/lcm accepts pari integers
  • undo the change in cypari2 that makes poldgree returns a Gen
  • in cypari2, return a Pyton integer instead of a Pari integer when natural (ie something that can be used as an index: poldegree, length of vectors, ...)
Last edited 2 years ago by Vincent Delecroix (previous) (diff)

comment:18 Changed 2 years ago by Vincent Delecroix

To my mind, fixing Sage gcd/lcm makes more sense.

comment:19 Changed 2 years ago by Vincent Delecroix

Related: the functions gcd/lcm are not commutative

sage: type(lcm(2, pari(3)))
<class 'sage.rings.integer.Integer'>
sage: type(lcm(pari(3), 2))
<class 'cypari2.gen.Gen'>
Last edited 2 years ago by Vincent Delecroix (previous) (diff)

comment:20 Changed 2 years ago by Vincent Delecroix

Dependencies: #30849

See #30849

comment:21 Changed 2 years ago by git

Commit: 390d8709cda46fe6e0b77c60c333b1d66d00439c7679cfa77b6a393bc7898865d43c3fb42a71cc35

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

75c1516make gcd/lcm of Integer works with gmpy2/pari types
8c675b4upgrade to pari 2.13.0
2833d06upgrade cypari2 to 2.1.2b1
886e9d2adapt libs/pari to 2.13.0
d93c350doctests in libs/pari
b7fed64fix some doctests and discard .pari_nf() in number_field.py
b4afd2adoctests in unit_group.py
7679cfafix cygwin patch

comment:22 Changed 2 years ago by Vincent Delecroix

(Rebased on top of #30849 dropping modification to gcd/lcm when poldegree is involved)

comment:23 Changed 2 years ago by git

Commit: 7679cfa77b6a393bc7898865d43c3fb42a71cc35e04727dcfc5bcc30601554480e247aee7fe7531f

Branch pushed to git repo; I updated commit sha1. New commits:

e04727dfix argument order in rnfisnorm call

comment:24 Changed 2 years ago by git

Commit: e04727dcfc5bcc30601554480e247aee7fe7531f92ad1bbc57038f2b4f31b54b4af3aec05f95d21e

Branch pushed to git repo; I updated commit sha1. New commits:

92ad1bbfix call to zetamultall

comment:25 Changed 2 years ago by git

Commit: 92ad1bbc57038f2b4f31b54b4af3aec05f95d21e1ce6c2271115018f2eeeb1d487a2cb131bc46689

Branch pushed to git repo; I updated commit sha1. New commits:

1ce6c22fix invalid t_INT + t_INFINITY operation in multi_polynomial_ring.py

comment:26 Changed 2 years ago by Antonio Rojas

For Simon's scripts, this works for me:

diff --git a/src/sage/ext_data/pari/simon/ell.gp b/src/sage/ext_data/pari/simon/ell.gp
index 74f0786646..21cff9cbb3 100644
--- a/src/sage/ext_data/pari/simon/ell.gp
+++ b/src/sage/ext_data/pari/simon/ell.gp
@@ -1038,7 +1038,7 @@ if( DEBUGLEVEL_ell >= 1, print(" trivial points on E(K) = ");
   KS2gen = KS2gen[1];
   for( i = 1, #KS2gen,
     KS2gen[i] = nfbasistoalg(bnf, KS2gen[i]));
-  KS2gen = concat(Mod(lift(bnf.tufu),bnf.pol),KS2gen);
+  KS2gen = concat(Mod(lift(concat(bnf.tu[2], bnf.fu)),bnf.pol),KS2gen);
 if( DEBUGLEVEL_ell >= 2,
   print("  #K(b,2)gen          = ",#KS2gen);
   print("  K(b,2)gen = ",KS2gen));
@@ -1072,7 +1072,7 @@ if( DEBUGLEVEL_ell >= 1,
   KS2gen = KS2gen[1];
   for( i = 1, #KS2gen,
     KS2gen[i] = nfbasistoalg(bnf, KS2gen[i]));
-  KS2gen = concat(Mod(lift(bnf.tufu),bnf.pol),KS2gen);
+  KS2gen = concat(Mod(lift(concat(bnf.tu[2], bnf.fu)),bnf.pol),KS2gen);
 if( DEBUGLEVEL_ell >= 2,
   print("  #K(a^2-4b,2)gen     = ",#KS2gen);
   print("  K(a^2-4b,2)gen     = ",KS2gen));
@@ -1244,11 +1244,11 @@ if( DEBUGLEVEL_ell >= 4, print("    bbbnf.clgp = ",bbbnf.clgp));
   SL1 = idealmul(bbbnf,SL0,rnfeltup(rrrnf,bleg));
   SL = idealfactor(bbbnf,SL1)[,1]~;
   sunL = bnfsunit(bbbnf,SL);
-  fondsunL = concat(bbbnf.futu,vector(#sunL[1],i,nfbasistoalg(bbbnf,sunL[1][i])));
+  fondsunL = concat(concat(bbbnf.fu, bbbnf.tu[2]),vector(#sunL[1],i,nfbasistoalg(bbbnf,sunL[1][i])));
   normfondsunL = vector(#fondsunL, i, norm(rnfeltabstorel(rrrnf,fondsunL[i])));
   SK = idealfactor(bnf,idealnorm(bbbnf,SL1))[,1]~;
   sunK = bnfsunit(bnf,SK);
-  fondsunK = concat(bnf.futu,vector(#sunK[1],i,nfbasistoalg(bnf,sunK[1][i])));
+  fondsunK = concat(concat(bnf.fu, bnf.tu[2]),vector(#sunK[1],i,nfbasistoalg(bnf,sunK[1][i])));
   vecbleg = bnfissunit(bnf,sunK,bleg);
   matnorm = matrix(#fondsunK,#normfondsunL,i,j,0);
   for( i = 1, #normfondsunL,
@@ -1345,7 +1345,7 @@ if( DEBUGLEVEL_ell >= 4, print("on factorise bb = ",bb));
       sun = bnfsunit(bnf,idealfactor(bnf,bb)[,1]~);
       fact = lift(bnfissunit(bnf,sun,bb));
 if( DEBUGLEVEL_ell >= 4, print("fact = ",fact));
-      suni = concat(bnf.futu,vector(#sun[1],i,nfbasistoalg(bnf,sun[1][i])));
+      suni = concat(concat(bnf.fu, bnf.tu[2]),vector(#sun[1],i,nfbasistoalg(bnf,sun[1][i])));
       for( i = 1, #suni,
         if( (f = fact[i]>>1), 
           test =0;
@@ -1554,7 +1554,7 @@ if( DEBUGLEVEL_ell >= 3, print("    KS2gen = ",KS2gen[1]));
 
   LS2gen = LS2gen[1];
   LS2 = vector(#LS2gen,i,lift(nfbasistoalg(Lrnf,LS2gen[i])));
-  LS2 = concat(lift(Lrnf.futu),LS2);
+  LS2 = concat(lift(concat(Lrnf.fu, Lrnf.tu[2])),LS2);
 
   LS2 = subst(LS2,'x,ttheta);
   LS2 = LS2*Mod(1,polrel);
@@ -1992,7 +1992,7 @@ if( DEBUGLEVEL_ell >= 2, print("  Algorithm of complete 2-descent"));
   KS2gen = KS2gen[1];
   for( i = 1, #KS2gen,
     KS2gen[i] = nfbasistoalg(bnf, KS2gen[i]));
-  KS2gen = concat(Mod(lift(bnf.tufu),bnf.pol),KS2gen);
+  KS2gen = concat(Mod(lift(concat(bnf.tu[2], bnf.fu)),bnf.pol),KS2gen);
 if( DEBUGLEVEL_ell >= 2,
   print("  #K(S,2)gen = ",#KS2gen);
   print("   K(S,2)gen = ",KS2gen)
diff --git a/src/sage/ext_data/pari/simon/ellQ.gp b/src/sage/ext_data/pari/simon/ellQ.gp
index aede9fc941..27cc124372 100644
--- a/src/sage/ext_data/pari/simon/ellQ.gp
+++ b/src/sage/ext_data/pari/simon/ellQ.gp
@@ -1162,7 +1162,7 @@ if( DEBUGLEVEL_ell >= 4, print("    kerval = ",kerval));
       LS2gen[j]^kerval[j,i]));
 
 \\ Add the units
-  LS2gen = concat(Mod(bnf[8][5],bnf.pol),LS2gen); \\ LS2gen = concat(bnf.fu,LS2gen);
+  LS2gen = concat(bnf.fu,LS2gen); \\ LS2gen = concat(bnf.fu,LS2gen);
 \\ Add also the torsion unit if its order is divisible by p.
   if( bnf[8][4][1]%p == 0, \\ if( bnf.tu[1]%p == 0,
     LS2gen = concat( [Mod(bnf[8][4][2],bnf.pol)], LS2gen)); \\ LS2gen = concat( [Mod(bnf.tu[2],bnf.pol)], LS2gen));

comment:27 Changed 2 years ago by git

Commit: 1ce6c2271115018f2eeeb1d487a2cb131bc46689264775b12d89cade4180370aae3e3cbf0da5f875

Branch pushed to git repo; I updated commit sha1. New commits:

264775bFix Denis Simon GP script (following A. Rojas)

comment:28 Changed 2 years ago by git

Commit: 264775b12d89cade4180370aae3e3cbf0da5f875165c470d30641da4d37e7042886c8bf70e1d40ae

Branch pushed to git repo; I updated commit sha1. New commits:

165c470fix units in number fields

comment:29 Changed 2 years ago by Vincent Delecroix

I think all necessary adaptations in Sage code (not doctests) are done. The only remaining trouble is the following computation that now hangs

sage: p = next_prime(10^40); q = next_prime(10^41)
sage: K.<a> = NumberField(x^2 - p*q, maximize_at_primes=[p])
sage: K.pari_nf()

(see doctest line 4134 in sage/rings/number_field/number_field.py)

NB: In unit_group.py I am using methods bnf_get_fu and bnf_get_tu from cypari2 that are not yet in 2.1.2b1.

comment:30 in reply to:  29 ; Changed 2 years ago by Antonio Rojas

Replying to vdelecroix:

I think all necessary adaptations in Sage code (not doctests) are done. The only remaining trouble is the following computation that now hangs

It doesn't really hang, it's just factoring the discriminant. See https://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=2257

comment:31 Changed 2 years ago by Antonio Rojas

Any reason you're including the deprecation warnings for Strchr and besseln instead of replacing the functions themselves (with strchr and bessely)? Do we want to keep compatibility with older pari?

comment:32 Changed 2 years ago by Vincent Delecroix

No. Let me change that.

comment:33 Changed 2 years ago by git

Commit: 165c470d30641da4d37e7042886c8bf70e1d40aedb1f7a448fe2ddea0d839aa2354a8d03b0384f89

Branch pushed to git repo; I updated commit sha1. New commits:

8879b2cStrchr -> strchr and .besseln -> bessely
f63e719add primes to pari when maximize_at_primes is provided
7eac5c9use bnf_get_fu instead of bnfunit
db1f7a4be more precise in documentation

comment:34 in reply to:  30 Changed 2 years ago by Vincent Delecroix

Replying to arojas:

Replying to vdelecroix:

I think all necessary adaptations in Sage code (not doctests) are done. The only remaining trouble is the following computation that now hangs

It doesn't really hang, it's just factoring the discriminant. See https://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=2257

In f63e719I I tweaked the pari_nf to add primes to pari in this situation so that discriminant factorizes. The doctests pass now.

comment:35 Changed 2 years ago by Antonio Rojas

I'm also experiencing a hang in sage/tests/parigp.py, at

sage: pari('K = bnfinit(y^4-52*y^2+26,1); pol = rnfkummer(bnrinit(K,3,1),Mat(5)); L = rnfinit(K, pol); polredabs(polredbest(L.polabs))')  # long time

Running the same code in gp gives a stack overflow

? pol = rnfkummer(bnrinit(K,3,1),Mat(5))
  ***   at top-level: pol=rnfkummer(bnrinit(K,3,1),Mat(5))
  ***                     ^--------------------------------
  *** rnfkummer: the PARI stack overflows !
  current stack size: 8000000 (7.629 Mbytes)
  [hint] set 'parisizemax' to a nonzero value in your GPRC

comment:36 Changed 2 years ago by Vincent Delecroix

PARI doc says: rnfkummer is deprecated use bnrclassfield

comment:37 Changed 2 years ago by Vincent Delecroix

In GP, did you do add the relevant primes first?

? addprimes([31438243, 238576291, 18775387483, 24217212463267, 1427657500359111961, 135564809928627323997297867381959])

comment:38 in reply to:  37 Changed 2 years ago by Antonio Rojas

Replying to vdelecroix:

In GP, did you do add the relevant primes first?

? addprimes([31438243, 238576291, 18775387483, 24217212463267, 1427657500359111961, 135564809928627323997297867381959])

yes

comment:39 Changed 2 years ago by Vincent Delecroix

Description: modified (diff)

comment:40 Changed 2 years ago by Vincent Delecroix

Description: modified (diff)

comment:41 Changed 2 years ago by git

Commit: db1f7a448fe2ddea0d839aa2354a8d03b0384f894e43afdc511baf5b7880f8eae41bc56c9ca33bc9

Branch pushed to git repo; I updated commit sha1. New commits:

4e43afdupgrade cypari2 to 2.1.2b1

comment:42 Changed 2 years ago by git

Commit: 4e43afdc511baf5b7880f8eae41bc56c9ca33bc9b53bb878ff6cd78dd6e7c2344c994792087fc582

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b53bb87upgrade cypari2 to 2.1.2

comment:43 Changed 2 years ago by Vincent Delecroix

Remaining elementary doctest failures

sage -t --random-seed=0 schemes/elliptic_curves/ell_rational_field.py  # 1 doctest failed
sage -t --random-seed=0 schemes/elliptic_curves/ell_number_field.py  # 3 doctests failed
sage -t --random-seed=0 doctest/sources.py  # 1 doctest failed
sage -t --random-seed=0 misc/sagedoc.py  # 4 doctests failed
sage -t --random-seed=0 rings/number_field/number_field.py  # 1 doctest failed
sage -t --random-seed=0 schemes/elliptic_curves/isogeny_small_degree.py  # 1 doctest failed
sage -t --random-seed=0 schemes/elliptic_curves/gal_reps_number_field.py  # 1 doctest failed
sage -t --random-seed=0 modules/torsion_quadratic_module.py  # 3 doctests failed
sage -t --random-seed=0 arith/misc.py  # 2 doctests failed
sage -t --random-seed=0 groups/fqf_orthogonal.py  # 2 doctests failed
sage -t --random-seed=0 quadratic_forms/genera/genus.py  # 2 doctests failed
sage -t --random-seed=0 doctest/forker.py  # 1 doctest failed
sage -t --random-seed=0 rings/polynomial/polynomial_element.pyx  # 1 doctest failed
sage -t --random-seed=0 schemes/elliptic_curves/ell_finite_field.py  # 3 doctests failed
sage -t --random-seed=0 schemes/toric/chow_group.py  # 3 doctests failed
sage -t --random-seed=0 rings/finite_rings/finite_field_constructor.py  # 2 doctests failed
sage -t --random-seed=0 rings/number_field/number_field_ideal.py  # 4 doctests failed
sage -t --random-seed=0 schemes/elliptic_curves/ell_generic.py  # 1 doctest failed
sage -t --random-seed=0 modular/local_comp/smoothchar.py  # 2 doctests failed
sage -t --random-seed=0 rings/finite_rings/residue_field.pyx  # 7 doctests failed
sage -t --random-seed=0 rings/number_field/number_field_rel.py  # 2 doctests failed
sage -t --random-seed=0 modules/fg_pid/fgp_module.py  # 33 doctests failed
sage -t --random-seed=0 rings/number_field/order.py  # 1 doctest failed
sage -t --random-seed=0 rings/finite_rings/integer_mod_ring.py  # 1 doctest failed
sage -t --random-seed=0 schemes/elliptic_curves/period_lattice.py  # 1 doctest failed
sage -t --random-seed=0 modular/modsym/p1list_nf.py  # 1 doctest failed
sage -t --random-seed=0 rings/polynomial/polynomial_quotient_ring.py  # 1 doctest failed
sage -t --random-seed=0 groups/abelian_gps/abelian_group.py  # 1 doctest failed
sage -t --random-seed=0 schemes/toric/points.py  # 9 doctests failed
sage -t --random-seed=0 libs/pari/tests.py  # 2 doctests failed
sage -t --random-seed=0 rings/number_field/number_field_ideal_rel.py  # 1 doctest failed
sage -t --random-seed=0 rings/number_field/S_unit_solver.py  # 8 doctests failed
sage -t --random-seed=0 modular/local_comp/liftings.py  # 1 doctest failed
sage -t --random-seed=0 schemes/toric/homset.py  # 3 doctests failed
sage -t --random-seed=0 all.py  # 1 doctest failed
sage -t --random-seed=0 lfunctions/pari.py  # 1 doctest failed
sage -t --random-seed=0 rings/number_field/unit_group.py  # 10 doctests failed
sage -t --random-seed=0 tests/books/computational-mathematics-with-sagemath/linalg_doctest.py  # 1 doctest failed
sage -t --random-seed=0 rings/number_field/class_group.py  # 2 doctests failed
sage -t --random-seed=0 modules/fg_pid/fgp_morphism.py  # 5 doctests failed
sage -t --random-seed=0 structure/factorization.py  # 4 doctests failed
sage -t --random-seed=0 tests/books/judson-abstract-algebra/galois-sage.py  # 1 doctest failed
sage -t --random-seed=0 modules/free_module_morphism.py  # 3 doctests failed
sage -t --random-seed=0 groups/additive_abelian/additive_abelian_group.py  # 10 doctests failed
sage -t --random-seed=0 modules/fg_pid/fgp_element.py  # 8 doctests failed
sage -t --random-seed=0 libs/pari/__init__.py  # 1 doctest failed
sage -t --random-seed=0 docs/conf.py  # 1 doctest failed
sage -t --random-seed=0 matrix/matrix1.pyx  # 1 doctest failed
sage -t --random-seed=0 rings/number_field/number_field_element.pyx  # 3 doctests failed
sage -t --random-seed=0 rings/integer.pyx  # 1 doctest failed

comment:44 Changed 2 years ago by Tobias Hansen

Using sage 9.2 with the patch from this ticket, when spkg-configure is checking whether the pari 2.13 Debian package is usable, the check

checking whether bnfisunit bug of pari 2.11.3 is fixed...

now returns no. The check passed with Debian's pari 2.11.4.

Last edited 2 years ago by Tobias Hansen (previous) (diff)

comment:45 in reply to:  44 Changed 2 years ago by Dima Pasechnik

Replying to thansen:

Using sage 9.2 with the patch from this ticket, when spkg-configure is checking whether the pari 2.13 Debian package is usable, the check

checking whether bnfisunit bug of pari 2.11.3 is fixed...

now returns no. The check passed with Debian's pari 2.11.4.

yes, this the question of correct output format, which changed, see https://trac.sagemath.org/ticket/30801#comment:6

I've opened #30906 to fix this (the plan is to allow both formats, old and new)

comment:46 Changed 2 years ago by Tobias Hansen

I see, thanks.

comment:47 Changed 2 years ago by Matthias Köppe

build/pkgs/cypari/checksums.ini needs upstream_url. Also, perhaps the spkg should be renamed to cypari2?

comment:48 Changed 2 years ago by Matthias Köppe

Summary: Upgrade: pari 2.13Upgrade: pari 2.13, cypari2 2.1.2

comment:49 in reply to:  35 Changed 2 years ago by Tobias Hansen

Replying to arojas:

I'm also experiencing a hang in sage/tests/parigp.py, at

sage: pari('K = bnfinit(y^4-52*y^2+26,1); pol = rnfkummer(bnrinit(K,3,1),Mat(5)); L = rnfinit(K, pol); polredabs(polredbest(L.polabs))')  # long time

Running the same code in gp gives a stack overflow

? pol = rnfkummer(bnrinit(K,3,1),Mat(5))
  ***   at top-level: pol=rnfkummer(bnrinit(K,3,1),Mat(5))
  ***                     ^--------------------------------
  *** rnfkummer: the PARI stack overflows !
  current stack size: 8000000 (7.629 Mbytes)
  [hint] set 'parisizemax' to a nonzero value in your GPRC

After setting parisizemax to 32000000, the rnfinit(K, pol) also seems to hang directly in gp.

comment:50 Changed 2 years ago by Tobias Hansen

Here is the upstream bug report for the timeout, with an answer from Bill Allombert: https://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=2259

comment:51 Changed 2 years ago by Vincent Delecroix

Dependencies: #30849#30849, #31029
Description: modified (diff)
Summary: Upgrade: pari 2.13, cypari2 2.1.2Upgrade: pari 2.13

I decided to split cypari2 upgrade which is ready to go #31029.

comment:52 Changed 2 years ago by Sébastien Labbé

I don't know if it can be useful, but archlinux has a patch called sagemath-pari-2.13.patch here: https://aur.archlinux.org/cgit/aur.git/tree/?h=sagemath-git and debian too here: https://sources.debian.org/patches/sagemath/9.2-2/ (links from #31016)

Last edited 2 years ago by Sébastien Labbé (previous) (diff)

comment:53 in reply to:  52 Changed 2 years ago by Antonio Rojas

Replying to slabbe:

I don't know if it can be useful, but archlinux has a patch called sagemath-pari-2.13.patch here: https://aur.archlinux.org/cgit/aur.git/tree/?h=sagemath-git and debian too here: https://sources.debian.org/patches/sagemath/9.2-2/ (links from #31016)

The Arch one is essentially this branch plus some trivial test output fixes to account for the new Smith normal form output. The main remaining blocker is comment:13 - adapt the free abelian group quotient construction to the Smith normal form changes. Does anybody know where the quotient map is defined?

comment:54 Changed 23 months ago by Matthias Köppe

Dependencies: #30849, #31029

Needs rebase

comment:55 Changed 21 months ago by Matthias Köppe

Any progress here?

comment:56 Changed 21 months ago by Antonio Rojas

Branch: u/vdelecroix/pari-2.13.0-cypari-2.1.2u/arojas/pari-2.13.0-cypari-2.1.2

comment:57 Changed 21 months ago by git

Commit: b53bb878ff6cd78dd6e7c2344c994792087fc582604dfe41ef8c5065eb90f1e3beb83ae9bbc15617

Branch pushed to git repo; I updated commit sha1. New commits:

604dfe4Update pari to 2.13.1

comment:58 Changed 21 months ago by git

Commit: 604dfe41ef8c5065eb90f1e3beb83ae9bbc15617574f98d5e069db4875b1b12851ed6b3b64b41813

Branch pushed to git repo; I updated commit sha1. New commits:

574f98dFix merge

comment:59 Changed 21 months ago by git

Commit: 574f98d5e069db4875b1b12851ed6b3b64b41813b6c0c9a2c64ade403c829383e249694d6a5aa99e

Branch pushed to git repo; I updated commit sha1. New commits:

3b148faUpdate doctests
b6c0c9aFix doctest failures caused by different choices of generators

comment:60 Changed 21 months ago by git

Commit: b6c0c9a2c64ade403c829383e249694d6a5aa99ee99a2034a92ac4f399500db73fc2ca357401e392

Branch pushed to git repo; I updated commit sha1. New commits:

e99a203Fix more tests

comment:61 Changed 21 months ago by Gonzalo Tornaría

comment:62 Changed 21 months ago by git

Commit: e99a2034a92ac4f399500db73fc2ca357401e392f79004d37b0d3852a429c1b4915a1635c1740f19

Branch pushed to git repo; I updated commit sha1. New commits:

f79004dMerge branch 'develop' of git://git.sagemath.org/sage into t/30801/pari-2.13.0-cypari-2.1.2

comment:63 Changed 21 months ago by git

Commit: f79004d37b0d3852a429c1b4915a1635c1740f194d92bcae4f91005a1cd85a3509eaf6a159cdabe6

Branch pushed to git repo; I updated commit sha1. New commits:

4d92bcado the bnfisunit check in Pari, not rely on output

comment:64 Changed 21 months ago by Gonzalo Tornaría

Thanks. This is what I got with e99a203 using system pari-2.13.1 (voidlinux).

----------------------------------------------------------------------
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/ell_number_field.py  # 3 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/gal_reps_number_field.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/ell_rational_field.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/isogeny_small_degree.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/doc/en/thematic_tutorials/sandpile.rst  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/quadratic_forms/genera/genus.py  # 2 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/toric/chow_group.py  # 3 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/ell_generic.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modular/local_comp/smoothchar.py  # 2 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/doc/en/thematic_tutorials/explicit_methods_in_number_theory/nf_galois_groups.rst  # 5 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/rings/number_field/number_field_rel.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modular/modsym/p1list_nf.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modules/torsion_quadratic_module.py  # 3 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/groups/fqf_orthogonal.py  # 2 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/period_lattice.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/rings/number_field/number_field_ideal.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/elliptic_curves/ell_finite_field.py  # 3 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modules/fg_pid/fgp_module.py  # 30 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/toric/points.py  # 9 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/groups/abelian_gps/abelian_group.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/libs/pari/tests.py  # 2 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/schemes/toric/homset.py  # 3 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modular/local_comp/liftings.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/lfunctions/pari.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/structure/factorization.py  # 4 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/tests/books/computational-mathematics-with-sagemath/linalg_doctest.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modules/fg_pid/fgp_morphism.py  # 5 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/tests/books/judson-abstract-algebra/galois-sage.py  # 1 doctest failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/groups/additive_abelian/additive_abelian_group.py  # 10 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/modules/fg_pid/fgp_element.py  # 8 doctests failed
sage -t --warn-long 31.3 --random-seed=0 src/sage/libs/pari/__init__.py  # 1 doctest failed
----------------------------------------------------------------------

New commits:

f79004dMerge branch 'develop' of git://git.sagemath.org/sage into t/30801/pari-2.13.0-cypari-2.1.2
4d92bcado the bnfisunit check in Pari, not rely on output

comment:65 Changed 21 months ago by Antonio Rojas

Yes, this still needs a lot of work. In particular, someone needs to look into comment:13

comment:66 Changed 21 months ago by Gonzalo Tornaría

Regarding comment:13, it seems to me an issue migth be that smith_form() is not "idempotent" and maybe abelian groups expect it to be, as in:

  • if X is already in smith form, then the output is (X, id, id)

This seems to be true in pari (both 2.11 and 2.13) and it seems to be true in sage using pari 2.11 but 'not' using pari 2.13.

Sage with pari 2.11:

sage: Matrix([[1,0],[0,6]]).smith_form()                                                                                                                                      
(
[1 0]  [1 0]  [1 0]
[0 6], [0 1], [0 1]
)

Sage with pari 2.13:

sage: Matrix([[1,0],[0,6]]).smith_form()                                                                                                                                      
(
[1 0]  [ 1  1]  [ 1  6]
[0 6], [ 0 -1], [ 0 -1]
)

The mismatch is that the smith form in sage is the reverse of the smith form in pari, and the way this is fixed is not "stable". It just happened to work with pari-2.11 by chance.

If this is the case, then a fix might be to reverse the matrix before passing it to pari, etc. In fact, doing that fixes the issue in comment:13, and it also fixes 9 out of 10 test failures in src/sage/groups/additive_abelian/additive_abelian_group.py. It doesn't seem to fix anything in src/sage/modules/fg_pid/fgp_element.py, or anything else, though.

I didn't really look into additive abelian group, so I might be wrong.

The patch I'm trying is this:

--- a/src/sage/matrix/matrix_integer_dense.pyx
+++ b/src/sage/matrix/matrix_integer_dense.pyx
@@ -2444,7 +2444,8 @@ cdef class Matrix_integer_dense(Matrix_dense):
 
            :meth:`elementary_divisors`
         """
-        v = self.__pari__().matsnf(1).sage()
+        X = self.matrix_space()([self[i,j] for i in xrange(self._nrows-1,-1,-1) for j in xrange(self._ncols-1,-1,-1)])
+        v = X.__pari__().matsnf(1).sage()
         # need to reverse order of rows of U, columns of V, and both of D.
         D = self.matrix_space()([v[2][i,j] for i in xrange(self._nrows-1,-1,-1) for j in xrange(self._ncols-1,-1,-1)])
 
@@ -2460,13 +2461,13 @@ cdef class Matrix_integer_dense(Matrix_dense):
             # silly special cases for matrices with 0 columns (PARI has a unique empty matrix)
             U = self.matrix_space(ncols = self._nrows)(1)
         else:
-            U = self.matrix_space(ncols = self._nrows)([v[0][i,j] for i in xrange(self._nrows-1,-1,-1) for j in xrange(self._nrows)])
+            U = self.matrix_space(ncols = self._nrows)([v[0][i,j] for i in xrange(self._nrows-1,-1,-1) for j in xrange(self._nrows-1,-1,-1)])
 
         if self._nrows == 0:
             # silly special cases for matrices with 0 rows (PARI has a unique empty matrix)
             V = self.matrix_space(nrows = self._ncols)(1)
         else:
-            V = self.matrix_space(nrows = self._ncols)([v[1][i,j] for i in xrange(self._ncols) for j in xrange(self._ncols-1,-1,-1)])
+            V = self.matrix_space(nrows = self._ncols)([v[1][i,j] for i in xrange(self._ncols-1,-1,-1) for j in xrange(self._ncols-1,-1,-1)])
 
         return D, U, V
 

comment:67 Changed 21 months ago by Gonzalo Tornaría

The patch in the last comment should be in branch u/tornaria/pari-snf, commit f893a25.

comment:68 Changed 21 months ago by git

Commit: 4d92bcae4f91005a1cd85a3509eaf6a159cdabe69c8e76e055939be74aadeb49a4d9ed47a00ade1f

Branch pushed to git repo; I updated commit sha1. New commits:

9c8e76etry to make smith_form() 'idempotent'

comment:69 in reply to:  67 Changed 21 months ago by Antonio Rojas

Authors: Vincent DelecroixVincent Delecroix, Antonio Rojas, Gonzalo Tornaría

Replying to tornaria:

The patch in the last comment should be in branch u/tornaria/pari-snf, commit f893a25.

Thanks, this looks reasonable and fixes the weirdnesses in abelian group quotients. Unfortunately it introduces a bunch of new test failures due the change of transition matrices, but those seem easy to deal with.

comment:70 Changed 21 months ago by git

Commit: 9c8e76e055939be74aadeb49a4d9ed47a00ade1fb2eac8c9b53d3f9fd5fa6c0da46a14be35dc4bb4

Branch pushed to git repo; I updated commit sha1. New commits:

b2eac8cUpdate some tests for new Smith form transition matrices

comment:71 Changed 21 months ago by Gonzalo Tornaría

It probably needs more work. In particular I think there may still be bad cases, so let's think a bit more about this.

My current thinking (after glancing at smith_form_gens() in fgp_module.py) that maybe something stronger is required, namely:

  • if X in HNF, the corresponding U is a permutation matrix and U.V is upper triangular with 1s in the diagonal or something like that

OTOH I do wonder if this is really a strong requirement for correctness or just tradition.

I mean: what exactly is wrong with the answer in comment:13? I mean: (0,-1) is indeed a generator of V/W. Granted we were expecting (0,1) but the answer is still correct, isn't it?


New commits:

b2eac8cUpdate some tests for new Smith form transition matrices

comment:72 Changed 21 months ago by Mauricio Collares

Cc: Mauricio Collares added

comment:73 Changed 21 months ago by git

Commit: b2eac8c9b53d3f9fd5fa6c0da46a14be35dc4bb452d48d5105dad1b24c332db0ee73be01538c57c2

Branch pushed to git repo; I updated commit sha1. New commits:

6cc043fUpdate some tests for different ideal and unit group generators
52d48d5Fix some tests in sage.schemes.elliptic_curves

comment:74 Changed 21 months ago by git

Commit: 52d48d5105dad1b24c332db0ee73be01538c57c2bb3f783b30a02fb09113950d12b7cb73df4311d5

Branch pushed to git repo; I updated commit sha1. New commits:

1080683Merge branch 'develop' of git://git.sagemath.org/sage into t/30801/pari-2.13.0-cypari-2.1.2
bb3f783Fix a few more tests

comment:75 Changed 21 months ago by Matthias Köppe

Description: modified (diff)

comment:76 Changed 21 months ago by Matthias Köppe

Is the plan to drop support for system PARI < 2.13.1? Then spkg-configure.m4 will need changing

comment:77 Changed 21 months ago by Dima Pasechnik

Cc: Isuru Fernando added

I think it's OK to drop older PARIs. Several Linux distros (Debian, Gentoo, arch (I guess)) and Homebrew already carry PARI 2.13.

It seems that only Conda is behind. Isuru, is Conda's PARI update in the works?

comment:78 Changed 21 months ago by gh-dkwo

Do you know whether the error isirreducible: symbol not found that I get in #31392 is related to this ticket?

comment:79 Changed 21 months ago by gh-dkwo

Cc: gh-dkwo added

comment:80 in reply to:  78 Changed 21 months ago by Dima Pasechnik

Replying to gh-dkwo:

Do you know whether the error isirreducible: symbol not found that I get in #31392 is related to this ticket?

Most probably (there you build Sage 9.2 with system's Pari 2.13, if I am not mistaken). Try to apply the branch from here and see if it helps.

comment:81 in reply to:  77 Changed 21 months ago by Isuru Fernando

Replying to dimpase:

It seems that only Conda is behind. Isuru, is Conda's PARI update in the works?

I don't have plans yet, but a PR is always welcome.

comment:82 Changed 21 months ago by git

Commit: bb3f783b30a02fb09113950d12b7cb73df4311d5a476ca3f24fea5afaef64a1f6225719050b79d76

Branch pushed to git repo; I updated commit sha1. New commits:

d66a905Fix some more tests
55afe2eFix timeout in sage/tests/parigp.py
a476ca3Fix some tests in sage.schemes.toric

comment:83 in reply to:  71 Changed 21 months ago by Antonio Rojas

Replying to tornaria:

It probably needs more work. In particular I think there may still be bad cases, so let's think a bit more about this.

My current thinking (after glancing at smith_form_gens() in fgp_module.py) that maybe something stronger is required, namely:

  • if X in HNF, the corresponding U is a permutation matrix and U.V is upper triangular with 1s in the diagonal or something like that

OTOH I do wonder if this is really a strong requirement for correctness or just tradition.

I mean: what exactly is wrong with the answer in comment:13? I mean: (0,-1) is indeed a generator of V/W. Granted we were expecting (0,1) but the answer is still correct, isn't it?

So, how do we proceed? Sure, both answers are mathematically correct, strictly speaking. But the fact that things like this now give the "natural" answer makes me think that your change is an overall improvement:

sage: V = ZZ^3; W = V.span([2*V.0, 2*V.1, 2*V.2])                                                                                                         
sage: Q = V/W                                                                                                                                             
sage: Q.gens()                                                                                                                                            
((1, 0, 0), (0, 1, 0), (0, 0, 1))
sage: [x.lift() for x in Q.gens()]                                                                                                                        
[(1, 0, 0), (0, 1, 0), (0, 0, 1)]

Before:

sage: [x.lift() for x in Q.gens()]                                                                                                                        
[(0, 0, 1), (0, 1, 0), (1, 0, 0)]

comment:84 Changed 21 months ago by Antonio Rojas

sage -t --long --random-seed=0 src/sage/schemes/elliptic_curves/gal_reps_number_field.py  # 1 doctest failed
sage -t --long --random-seed=0 src/sage/schemes/elliptic_curves/kraus.py  # 1 doctest failed

These fail because pari throws multiple precision too low for generators, not given. warnings. Is this something we should care about, or should we just filter them out?

comment:85 Changed 21 months ago by git

Commit: a476ca3f24fea5afaef64a1f6225719050b79d76273b2b2868c0cd7163b6676d2b769c582bbd3f98

Branch pushed to git repo; I updated commit sha1. New commits:

6144631Fix doctests in sage.modular for new basis choices
273b2b2Fix some tests in sage/schemes/elliptic_curves

comment:86 Changed 21 months ago by git

Commit: 273b2b2868c0cd7163b6676d2b769c582bbd3f98c3f2aad8fe17d92629994a395e9982b95aaff1e9

Branch pushed to git repo; I updated commit sha1. New commits:

e3be5e5Update tests for new precision() function
c3f2aadUpdate tests for updated quadhilbert() output in pari

comment:87 Changed 21 months ago by Dima Pasechnik

what is the status here? still "needs work"?

comment:88 in reply to:  87 Changed 21 months ago by Antonio Rojas

Replying to dimpase:

what is the status here? still "needs work"?

yes (a lot)

comment:89 Changed 21 months ago by Matthias Köppe

Milestone: sage-9.3sage-9.4

comment:90 Changed 21 months ago by git

Commit: c3f2aad8fe17d92629994a395e9982b95aaff1e9756781e58cdc0b8d64b1c4c129100dfe544261fd

Branch pushed to git repo; I updated commit sha1. New commits:

4686445Merge branch 'develop' of git://git.sagemath.org/sage into t/30801/pari-2.13.0-cypari-2.1.2
ec66c3cChange example to one where the is actually no coercion map
7c7843dFix tests in sage.geometry for new lattice generators
3dc3251Fix tests in sage.schemes.toric for new lattice generators
530914dFix one test in judson-abstract-algebra
756781eFix tests in homology for different choice of generators

comment:91 Changed 21 months ago by git

Commit: 756781e58cdc0b8d64b1c4c129100dfe544261fdd7cf52798714865e75537bd080345b62ca9d78b0

Branch pushed to git repo; I updated commit sha1. New commits:

a9fb51cUpdate tests in modules.fg_pid for new Smith normal form
d7cf527Fix tests in torsion quadratic modules for new Smith normal form

comment:92 Changed 21 months ago by Antonio Rojas

Remaining issues:

sage -t --long --random-seed=0 src/sage/sandpiles/sandpile.py  # 5 doctests failed
sage -t --long --random-seed=0 src/doc/en/thematic_tutorials/sandpile.rst  # 5 doctests failed

These are probably OK, but someone who understands sandpiles should look at them to make sure.

sage -t --long --random-seed=0 src/sage/schemes/elliptic_curves/gal_reps_number_field.py  # 1 doctest failed
sage -t --long --random-seed=0 src/sage/schemes/elliptic_curves/kraus.py  # 1 doctest failed

See comment:84

sage -t --long --random-seed=0 src/sage/schemes/elliptic_curves/ell_number_field.py  # 5 doctests failed

simon_two_descent gives some different answers, someone should check if they're correct.

sage -t --long --random-seed=0 src/sage/rings/number_field/number_field_ideal.py  # 1 doctest failed

stack overflow: cypari2.handle_error.PariError: the PARI stack overflows (current size: 8000000; maximum size: 865075200)

sage -t --long --random-seed=0 src/sage/rings/number_field/number_field_rel.py  # 1 doctest failed

This one looks fishy: a relative discriminant gives a mathematically different answer now.

comment:93 in reply to:  92 Changed 21 months ago by Antonio Rojas

Report Upstream: N/AReported upstream. No feedback yet.

Replying to arojas:

sage -t --long --random-seed=0 src/sage/rings/number_field/number_field_rel.py  # 1 doctest failed

This one looks fishy: a relative discriminant gives a mathematically different answer now.

The new answer seems wrong, I've reported it upstream

https://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=2284

comment:94 Changed 21 months ago by git

Commit: d7cf52798714865e75537bd080345b62ca9d78b0b21951e876b318dcdf3fff1467f37136e9dbc854

Branch pushed to git repo; I updated commit sha1. New commits:

b21951eBackport rnfdisc fix

comment:95 Changed 21 months ago by Antonio Rojas

Report Upstream: Reported upstream. No feedback yet.Fixed upstream, but not in a stable release.

And fixed

comment:96 Changed 20 months ago by git

Commit: b21951e876b318dcdf3fff1467f37136e9dbc854b11ff20d03e21a98e0122f9d0adfe4dd72c0e042

Branch pushed to git repo; I updated commit sha1. New commits:

eaa1e38A few more test fixes in schemes.toric
df8c4d0One trivial fix in schemes.elliptic_curves
b11ff20Elliptic curve group generators can be different on each run, mark tests as random

comment:97 Changed 20 months ago by Vincent Delecroix

How did it happen that the cardinality of the elliptic curve changed? Not by much but still... at most one of the proposed result is correct.

  • src/sage/schemes/elliptic_curves/ell_finite_field.py

    a b class EllipticCurve_finite_field(EllipticCurve_field, HyperellipticCurve_finite_ 
    785785            sage: len(E.gens())
    786786            2
    787787            sage: E.cardinality()
    788             867361737988403547207212930746733987710588
    789             sage: E.gens()[0].order()
    790             433680868994201773603606465373366993855294
    791             sage: E.gens()[1].order()
    792             433680868994201773603606465373366993855294
     788            867361737988403547206134229616487867594472
     789            sage: a = E.gens()[0].order(); a # random
     790            433680868994201773603067114808243933797236
     791            sage: b = E.gens()[1].order(); b # random
     792            30977204928157269543076222486303138128374
     793            sage: lcm(a,b)
     794            433680868994201773603067114808243933797236
    793795        """
    794796        G = self.__pari__().ellgroup(flag=1)
    795797        return tuple(self.point(list(pt)) for pt in G[2])

comment:98 in reply to:  97 ; Changed 20 months ago by Antonio Rojas

Replying to vdelecroix:

How did it happen that the cardinality of the elliptic curve changed? Not by much but still... at most one of the proposed result is correct.

Because the default generators for finite fields have changed between 2.11 and 2.13. So, in k.<a> = GF(5^60), 'a' is a different element now. See also the related changes in finite_field_constructor.py

comment:99 Changed 20 months ago by Vincent Delecroix

I will have a look at the sandpile.

comment:100 in reply to:  98 Changed 20 months ago by Antonio Rojas

Replying to arojas:

Replying to vdelecroix:

How did it happen that the cardinality of the elliptic curve changed? Not by much but still... at most one of the proposed result is correct.

Because the default generators for finite fields have changed between 2.11 and 2.13. So, in k.<a> = GF(5^60), 'a' is a different element now. See also the related changes in finite_field_constructor.py

Just to double check:

With 2.11

sage: k.<a> = GF(5^60)
sage: a.minpoly()
x^60 + 2*x^59 + x^58 + 3*x^57 + 2*x^56 + x^55 + 2*x^54 + 4*x^53 + 4*x^52 + 4*x^51 + 2*x^50 + x^49 + 4*x^47 + 3*x^45 + 3*x^44 + 3*x^43 + 2*x^42 + 2*x^40 + 3*x^38 + 4*x^37 + 4*x^35 + x^34 + 3*x^33 + 4*x^32 + 3*x^31 + 2*x^30 + 4*x^29 + 3*x^28 + 2*x^27 + 3*x^25 + 4*x^24 + 4*x^23 + 3*x^21 + 4*x^20 + 2*x^19 + 4*x^18 + 4*x^17 + 4*x^16 + 3*x^15 + x^14 + 4*x^13 + 2*x^11 + x^10 + x^8 + 3*x^7 + 4*x^5 + 3*x^4 + 4*x^3 + 3*x^2 + 3
sage: E = EllipticCurve([a, a])
sage: E.cardinality()
867361737988403547207212930746733987710588

With 2.13:

sage: k.<a> = GF(5^60)
sage: a.minpoly()
x^60 + 3*x^56 + 3*x^55 + x^52 + 2*x^51 + 3*x^50 + x^45 + x^41 + x^37 + 3*x^36 + 3*x^35 + x^33 + 4*x^32 + 3*x^31 + 2*x^30 + x^29 + 3*x^28 + 4*x^27 + 2*x^26 + 4*x^21 + 4*x^20 + 4*x^17 + 4*x^16 + 2*x^15 + 4*x^13 + x^11 + 4*x^10 + 4*x^9 + 2*x^8 + 2*x^5 + 2*x^3 + 4*x^2 + 2*x + 4
sage: E = EllipticCurve([a, a])
sage: E.cardinality()
867361737988403547206134229616487867594472
sage: k.<b> = GF(5^60, modulus=x^60 + 2*x^59 + x^58 + 3*x^57 + 2*x^56 + x^55 + 2*x^54 + 4*x^53 + 4*x^52 + 4*x^51 + 2*x^50 + x^49 + 4*x^47 + 3*x^45 + 3*x^4
....: 4 + 3*x^43 + 2*x^42 + 2*x^40 + 3*x^38 + 4*x^37 + 4*x^35 + x^34 + 3*x^33 + 4*x^32 + 3*x^31 + 2*x^30 + 4*x^29 + 3*x^28 + 2*x^27 + 3*x^25 + 4*x^24 + 4*
....: x^23 + 3*x^21 + 4*x^20 + 2*x^19 + 4*x^18 + 4*x^17 + 4*x^16 + 3*x^15 + x^14 + 4*x^13 + 2*x^11 + x^10 + x^8 + 3*x^7 + 4*x^5 + 3*x^4 + 4*x^3 + 3*x^2 + 
....: 3)
sage: E2=EllipticCurve([b,b])
sage: E2.cardinality()
867361737988403547207212930746733987710588

so it's all good.

comment:101 Changed 20 months ago by Vincent Delecroix

:)

comment:102 Changed 20 months ago by Vincent Delecroix

Nothing to worry about in sandpile. All come from the change in Smith form and are valid answers.

comment:103 Changed 20 months ago by git

Commit: b11ff20d03e21a98e0122f9d0adfe4dd72c0e0424f53a36c00471a21fc465fb0d798c9e6f6b3c49e

Branch pushed to git repo; I updated commit sha1. New commits:

4f53a36Update sandpiles tests for new Smith form

comment:104 Changed 20 months ago by Vincent Delecroix

The precision too low in 84 comes from

sage: from sage.schemes.elliptic_curves.gal_reps_number_field import deg_one_primes_iter
sage: K.<a> = NumberField(x^2 - x + 112941801)
sage: P = deg_one_primes_iter(K, principal_only=True)
sage: _ = [next(P) for _ in range(3)]
  ***   Warning: precision too low for generators, not given.
  ***   Warning: precision too low for generators, not given.
  ***   Warning: precision too low for generators, not given.
  ***   Warning: precision too low for generators, not given.
Last edited 20 months ago by Vincent Delecroix (previous) (diff)

comment:105 Changed 20 months ago by Vincent Delecroix

Actully, no need for elliptic curve to trigger the warning

sage: K.<a> = NumberField(x^2 - x + 112941801)
sage: I = K.ideal((112941823, a + 49942513))
sage: I.is_principal()
  ***   Warning: precision too low for generators, not given.
False

(this is called in deg_one_primes_iter at line 803)

The above ultimately calls PARI/GP with

sage: bnf = K.pari_bnf()
sage: hnf = I.pari_hnf()
sage: bnf.bnfisprincipal(hnf, 1)
  ***   Warning: precision too low for generators, not given.
[[5665]~, []~]

And this is very bad: pari does not return the generator (the second item in the last output is empty). In the current sage version we have

bnf.bnfisprincipal(hnf, 1)
[[5665]~, [1005680112856001610090073934296316829467138866361766894525257437882253430879942065774355907438675356488324134889474584998905311415284353681043276105133423115318683310543272073477637300589338960081968708361807703046979508313368984026176585841244175713647681497236281752376606509937398970801793267104940057403236112391113303692273201879958967939674488863025182696946519634023730942473349983082215000407834010466580069375608789784758048104152978461419834803278904319596087872341193829514778888063778316177866395087779299764551322641650792423027180726365995085392258048238098639878001287457838906927025559523280511549386087090336592473896760739042298071391160432291174394824137946860118124404282979229184143719453742756269143097680009342314611948051930283376153131414706670056829331624943848792303898466070751623601801796312310892281516309119701337857274762757320078343943248940019654812535379459245841424571927506504714401584829273515275425263244136314100811717437028483821355172088608914231765343018676088429468602974870678332388362740795503483323546438590543834668601752766799279426809571588639840863182899151994294685170648109254538841007733841265757195339003000012740105833998913751748156278542317450865032226724534159549322507469897715570883846178284884934925927139411108181347492998550960796623038937957015115688920912058287132670816712787077722652142302022813561941270738322779237721182895820974361805531160340984165267884067984384934507471250082396847833818524050466962904745938171905956284686780241294207373575898758348602786420282414104418636033478629514453673727120417545486979349262724942498729615663533408933098472716462610746223340873224916356325049798415709852815179098082945068594726886769245691259236979553269715128964045161931272306583456510695445900210536294492420223945603966435638006097208106702624401128710741275008267785262364203915818653235371290410536706152509400193957786742250154595545629477334640639594843037280585496354412724040607706629548975485280239036672861403792257228273873223951366142131173208977036220063177945056305141723694419208202343495006886847255872360159457890094343404042121364030561355625107989431761677290762878761997805075720739609638293524879891418269558304984789737584642947688856075183040681127570913020836885548135902486022233122268518989917470477171463376207094489443300112083453671975520085981493122180011312717181508397266867443252468650087342491075278577219226483838834975922132435587687556334826969872403765974819119977356548475185141243447860028723934682964287752517823689498457113990739904389583066723293899687467098816867112903821484911791259602500118863723872580834212360274028751115910560284913196192572816918735831007543242727648234580661154261962178769898852141312976299160312006502841696191208025270422489394839650712002785292486960561705299241475775109545547817251668908605704108623609404929381572764215067827488035601722812024134189333895361036686155143470926041441422203185178683221543807531938314899725661504990537887938732251404985938041908193221724324794921781136390123869936226543519311489427540392248343667865593895153766932118235657601052305049272679149137935156623459052384644720342362619275137186290731483488444286389304271148816561733875957377593878312080879984923339287624638790229257972115601321914687187199777361516950494803326116009120509888689926688364677417293793966320775758584809271424360951590676965260954521735044247986018801311108069585561143045069036786312483681260909625432603456526446195112551447608340931959081805965503826397916563831134064470426073588909204625989884612026636269631215105938606528702826734757923982551326485015821546479886614211824267993197710265817389575724543479040788007689943671675047485899341798381799840297618920999806359257046091539158825239171382675316036787306863061087840377357239319898790867517267398778711399174086070859714153327908060380687112/15423787731598121784065181479851317095207965562744556951707615327319988174036900996596787851948102162487535818533246167408447397419284484962946343461932033151014987391818944468080419685988864089974219387374007662090557424422006655336216069435379908000252606721025845935267976159557298178922550487009997527879021269617041774920220964072886295088171880156639876949162194703408177549668332199744187466447813223174472717318384086294240294157543468652016206900519356895158706609190599420181623703486286482521072509532465042319426574643862112732814130061599984365389558490188838773162462651442384961798679610173832674655481574138416535083108050578888973213643445741144149495998169162177225073633210231935418464188261091635402084694759226846304163564397221923885069149960719552734337534114425497804201197146847638901747707144721006983429414898366581039987575516241530461997779437385212642295272483016600460552013332471354322105454747556599800164269728220545584715014879149173700602874605477747964274313916325623253519475913239766035219024045504100513780608732659125134344932814912834698269008084090017279295501852966639433334759085957736584349193299457808760025568726815347265156551997288569855520303083184580727093894206286099137444945253451384172170466112908530803958111097158459503468307451437534445133828888053117285991114033865881137411631835485708185248281679456128881401285564257032504189755725286118402351081317171302436222963916021942398932998857105632612418083654361791817954860417505656364464737860211461429324793494988542438129691273953575854400504405887003002082269313053108684985717274205390809192507223358732261077862527662595468692374702296371975940389359897955820467848264446200797116128658577897255361319125542970727979855299539164774736086342352390472332507124438834338187016413675061887871814555398868333435648711782210171158225697540592179316421357559768754046034560557045147158235469854157988886367295602929376013177169182332473063499496895922329510277364867631515203675503190910661772221158177549639625649093571783636989177154003958257267668260730805197733266709688283450995507923855449160848180441674759974202869687531620064611579801463991337558463643743997267135164234077639052966893238542545548750208212696980293483570717002942340339245397518062128505031494706881569437150040534163842562199131899755456505291752475319505037337714271025136002335192913569940956616169163902567800862024890258150353687883247496151620518758662190976990502427960537248635880282596773227140004292177736957176659160452235030295947942744100500254526493298850043601884234253104837163107628291336853849047349032922715961917418118153997126635661942563766184588457645980308034118788430236317091146609444645125885711112540315091390839292719192995927368217307862766487885884952909747229870463017592863289829170532967404520292121820993254273738890627629459722817897215980320122687403165592871004471038211167521547160486399318758271075144315508301920241394650251902061241787021841817326916417378824310961111757983639607837257509009072952188013741423402177546450623584713467936575516761749606129388853998090896099212762293594464289360758732029903043915542708445230346027591599147100251921863870112777155957711817988808198453771165908839240992119035489368255918464986846384897011045868602012809996701911417125497056612699480011029608303241300120573759919940809625031873997617290638206296876336664521749895994133364527996577259871662621348658003854081116617611523334893799895233315454117878627482715865092777403404290000039905748517139258316452750658333207122746504771475606862464541352233614692387919693313237801391454030184297215851376265590972276932615740994094543032570059914732358087315969509129131465213226337964364991060099390886823404506850632009502355939964365256980689737950555014650360875455621433531197686163996566785513792245023457838396912973254127473243910510573654688099195417343494453770602616193642848137384426169124577035126898650245026236066602939323948634836356704127035766759571373358679773372404475881623110732324054456140506141191811012149002547655229014812462255230135303720711836610126764747857578908314055890267904362154746880790791774593266985409206605469722072489612121470126191161170640137585344788894564385119276745912123555335802678514136687343422978694283526627243455011455093587990305315015794682183783903880302546533554335993322381960581092777146074262180395132207046858557700961280700201084502268413345545221557273152816254355176262742794802475764425737698580710739268628087195538972090318590912938020293852841408652229509844483656138375482948708242772482936665361990110583293262860768893793590172955556511058219658424203481764415924430012043594138005655933182334690019308210425377503727370100872617821601523164268595360660443304798383312297467650776852900112461563735643755909352585538493331481767789166091729453434978046334774247140179202927879199746225254274607506472866113793064345650274075194361194504227747371277617862767557668883880677397101655447974832622218039910391889660315183407560256559581823634686348751999970073338507159538586301746001898994233008724667075468873273429268909954946108472374015002132536966251343158102124440238347206518714347415616357288411158160813885068471448626549535529940737633555166867630276885014395911997648232176044478964205117102414922032065314960549290892077998179541417540291306137128346483175222986932978104286191392167137119056222817931540779913757159699677813708254999512499231303766256633170063282119793861096738828373012184921029465293667029308508857426112584333691741309463516608659736052588112677813543610866157072931265964949327385236209264935350001674757717532833782002008099641314548159468639777158514375801335086321588366628463233073085745744392982060077685319129024219154322573134525849980307737233121167335938628182347209217167257690130545125177873969108518551500995065686858185161519541397222934215544055392555177918529058302177084272491874075954479001465247841514195711761279015092461944770884666266943387468709452105804512201307050102059977841379810443691433051355935697950008342419711170303212511921090976858539118036164631025794898740862743011347633954749555625930473280961552555892377165479621241294839943992032801765507163708685747594848559555727439321273459887604115005341562064199786868571436158926411091472850719249117346546428550026090179384039612628802000610957776293412806618447051914767328893645889410257440364847398021323417813048815233364264704273114718271213520425340337069939876396133983864910147574811777818543655683927575470210079211268820856825639726836979792442447758820728867752361336008456238733666620390993350011824106274337041010679117648752791899869905198820670841771634746344788226807975414277232582151381553005526999373425378324517629421191169842538888155843366625374977129118657562608640832873287338553118786063859419439411462334285128905401552149497810650502735025427299551648144515157989172135679890354595771284131765172232889815864166054850412626480063621939725689936866067157580937044264830849522499516758430087713373429359189895069344766454187296118356652885297735547623443969837999386635109225628187815161497757080986429707705353219928565814662445205762922278917733114362603000541525110244727231616586879079658566405651276697894764234864067938500629562180085344452250106691572409406213072787048805913717109483763718856416913613012912470747632930718980052943756681766116792390349123276672023134828456060499646759933837264765656665596793330816685754103523308815555759665179511327163232285351996699699106863804633863524215279181005741744549306142485407499635207079986089109016524801564602163033986003363161616947551666459438302810674680053735915568635864006123325422535582785394986216592023329686223387365143, 80424414550646364071109413951047320548154724203692500102475964387152343735641955784748411247651791176225695132224501545830655292135351971248388164806308678489271313208673123149238617684205510067472873708708483823065195928585185648791603337039693017693486866125305984139436780928201920611463146920235561398002070224455183901721242650641177500019940806555610990907391822148892007049211319215495906436957606086125307785048118904799833786217368765724665678554101127485210401534445690306996616174545614076586094966258159679745118019220140115744652845218295837382867761138288409776383179585311827387445647338646805169652951119331493582422078526886894128599184974560242366515022171590686902473697233196797944475259058639778693166998923852909777132281665895399358776875234767137523959116595394308077153814711457093749656392193417265063364965333812703551207285083280756261764490953658945584325994516603427977110921650282238227257470153097990914229184083901901982724200800503611983577076526201217189432402688528089891378733592482177343767937677247399457485920975348061621977092999746535686758153425684676668259602753933634715803656277530589863060704342530564598719230638873653686786414955896322303376523061138894646105112943519443671859198816340598881690177758116581556429391931665294816807746500710413244921980760611251491553793888165001492165751540776225622801827638445075682057633889388301297265501704185423645901990910210801486179437831955356267663425311218493520756785259263203713453680163730064600912795788449394635676375721676048771986933389056386796312182892604103410019443042697908355325711801958646674952662128584387098996836292125611626372703986542930508710626471280499507014528354954848914796541777197938611708770915539296664904665044113593147207425231221112425309981190025635812046819915842604972600232502808914299802384397159301990409524126425347635590083353741458937327495404417020380441064365822826273561624208518035359749923593064851368841657912495054172023620903390834252569111873909957574204610904772251677622139737984654424657068605015847311934418911856641140432075855834511260018099715277396427903552075951626852693462074284175653131779541688496384571778652807326845464118195746102804268461777360351306772152021471784506507319736804220662526887565551058168012792652509842056790640954122939876970752988169817651033778764949620315241243314479956742969189135101963641187466734432347780364855216638013470784967435974681028092790311402177665237695821379246728951843415834902614931124860672143148717148526022586488021581461784461336032624243273814257434017810755360044702330597371456403824227161101442131231744253034892467972132649415605601673192023483395081342369752185008081115239666327009462376094472816592692539530899042557145340903398573562268497002949848267741141447119509197909912576437634009706432164020263232320876980120502517199066383470516756823139498324150396385879218790329382427067419359993727617945463698360039605777660585670770078655930977907651587050396042759224406618013258910982169771955461839612415237408978481404186490294047631083557192178973371031103764454876477336071248813527093035819650799905098374703780458879414309264999204021172480163195228510828630921369111155828207883643918569091526307054804267380299993417830166471691160528127969340142505679581805445667946924676591605909584461111115238038985017530634945670189726542180631513464043869965268412793391361182154982878554215607971566272512959835768632115951775492653917164292515967255941162972157441390277695353975991128891020834954034076285000697717265364379515802935964394330065554556407534442672392874973649265676435341099097627811910958024024211733583256194016703868231653619105438392223351899675954867203907924007640153606170556519540171057468097409787451905933867798282836537136835120330461390973442293633912548399707042995856102644430120454632706749426409766313204453/15423787731598121784065181479851317095207965562744556951707615327319988174036900996596787851948102162487535818533246167408447397419284484962946343461932033151014987391818944468080419685988864089974219387374007662090557424422006655336216069435379908000252606721025845935267976159557298178922550487009997527879021269617041774920220964072886295088171880156639876949162194703408177549668332199744187466447813223174472717318384086294240294157543468652016206900519356895158706609190599420181623703486286482521072509532465042319426574643862112732814130061599984365389558490188838773162462651442384961798679610173832674655481574138416535083108050578888973213643445741144149495998169162177225073633210231935418464188261091635402084694759226846304163564397221923885069149960719552734337534114425497804201197146847638901747707144721006983429414898366581039987575516241530461997779437385212642295272483016600460552013332471354322105454747556599800164269728220545584715014879149173700602874605477747964274313916325623253519475913239766035219024045504100513780608732659125134344932814912834698269008084090017279295501852966639433334759085957736584349193299457808760025568726815347265156551997288569855520303083184580727093894206286099137444945253451384172170466112908530803958111097158459503468307451437534445133828888053117285991114033865881137411631835485708185248281679456128881401285564257032504189755725286118402351081317171302436222963916021942398932998857105632612418083654361791817954860417505656364464737860211461429324793494988542438129691273953575854400504405887003002082269313053108684985717274205390809192507223358732261077862527662595468692374702296371975940389359897955820467848264446200797116128658577897255361319125542970727979855299539164774736086342352390472332507124438834338187016413675061887871814555398868333435648711782210171158225697540592179316421357559768754046034560557045147158235469854157988886367295602929376013177169182332473063499496895922329510277364867631515203675503190910661772221158177549639625649093571783636989177154003958257267668260730805197733266709688283450995507923855449160848180441674759974202869687531620064611579801463991337558463643743997267135164234077639052966893238542545548750208212696980293483570717002942340339245397518062128505031494706881569437150040534163842562199131899755456505291752475319505037337714271025136002335192913569940956616169163902567800862024890258150353687883247496151620518758662190976990502427960537248635880282596773227140004292177736957176659160452235030295947942744100500254526493298850043601884234253104837163107628291336853849047349032922715961917418118153997126635661942563766184588457645980308034118788430236317091146609444645125885711112540315091390839292719192995927368217307862766487885884952909747229870463017592863289829170532967404520292121820993254273738890627629459722817897215980320122687403165592871004471038211167521547160486399318758271075144315508301920241394650251902061241787021841817326916417378824310961111757983639607837257509009072952188013741423402177546450623584713467936575516761749606129388853998090896099212762293594464289360758732029903043915542708445230346027591599147100251921863870112777155957711817988808198453771165908839240992119035489368255918464986846384897011045868602012809996701911417125497056612699480011029608303241300120573759919940809625031873997617290638206296876336664521749895994133364527996577259871662621348658003854081116617611523334893799895233315454117878627482715865092777403404290000039905748517139258316452750658333207122746504771475606862464541352233614692387919693313237801391454030184297215851376265590972276932615740994094543032570059914732358087315969509129131465213226337964364991060099390886823404506850632009502355939964365256980689737950555014650360875455621433531197686163996566785513792245023457838396912973254127473243910510573654688099195417343494453770602616193642848137384426169124577035126898650245026236066602939323948634836356704127035766759571373358679773372404475881623110732324054456140506141191811012149002547655229014812462255230135303720711836610126764747857578908314055890267904362154746880790791774593266985409206605469722072489612121470126191161170640137585344788894564385119276745912123555335802678514136687343422978694283526627243455011455093587990305315015794682183783903880302546533554335993322381960581092777146074262180395132207046858557700961280700201084502268413345545221557273152816254355176262742794802475764425737698580710739268628087195538972090318590912938020293852841408652229509844483656138375482948708242772482936665361990110583293262860768893793590172955556511058219658424203481764415924430012043594138005655933182334690019308210425377503727370100872617821601523164268595360660443304798383312297467650776852900112461563735643755909352585538493331481767789166091729453434978046334774247140179202927879199746225254274607506472866113793064345650274075194361194504227747371277617862767557668883880677397101655447974832622218039910391889660315183407560256559581823634686348751999970073338507159538586301746001898994233008724667075468873273429268909954946108472374015002132536966251343158102124440238347206518714347415616357288411158160813885068471448626549535529940737633555166867630276885014395911997648232176044478964205117102414922032065314960549290892077998179541417540291306137128346483175222986932978104286191392167137119056222817931540779913757159699677813708254999512499231303766256633170063282119793861096738828373012184921029465293667029308508857426112584333691741309463516608659736052588112677813543610866157072931265964949327385236209264935350001674757717532833782002008099641314548159468639777158514375801335086321588366628463233073085745744392982060077685319129024219154322573134525849980307737233121167335938628182347209217167257690130545125177873969108518551500995065686858185161519541397222934215544055392555177918529058302177084272491874075954479001465247841514195711761279015092461944770884666266943387468709452105804512201307050102059977841379810443691433051355935697950008342419711170303212511921090976858539118036164631025794898740862743011347633954749555625930473280961552555892377165479621241294839943992032801765507163708685747594848559555727439321273459887604115005341562064199786868571436158926411091472850719249117346546428550026090179384039612628802000610957776293412806618447051914767328893645889410257440364847398021323417813048815233364264704273114718271213520425340337069939876396133983864910147574811777818543655683927575470210079211268820856825639726836979792442447758820728867752361336008456238733666620390993350011824106274337041010679117648752791899869905198820670841771634746344788226807975414277232582151381553005526999373425378324517629421191169842538888155843366625374977129118657562608640832873287338553118786063859419439411462334285128905401552149497810650502735025427299551648144515157989172135679890354595771284131765172232889815864166054850412626480063621939725689936866067157580937044264830849522499516758430087713373429359189895069344766454187296118356652885297735547623443969837999386635109225628187815161497757080986429707705353219928565814662445205762922278917733114362603000541525110244727231616586879079658566405651276697894764234864067938500629562180085344452250106691572409406213072787048805913717109483763718856416913613012912470747632930718980052943756681766116792390349123276672023134828456060499646759933837264765656665596793330816685754103523308815555759665179511327163232285351996699699106863804633863524215279181005741744549306142485407499635207079986089109016524801564602163033986003363161616947551666459438302810674680053735915568635864006123325422535582785394986216592023329686223387365143]~]
Last edited 20 months ago by Vincent Delecroix (previous) (diff)

comment:106 in reply to:  105 ; Changed 20 months ago by Antonio Rojas

Replying to vdelecroix:

And this is very bad: pari does not return the generator (the second item in the last output is empty). In the current sage version we have

Do we actually need the generator here? From number_field_ideal.py:

        # Call bnfisprincipal().
        # If gens_needed, use flag=3 which will insist on computing
        # the generator.  Otherwise, use flag=1, where the generator
        # may or may not be computed.
        v = bnf.bnfisprincipal(self.pari_hnf(), 3 if gens_needed else 1)

So it looks like sage is trying to find a generator even if it is not requested. According to the bnfisprincipal docs, this makes the computation unnecessarily complex:

   * 1: If set, outputs [e,t] as explained above, otherwise returns
     only e, which is much easier to compute.

Should we change this to use the flag 0 when a generator is not needed?

comment:107 Changed 20 months ago by Vincent Delecroix

Branch: u/arojas/pari-2.13.0-cypari-2.1.2public/30801
Commit: 4f53a36c00471a21fc465fb0d798c9e6f6b3c49e

comment:108 Changed 20 months ago by git

Commit: 58b84fd3966c54575df4b0269ff7f99a9bb2aac0

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

756781eFix tests in homology for different choice of generators
a9fb51cUpdate tests in modules.fg_pid for new Smith normal form
d7cf527Fix tests in torsion quadratic modules for new Smith normal form
b21951eBackport rnfdisc fix
eaa1e38A few more test fixes in schemes.toric
df8c4d0One trivial fix in schemes.elliptic_curves
b11ff20Elliptic curve group generators can be different on each run, mark tests as random
4f53a36Update sandpiles tests for new Smith form
521d582Merge branch 'develop' into pari-2.13.0-arojas
58b84fdfix a call to bnfisprincipal

comment:109 in reply to:  106 Changed 20 months ago by Vincent Delecroix

Replying to arojas:

Replying to vdelecroix:

And this is very bad: pari does not return the generator (the second item in the last output is empty). In the current sage version we have

Do we actually need the generator here? From number_field_ideal.py:

        # Call bnfisprincipal().
        # If gens_needed, use flag=3 which will insist on computing
        # the generator.  Otherwise, use flag=1, where the generator
        # may or may not be computed.
        v = bnf.bnfisprincipal(self.pari_hnf(), 3 if gens_needed else 1)

So it looks like sage is trying to find a generator even if it is not requested. According to the bnfisprincipal docs, this makes the computation unnecessarily complex:

   * 1: If set, outputs [e,t] as explained above, otherwise returns
     only e, which is much easier to compute.

Should we change this to use the flag 0 when a generator is not needed?

Good suggestion. Done in 58b84fd.

comment:110 Changed 20 months ago by Vincent Delecroix

hmm. After this change, the following test in schemes/elliptic_curves/isogeny_small_degree.py hangs

Trying (line 1555):    Psi2(71)  # long time (1 second)
Expecting:
    -2209380711722505179506258739515288584116147237393815266468076436521/71*u^210 + ... - 14790739586438315394567393301990769678157425619440464678252277649/71

comment:111 Changed 20 months ago by git

Commit: 58b84fd3966c54575df4b0269ff7f99a9bb2aac003812531824656bac3aae2b846584f2bc5d43dab

Branch pushed to git repo; I updated commit sha1. New commits:

0381253tweak bnfisprincipal

comment:112 Changed 20 months ago by Vincent Delecroix

Haha: calling the compact form and expanding it works :) It is worth a report to PARI/GP.

comment:113 Changed 20 months ago by git

Commit: 03812531824656bac3aae2b846584f2bc5d43dab7e98037245a9b3fd1cc8fbb5fd62af33f6637a1c

Branch pushed to git repo; I updated commit sha1. New commits:

7e98037clarify comment

comment:114 Changed 20 months ago by Vincent Delecroix

Great, only the Simon 2-descent breaks doctest now. I suggest that we leave it aside and triggers a warning anytime it is called. At the same time I would send a message on sage-devel/sage-nt so that interested people if any can fix it.

comment:115 Changed 20 months ago by Vincent Delecroix

Actually, the changes are clear regressions, but given the specification of simon_two_descent the output is correct in all cases.

comment:116 Changed 20 months ago by git

Commit: 7e98037245a9b3fd1cc8fbb5fd62af33f6637a1c86980a3b67a7e871e11135d65bc648f7b77a961e

Branch pushed to git repo; I updated commit sha1. New commits:

86980a3fix output of simon_two_descent

comment:117 Changed 20 months ago by Vincent Delecroix

Status: newneeds_review

comment:118 Changed 20 months ago by Antonio Rojas

Status: needs_reviewneeds_work

There's still one simon-two-descent related failure in schemes/elliptic_curves/ell_number_field.py - I guess you didn't run --long tests?

File "/usr/lib/python3.9/site-packages/sage/schemes/elliptic_curves/ell_number_field.py", line 270, in sage.schemes.elliptic_curves.ell_number_field.EllipticCurve_number_field.?
Failed example:
    E.simon_two_descent()  # long time (4s on sage.math, 2013)
Expected:
    (3,
     3,
     [(0 : 0 : 1),
      (-1/2*zeta43_0^2 - 1/2*zeta43_0 + 7 : -3/2*zeta43_0^2 - 5/2*zeta43_0 + 18 : 1)...)
Got:
    (3,
     3,
     [(5/8*zeta43_0^2 + 17/8*zeta43_0 - 9/4 : -27/16*zeta43_0^2 - 103/16*zeta43_0 + 39/8 : 1),
      (0 : 0 : 1)])

Also, spkg-configure.m4 needs updating as per comment:76 to reject pari <2.13. Do we want to also add a check for the rnfdisc fix?

comment:119 Changed 20 months ago by git

Commit: 86980a3b67a7e871e11135d65bc648f7b77a961e860a1ea4f719ebac4141d635b4a9331dd96806ab

Branch pushed to git repo; I updated commit sha1. New commits:

860a1eaone long doctest

comment:120 Changed 20 months ago by Vincent Delecroix

I will fix spkg-configure.m4 and add check for rnfdisc.

Last edited 20 months ago by Vincent Delecroix (previous) (diff)

comment:121 Changed 20 months ago by git

Commit: 860a1ea4f719ebac4141d635b4a9331dd96806ab558b5e820c169b616477f8a2e0eac47f07e41b91

Branch pushed to git repo; I updated commit sha1. New commits:

558b5e8rewrite spkg-configure.m4

comment:122 Changed 20 months ago by Vincent Delecroix

Status: needs_workneeds_review

comment:123 Changed 20 months ago by Vincent Delecroix

Description: modified (diff)

comment:124 Changed 19 months ago by Dima Pasechnik

I'm getting

sage -t --random-seed=0 src/sage/rings/number_field/number_field_rel.py
**********************************************************************
File "src/sage/rings/number_field/number_field_rel.py", line 2251, in sage.rings.number_field.number_field_rel.NumberField_relative.relative_discriminant
Failed example:
    K.relative_discriminant() == F.ideal(4*b)
Expected:
    True
Got:
    False

not 100% sure it's related to this ticket, though.

comment:125 Changed 19 months ago by Mauricio Collares

This ticket adds a Pari patch in build/pkgs/pari to fix https://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=2284, which causes the failure above.

comment:126 Changed 19 months ago by Dima Pasechnik

Ah, OK, this explains - I'm trying this with 2.13.1 from Gentoo. So this means it's not detected by spkg-configure,m4, this bug.

comment:127 Changed 19 months ago by Dima Pasechnik

Reviewers: Dima Pasechnik
Status: needs_reviewneeds_work

add a test for the latest patch in spkg-configure

comment:128 Changed 19 months ago by git

Commit: 558b5e820c169b616477f8a2e0eac47f07e41b91f014e79c131285fb6ea74c9b721998ed7bfe8bff

Branch pushed to git repo; I updated commit sha1. New commits:

b0814bdMerge branch 'develop' of https://github.com/sagemath/sage into t/30801/public/30801
f014e79Install pari spkg if test for rnfdisc bug fix fails

comment:129 Changed 19 months ago by Antonio Rojas

Status: needs_workneeds_review

comment:130 Changed 19 months ago by Dima Pasechnik

Status: needs_reviewpositive_review

lgtm.

Francois, by the way, could you add the latest patch (build/pkgs/pari/patches/pari-rnfdisc.patch), not in 2.13.1, to Gentoo's pari?

comment:131 in reply to:  130 Changed 19 months ago by François Bissey

Replying to dimpase:

lgtm.

Francois, by the way, could you add the latest patch (build/pkgs/pari/patches/pari-rnfdisc.patch), not in 2.13.1, to Gentoo's pari?

That would be @mjo's job :) but I guess I can make a PR for it.

comment:132 Changed 19 months ago by Michael Orlitzky

The current patch is missing the changes to the test cases:

https://pari.math.u-bordeaux.fr/cgi-bin/gitweb.cgi?p=pari.git;a=commitdiff;h=3edb98db78dd49bb8b4137b46781a7cd570c2556

Won't that affect SAGE_CHECK=y?

I am also required to complain about copy/pasting a test into spkg-configure.m4 that will reject pari-2.13.1 for having this one particular bug but not any of the other 47 bugs that have been fixed since: https://pari.math.u-bordeaux.fr/cgi-bin/gitweb.cgi?p=pari.git;a=blob;f=CHANGES

The cost/benefit of using the system package jumps when you go from zero custom patches to one. We should allow people to proceed with the upstream version if that's what they have installed.

(Having completed my rant, I will still add this to the Gentoo package)

comment:133 Changed 19 months ago by Dima Pasechnik

2.13 broke a test in Sage, and it was not fixed in 2.13.1, but only later. No changes in Sage needed.

Yes, this points at poor quality of Pari's own testsuite.

you can force Sage to use system version via a configure option.

comment:134 Changed 19 months ago by Dima Pasechnik

good point about SAGE_CHECK.

comment:135 Changed 19 months ago by Dima Pasechnik

Status: positive_reviewneeds_work

comment:136 Changed 19 months ago by Dima Pasechnik

and of course the full patch does not apply cleanly to 2.13.1, grrr...

comment:137 Changed 19 months ago by git

Commit: f014e79c131285fb6ea74c9b721998ed7bfe8bff10a4531721d2700fd717e2b3a1364508ffd971c3

Branch pushed to git repo; I updated commit sha1. New commits:

10a4531Fix pari test suite after the rnfdisc fix

comment:138 Changed 19 months ago by Antonio Rojas

Status: needs_workneeds_review

comment:139 Changed 19 months ago by Mauricio Collares

comment:140 Changed 19 months ago by Mauricio Collares

Ah, nevermind, I see you did remove the new test's output. My bad.

comment:141 Changed 19 months ago by Dima Pasechnik

Status: needs_reviewpositive_review

ok, this works for me.

comment:142 Changed 19 months ago by Volker Braun

On 32-bit:

**********************************************************************
File "src/sage/libs/pari/__init__.py", line 165, in sage.libs.pari
Failed example:
    eta1.sage()
Expected:
    3.6054636014326520859158205642077267748
Got:
    3.605463601432652085915820564
**********************************************************************
File "src/sage/libs/pari/__init__.py", line 168, in sage.libs.pari
Failed example:
    eta1.sage()
Expected:
    3.605463601432652085915820564207726774810268996598024745444380641429820491740
Got:
    3.60546360143265208591582056420772677481026899659802474544
**********************************************************************
1 item had failures:
   2 of  41 in sage.libs.pari
    [40 tests, 2 failures, 0.02 s]
**********************************************************************
File "src/sage/libs/pari/tests.py", line 1764, in sage.libs.pari.tests
Failed example:
    eta1.sage()
Expected:
    3.6054636014326520859158205642077267748
Got:
    3.605463601432652085915820564
**********************************************************************
File "src/sage/libs/pari/tests.py", line 1767, in sage.libs.pari.tests
Failed example:
    eta1.sage()
Expected:
    3.605463601432652085915820564207726774810268996598024745444380641429820491740
Got:
    3.60546360143265208591582056420772677481026899659802474544
**********************************************************************
1 item had failures:
   2 of 869 in sage.libs.pari.tests
    [868 tests, 2 failures, 1.54 s]
**********************************************************************

and

**********************************************************************
File "src/sage/rings/padics/relaxed_template.pxi", line 2640, in sage.rings.padics.relaxed_template.RelaxedElement_random.__reduce__
Failed example:
    a[:30]
Expected:
    ...?343214211432220241412003314311
Got:
    ...?311111122142113024041330312403
**********************************************************************
File "src/sage/rings/padics/relaxed_template.pxi", line 2642, in sage.rings.padics.relaxed_template.RelaxedElement_random.__reduce__
Failed example:
    b[:30]
Expected:
    ...?343214211432220241412003314311
Got:
    ...?311111122142113024041330312403
**********************************************************************
Last edited 19 months ago by Volker Braun (previous) (diff)

comment:143 Changed 19 months ago by Volker Braun

Status: positive_reviewneeds_work

comment:144 Changed 19 months ago by Dima Pasechnik

Cc: Xavier Caruso added

tests in src/sage/rings/padics/relaxed_template.pxi were added by @caruso very recently. Perhaps he can comment whether this is a bug.

the other 32-bit errors are obviously just numerical noise, I'll fix these.

comment:145 Changed 19 months ago by git

Commit: 10a4531721d2700fd717e2b3a1364508ffd971c30bba178a95a4452e33981b966a7a88d1f3f4a92a

Branch pushed to git repo; I updated commit sha1. New commits:

fb5a9a4Merge remote-tracking branch 'trac/develop' into public/30801
0bba178fix some 32 vs 64 bit discrepancies in tests

comment:146 in reply to:  142 Changed 19 months ago by Vincent Delecroix

Replying to vbraun:

On 32-bit:

**********************************************************************
File "src/sage/rings/padics/relaxed_template.pxi", line 2640, in sage.rings.padics.relaxed_template.RelaxedElement_random.__reduce__
Failed example:
    a[:30]
Expected:
    ...?343214211432220241412003314311
Got:
    ...?311111122142113024041330312403
**********************************************************************
File "src/sage/rings/padics/relaxed_template.pxi", line 2642, in sage.rings.padics.relaxed_template.RelaxedElement_random.__reduce__
Failed example:
    b[:30]
Expected:
    ...?343214211432220241412003314311
Got:
    ...?311111122142113024041330312403
**********************************************************************

Printing a random element in a doctest is usually not what is intended to be tested. As explained in the documentation the above code is supposed to test whether a[:30] and b[:30] are equal.

comment:147 Changed 19 months ago by Vincent Delecroix

Moreover, few lines above in the code, the tests involve a # random tag...

comment:148 Changed 19 months ago by Xavier Caruso

Oops sorry, a tag #random was missing; it's actually weird that this test used to pass earlier.

comment:149 Changed 19 months ago by git

Commit: 0bba178a95a4452e33981b966a7a88d1f3f4a92a32b52cc022fbc68463bb0206c8e1fe1e181bec84

Branch pushed to git repo; I updated commit sha1. New commits:

32b52ccadd tag #random

comment:150 in reply to:  148 Changed 19 months ago by Dima Pasechnik

Replying to caruso:

Oops sorry, a tag #random was missing; it's actually weird that this test used to pass earlier.

hmm, so somehow 32-bit vs 64-bit rngs now behave differently?

It passed before on with whatever value you put in, as it's pseudo-random, and as long as you didn't change the rng seed, it should be the same.

comment:151 Changed 19 months ago by Dima Pasechnik

Status: needs_workpositive_review

IMHO the difference in RNG values on different platforms here may be ignored.

comment:152 in reply to:  148 Changed 19 months ago by Vincent Delecroix

Replying to caruso:

Oops sorry, a tag #random was missing; it's actually weird that this test used to pass earlier.

Don't you want to actually compare the values of a[:30] and b[:30]? Adding the # random tag actually makes the test pass in any scenario.

comment:153 Changed 19 months ago by Xavier Caruso

Well, the test a == b below does this checking (precisely, equality is checked at the default halting precision of the parent, which is 40 in the example).

Last edited 19 months ago by Xavier Caruso (previous) (diff)

comment:154 Changed 18 months ago by Antonio Rojas

Status: positive_reviewneeds_work

Many new test failures (and a merge conflict) caused by #26635

sage -t --long --random-seed=0 src/sage/modular/local_comp/local_comp.py  # 7 doctests failed
sage -t --long --random-seed=0 src/sage/modular/local_comp/smoothchar.py  # 11 doctests failed
Last edited 18 months ago by Antonio Rojas (previous) (diff)

comment:155 Changed 18 months ago by git

Commit: 32b52cc022fbc68463bb0206c8e1fe1e181bec84316d62f945e2a301008a67d22c41cbfc61186321

Branch pushed to git repo; I updated commit sha1. New commits:

bf5a59aMerge branch 'develop' of https://github.com/sagemath/sage into t/30801/public/30801
316d62fMerge branch 'public/30801' of git://trac.sagemath.org/sage into t/30801/public/30801

comment:156 Changed 18 months ago by Dima Pasechnik

It's not hard to fix the tests to pass on pari 2.13, but it's much harder to make them also work for 2.11 (do we need ther latter?) Here is the diff for the most affected file (only the 1st test is made version-free)

  • src/sage/modular/local_comp/smoothchar.py

    diff --git a/src/sage/modular/local_comp/smoothchar.py b/src/sage/modular/local_comp/smoothchar.py
    index e54d5ad1d4..92135f7f71 100644
    a b class SmoothCharacterGroupQuadratic(SmoothCharacterGroupGeneric): 
    11311131            sage: from sage.modular.local_comp.smoothchar import SmoothCharacterGroupRamifiedQuadratic
    11321132            sage: G = SmoothCharacterGroupRamifiedQuadratic(3, 1, QQ)
    11331133            sage: s = G.number_field().gen()
    1134             sage: G.discrete_log(4, 3 + 2*s)
    1135             [1, 2, 2, 1]
    1136             sage: gs = G.unit_gens(4); gs[0] * gs[1]^2 * gs[2]^2 * gs[3] - (3 + 2*s) in G.ideal(4)
     1134            sage: dl = G.discrete_log(4, 3 + 2*s)
     1135            sage: gs = G.unit_gens(4); gs[0]^dl[0] * gs[1]^dl[1] * gs[2]^dl[2] * gs[3]^dl[3] - (3 + 2*s) in G.ideal(4)
    11371136            True
    11381137
    11391138        An example with a custom generating set::
    class SmoothCharacterGroupQuadratic(SmoothCharacterGroupGeneric): 
    11911190            sage: from sage.modular.local_comp.smoothchar import SmoothCharacterGroupUnramifiedQuadratic
    11921191            sage: G = SmoothCharacterGroupUnramifiedQuadratic(7,QQ)
    11931192            sage: G.quotient_gens(1)
    1194             [s]
     1193            [2*s - 2]
    11951194            sage: G.quotient_gens(2)
    1196             [23*s - 16]
     1195            [15*s + 21]
    11971196            sage: G.quotient_gens(3)
    1198             [-124*s - 2]
     1197            [-75*s + 33]
    11991198
    12001199        A ramified case::
    12011200
    class SmoothCharacterGroupQuadratic(SmoothCharacterGroupGeneric): 
    12081207
    12091208            sage: G = SmoothCharacterGroupUnramifiedQuadratic(2,QQ)
    12101209            sage: G.quotient_gens(1)
    1211             [s]
     1210            [s + 1]
    12121211            sage: G.quotient_gens(2)
    1213             [-s + 1]
     1212            [-s + 2]
    12141213            sage: G.quotient_gens(3)
    1215             [7*s + 5, -s + 3]
     1214            [-17*s - 14, 3*s - 2]
    12161215        """
    12171216
    12181217        # silly special case
    class SmoothCharacterGroupQuadratic(SmoothCharacterGroupGeneric): 
    13271326            sage: G.extend_character(3, chi, [1])
    13281327            Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 0, mapping 5 |--> 7
    13291328            sage: K.<z> = CyclotomicField(6); G.base_extend(K).extend_character(1, chi, [z])
    1330             Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> z, 5 |--> 7
     1329            Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -z + 1, 5 |--> 7
    13311330
    13321331        We extend the nontrivial quadratic character::
    13331332
    13341333            sage: chi = SmoothCharacterGroupQp(5, QQ).character(1, [-1, 7])
    13351334            sage: K.<z> = CyclotomicField(24); G.base_extend(K).extend_character(1, chi, [z^6])
    1336             Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> z^6, 5 |--> 7
     1335            Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -z^6, 5 |--> 7
    13371336
    13381337        Extensions of higher level::
    13391338
    13401339            sage: K.<z> = CyclotomicField(20); rho = G.base_extend(K).extend_character(2, chi, [z]); rho
    1341             Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 2, mapping 11*s - 10 |--> -z^5, 6 |--> 1, 5*s + 1 |--> z^4, 5 |--> 7
     1340            Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 2, mapping 11*s - 10 |--> z^5, 6 |--> 1, 5*s + 1 |--> z^4, 5 |--> 7
    13421341            sage: rho(3)
    13431342            -1
    13441343

comment:157 Changed 18 months ago by Dima Pasechnik

Cc: David Loeffler added

There are also failures in src/sage/modular/local_comp/local_comp.py, the 1st two trivial (different ordering of a list), but the following two look rather suspect to me:

File "src/sage/modular/local_comp/local_comp.py", line 656, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters
Failed example:
    Pi.characters()
Expected:
    [
    Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> d, 5 |--> 5,
    Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -d - 1/3*j0^3, 5 |--> 5
    ]
Got:
    [
    Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5,
    Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5
    ]
**********************************************************************
File "src/sage/modular/local_comp/local_comp.py", line 661, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters
Failed example:
    Pi.characters()[0].base_ring()
Expected:
    Number Field in d with defining polynomial x^2 + 1/3*j0^3*x - 1/3*j0^2 over its base field
Got:
    Number Field in d with defining polynomial x^2 - j0*x + 1/3*j0^2 over its base field

David, can you have a look?

comment:158 Changed 18 months ago by David Loeffler

Looks harmless to me. I gather from the discussions above that the default generators for multiplicative groups of non-prime finite fields have been changed, and that would inevitably lead to changes in the output of these doctests (without affecting the mathematical validity of the calculations).

comment:159 in reply to:  158 Changed 18 months ago by Dima Pasechnik

Replying to davidloeffler:

Looks harmless to me. I gather from the discussions above that the default generators for multiplicative groups of non-prime finite fields have been changed, and that would inevitably lead to changes in the output of these doctests (without affecting the mathematical validity of the calculations).

right under these two tests one has

        .. warning::

            The above output isn't actually the same as in Example 2 of
            [LW2012]_, due to an error in the published paper (correction
            pending) -- the published paper has the inverses of the above
            characters.

I didn't look at the paper cited there - but presumably you know what's going on. And I imagine it won't be "the inverses of the above characters" any more.

comment:160 Changed 18 months ago by David Loeffler

The characters returned by the doctest are the same characters as they were before -- just notated differently. So the warning is no more or less problematic than it was before.

comment:161 Changed 18 months ago by Dima Pasechnik

OK, thanks, we'll be able to proceed then.

comment:162 Changed 18 months ago by Dima Pasechnik

as we bump the Pari version to 2.13+, we should remove tests for bugs fixed earlier.

comment:163 Changed 18 months ago by git

Commit: 316d62f945e2a301008a67d22c41cbfc611863214a9c3b68b2f82681b15b85987a7bee432b70b1e1

Branch pushed to git repo; I updated commit sha1. New commits:

da40045doctest fixes for #26635 on Pari 2.13
73591c0do not test for bugs fixed in pari 2.13+
4a9c3b6remove space between 2 and >> in script

comment:164 Changed 18 months ago by Dima Pasechnik

Status: needs_workneeds_review

all fixed, please review

comment:165 Changed 18 months ago by David Loeffler

In case anybody cares: I did some digging about the doctests in the local-components code. It turns out this is not actually because of the changes to PARI's bnfisunit etc; rather, it is caused by the change to Smith normal form algorithms, which are used to find a generator of some quotient of abelian groups.

The test case in local_comp.py line 656, from Dima's comment above, depends on the following computation:

sage: A = ZZ^2                                                                  
sage: R = [A([6, 0]), A([0,1])]                                               
sage: [x.lift() for x in (A/R).gens()]                                          

The output in Sage 9.4.beta1 is [(1, 0)]. With the new Pari SPKG from this ticket, the output changes to [(-1, 0)], which is equally valid. Using this, the code gives output which is mathematically equivalent to the previous output, but using a different choice of generator of the coefficient field.

(One could rewrite the doctest in a more robust way using embeddings into QQbar; but that would make the doctest less readable as an example so I don't advocate doing that, unless it becomes a recurring problem.)

comment:166 Changed 18 months ago by Antonio Rojas

I'm still seeing 3 failures

**********************************************************************
File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/local_comp.py", line 669, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters
Failed example:
    LocalComponent(f, 3).characters()  # long time (12s on sage.math, 2012)
Expected:
    [
    Character of unramified extension Q_3(s)* (s^2 + 2*s + 2 = 0), of level 2, mapping -2*s |--> 2*d + j0, 4 |--> 1, 3*s + 1 |--> j0*d + 1, 3 |--> 1,
    Character of unramified extension Q_3(s)* (s^2 + 2*s + 2 = 0), of level 2, mapping -2*s |--> -2*d - j0, 4 |--> 1, 3*s + 1 |--> -j0*d - 2, 3 |--> 1
    ]
Got:
    [
    Character of unramified extension Q_3(s)* (s^2 + 2*s + 2 = 0), of level 2, mapping -2*s |--> -2*d + j0, 4 |--> 1, 3*s + 1 |--> -j0*d + 1, 3 |--> 1,
    Character of unramified extension Q_3(s)* (s^2 + 2*s + 2 = 0), of level 2, mapping -2*s |--> 2*d - j0, 4 |--> 1, 3*s + 1 |--> j0*d - 2, 3 |--> 1
    ]
**********************************************************************
File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/local_comp.py", line 698, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters
Failed example:
    Newforms(DirichletGroup(64, QQ).1, 2, names='a')[0].local_component(2).characters() # long time, random
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters[14]>", line 1, in <module>
        Newforms(DirichletGroup(Integer(64), QQ).gen(1), Integer(2), names='a')[Integer(0)].local_component(Integer(2)).characters() # long time, random
      File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/local_comp.py", line 757, in characters
        pairA = [ [g0vals[0], gvals[0]], [g0vals[1], gvals[1]] ]
    IndexError: list index out of range
**********************************************************************
File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/local_comp.py", line 703, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters
Failed example:
    Newform('243a',names='a').local_component(3).characters() # long time
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 714, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.9/site-packages/sage/doctest/forker.py", line 1133, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters[15]>", line 1, in <module>
        Newform('243a',names='a').local_component(Integer(3)).characters() # long time
      File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/local_comp.py", line 893, in characters
        ti = (~T.rho(x.matrix().list())).trace() * p**ZZ((k-2+self.twist_factor())/2)
      File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/type_space.py", line 674, in rho
        g = [ZZ(_) for _ in g]
      File "/usr/lib/python3.9/site-packages/sage/modular/local_comp/type_space.py", line 674, in <listcomp>
        g = [ZZ(_) for _ in g]
      File "sage/structure/parent.pyx", line 898, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9341)
        return mor._call_(x)
      File "sage/rings/rational.pyx", line 4164, in sage.rings.rational.Q_to_Z._call_ (build/cythonized/sage/rings/rational.cpp:32503)
        raise TypeError("no conversion of this rational to integer")
    TypeError: no conversion of this rational to integer
**********************************************************************

comment:167 Changed 18 months ago by Dima Pasechnik

Status: needs_reviewneeds_work

Oops, forgot long tests. The 1st failure is trivial to fix, but the other 2 look like genuine bugs, which David hopefully knows how to fix.

Specifically, in the 1st case, line 698, I debugged a bit, and got the following:

            gvals = [x[0] for x in theta_poly.roots(G.base_ring())]
            print("gvals=",gvals, "\n", theta_poly, "\n", theta_poly.roots(G.base_ring()), "\n")
            print(gvals[0]**2,"\n")
            if len(gs) == 1:
                # This is always the case if p != 2
                chi1, chi2 = [G.extend_character(n, self.central_character(), [x]) for x in gvals]
            else:
                # 2-adic cases, conductor >= 64. Here life is complicated
                # because the quotient (O_K* / p^n)^* / (image of Z_2^*) is not
                # cyclic.
                g0 = gs[0]
                try:
                    G._reduce_Qp(1, g0)
                    raise ZeroDivisionError
                except ValueError:
                    pass

                tr = (~T.rho(g0.matrix().list())).trace()
                X = polygen(G.base_ring())
                theta_poly = X**2 - (-1)**n*tr*X + self.central_character()(g0.norm())
                verbose("theta_poly for %s is %s" % (g0, theta_poly), level=1)
                if theta_poly.is_irreducible():
                    F = theta_poly.base_ring().extension(theta_poly, "e")
                    G = G.base_extend(F)
                g0vals = [y[0] for y in theta_poly.roots(G.base_ring())]
                print(g0vals, " ", gvals, "\n", theta_poly, "\n", theta_poly.roots(G.base_ring()), "\n")
                pairA = [ [g0vals[0], gvals[0]], [g0vals[1], gvals[1]] ] # - ERROR
                pairB = [ [g0vals[0], gvals[1]], [g0vals[1], gvals[0]] ]

which I ran as

sage: nn=Newforms(DirichletGroup(64, QQ).1, 2, names='a')[0]; nn                                                                                                                                                                                                                              
q - a0*q^3 + O(q^6)
sage: s=nn.local_component(2)                                                                                                                                                                                                                                                                 
sage: s.characters()                                                                                                                                                                                                                                                                          
gvals= [1/2*a0] 
 x^2 - a0*x - 1 
 [(1/2*a0, 2)] 

-1 

[1/2*a0, -1/2*a0]   [1/2*a0] 
 x^2 + 1 
 [(1/2*a0, 1), (-1/2*a0, 1)] 

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-3-ec8072600e6c> in <module>
----> 1 s.characters()

~/sage/local/lib/python3.9/site-packages/sage/modular/local_comp/local_comp.py in characters(self)
    755                 g0vals = [y[0] for y in theta_poly.roots(G.base_ring())]
    756                 print(g0vals, " ", gvals, "\n", theta_poly, "\n", theta_poly.roots(G.base_ring()), "\n")
--> 757                 pairA = [ [g0vals[0], gvals[0]], [g0vals[1], gvals[1]] ]
    758                 pairB = [ [g0vals[0], gvals[1]], [g0vals[1], gvals[0]] ]
    759 

IndexError: list index out of range

That is, theta_poly has a double root, something that is not guarded against in any way. David, your turn. If this is not a Pari bug I'd rather have this dealt with on a follow-up ticket.

comment:168 Changed 18 months ago by David Loeffler

I'll do some digging and see whether this is Pari's fault, or a bug in the local components code. Quite likely it's the latter -- the ZeroDivisionError lines look rather fishy.

comment:169 Changed 18 months ago by git

Commit: 4a9c3b68b2f82681b15b85987a7bee432b70b1e1d50e7e95475c7ac5f00da65816ddfedc4d7d5b5a

Branch pushed to git repo; I updated commit sha1. New commits:

d50e7e9Fix local-component bug revealed by pari upgrade

comment:170 Changed 18 months ago by git

Commit: d50e7e95475c7ac5f00da65816ddfedc4d7d5b5abb0245abf74aba706583c94724d7a3e5b06e2b63

Branch pushed to git repo; I updated commit sha1. New commits:

bb0245afix unhandled case in local_comp

comment:171 Changed 18 months ago by David Loeffler

Here's a small fix which sorts out the case when theta_poly has a repeated root.

Unfortunately, in the process of fixing this I spotted another bug elsewhere. Consider the code

K.<a> = NumberField(x^2 + x + 1)
list(K.ideal(8).invertible_residues())[:5]
list(K.ideal(16).invertible_residues())[:5]

In vanilla 9.4.beta1, both commands return lists of algebraic integers. With the new pari spkg, the first one returns a list of non-integral elements (contradicting the docstring of invertible_residues which says that the results must be in the ring of integers); and the second one crashes with an "Unhandled SIGABRT"!

comment:172 Changed 18 months ago by Dima Pasechnik

I don't get a crash:

sage: K.<a> = NumberField(x^2 + x + 1) 
....: list(K.ideal(8).invertible_residues())[:5] 
....: list(K.ideal(16).invertible_residues())[:5]                                                                                                                                                           
[1, 1/9*a + 2/9, 1/27*a + 1/27, 2/243*a + 1/243, 1/729*a]
[1,
 5/91*a + 11/91,
 85/8281*a + 96/8281,
 990/753571*a + 631/753571,
 9095/68574961*a + 1991/68574961]

but indeed it is a bug, again, not sure in Pari or in Sage. Can you repeat this computations in "pure" Pari/GP to check?

comment:173 Changed 18 months ago by Dima Pasechnik

As to the crash, it's hard to say what's wrong, maybe you build Pari/GP in an unusual way, such as using MPIR (one should not)?

Last edited 18 months ago by Dima Pasechnik (previous) (diff)

comment:174 Changed 18 months ago by David Loeffler

I didn't do anything "unusual" as far as I'm aware. I'm not using MPIR but the configure script is picking up my system-wide GMP library. I'm trying a rebuild forcing sage to build its own GMP (but that requires rebuilding about 50% of Sage so it will take a while).

(By the way: if MPIR is deprecated, why is it supported by the configure script?)

As for your previous comment: I don't think this command has an exact equivalent in "pure" Pari. It looks like the problem is (once again) caused by changes to Smith normal form algorithms.

comment:175 Changed 18 months ago by David Loeffler

As I suspected, the change to the output of "invertible_residues" is a consequence of the fact that

sage: diagonal_matrix([12,2,2]).smith_form()
# with old Pari
(
[ 2  0  0]  [0 0 1]  [0 0 1]
[ 0  2  0]  [0 1 0]  [0 1 0]
[ 0  0 12], [1 0 0], [1 0 0]
)
# with new Pari
(
[ 2  0  0]  [ 1  0  1]  [ 0  0 -1]
[ 0  2  0]  [ 0  1  0]  [ 0  1  0]
[ 0  0 12], [-1  0  0], [ 1  0  6]
)

This doesn't explain the crash though; still waiting for the build to complete.

comment:176 Changed 18 months ago by Dima Pasechnik

I now tested for crash on macOS 11.4 and on Fedora 32 Linux, no crash. Will do one more different system, just in case.

comment:177 Changed 18 months ago by Dima Pasechnik

with your latest patches I still get reported in comment:166

File "src/sage/modular/local_comp/local_comp.py", line 704, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters
Failed example:
    Newform('243a',names='a').local_component(3).characters() # long time
...

comment:178 Changed 18 months ago by Dima Pasechnik

So the Smith form stays the same, but the pivots it uses are now different, right? With just positively reviewed Singular update (not merged yet) and the branch on this ticket, I see

File "src/sage/modules/fg_pid/fgp_module.py", line 1274, in sage.modules.fg_pid.fgp_module.FGP_Module_class.coordinate_vector
Failed example:
    Q.coordinate_vector(Q.0 - Q.1)
Expected:
    (1, -1)
Got:
    (1, 47)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1294, in sage.modules.fg_pid.fgp_module.FGP_Module_class.coordinate_vector
Failed example:
    Q.coordinate_vector(x)
Expected:
    (0, -1)
Got:
    (0, 11)

Probably the same story?

comment:179 Changed 18 months ago by Dima Pasechnik

also on Gentoo Linux I don't get a crash as reported in comment:171 - so I am inclined to see it as a one-time irreproducible non-bug.

comment:180 Changed 18 months ago by David Loeffler

The problem in comment:177 is a knock-on effect of the invertible_residues bug. This can be fixed by changing the line

new_basis = [prod([g[j]**V[i, j] for j in range(n)]) for i in range(n)]

to

new_basis = [prod([g[j]**(V[i, j] % invs[j]) for j in range(n)]) for i in range(n)]

in the code for invertible_residues_mod, line 2777 of number_field_ideal.py. However, I cannot test this myself, since every time I run the code (with or without this fix), it crashes. I've spent most of the afternoon repeatedly recompiling Sage with different config options trying to get rid of this "one-time bug" and it still crashes every single time.

(More specifically: if I let Sage's "configure" pick up the system gmp libraries, then it builds, but I see the crash. If I tell Sage to build its own gmp, then it fails to compile giac. This is in Ubuntu 20 LTS, which is hardly an obscure operating system, surely?)

Last edited 18 months ago by David Loeffler (previous) (diff)

comment:181 Changed 18 months ago by Dima Pasechnik

oh dear. Is the system up to date? What is the version of gcc? Are you a lucky user of tigerlake CPU? (If yes, please see https://groups.google.com/g/sage-devel/c/XHzqSPyuYec/m/Iv5v5g6OCgAJ - update to gcc 10)

comment:182 Changed 18 months ago by Dima Pasechnik

in any event, please post the top-level config.log and more info on the crash(es), a backtrace, preferably. Does it happen in Pari?

comment:183 Changed 18 months ago by David Loeffler

It's not a tigerlake cpu (not that new). GCC version is 9.3.0. The operating system is Ubuntu 20.04 LTS, currently the most recent Ubuntu long-term-support version.

Here's the config.log file:

https://pastebin.com/3Kc7RNP0

Here's the traceback:

https://pastebin.com/bcW8GHVg

comment:184 Changed 18 months ago by Dima Pasechnik

You are apparently mixing two or more GMP versions, one in /usr/local, as seen in traceback

#13 0x00007f5ab24cd10b in __gmpq_div () from /usr/local/lib/libgmp.so.23

another in /usr/, as seen in config.log (//usr/include/x86_64-linux-gnu/gmp.h)

I'd suggest getting rid of whatever is in /usr/local, and stick to the system version. (And perhaps do make distclean, to make sure it's all back to normal)

comment:185 Changed 18 months ago by Dima Pasechnik

I can confirm that the fix in comment:180 does the job, great! I'll add it and the rest of fixes, should be ready to go then, hopefully.

comment:186 Changed 18 months ago by git

Commit: bb0245abf74aba706583c94724d7a3e5b06e2b63c78b1470fccd915e2fa93f95f2fefba6220fb1f7

Branch pushed to git repo; I updated commit sha1. New commits:

c78b147fix pari 2.13-related issue of different Smith form

comment:187 Changed 18 months ago by Dima Pasechnik

Status: needs_workneeds_review

back to needs review!

comment:188 Changed 18 months ago by Christopher D. Long

Confirming this branch eliminates the error I was seeing in PARI/GP's qfminim(), due to a bug in PARI/GP that's since been corrected in that libraries source code. My elliptic curve code is now running properly again.


New commits:

c78b147fix pari 2.13-related issue of different Smith form

comment:189 Changed 18 months ago by David Loeffler

Status: needs_reviewpositive_review

This had a positive review before the issues with local components came up, and I'm confident those are fixed, so I think we can reinstate the positive review.

comment:190 Changed 18 months ago by Dima Pasechnik

Reviewers: Dima PasechnikDima Pasechnik, David Loeffler

comment:191 Changed 18 months ago by Volker Braun

Branch: public/30801c78b1470fccd915e2fa93f95f2fefba6220fb1f7
Resolution: fixed
Status: positive_reviewclosed

comment:192 Changed 18 months ago by Frédéric Chapoton

Commit: c78b1470fccd915e2fa93f95f2fefba6220fb1f7

With 9.4.b3 and pari 2.13, my patchbot petitbonum has now two errors in src/sage/modules/fg_pid/fgp_module.py

https://patchbot.sagemath.org/ticket/0/

comment:193 in reply to:  192 Changed 18 months ago by Dima Pasechnik

Replying to chapoton:

With 9.4.b3 and pari 2.13, my patchbot petitbonum has now two errors in src/sage/modules/fg_pid/fgp_module.py

https://patchbot.sagemath.org/ticket/0/

is it the same as in comment:178 ? I tried reproducing this, but failed.

comment:195 Changed 18 months ago by Dima Pasechnik

Cc: gh-kliem added

Right, I see, it actually depends upon the random seed. E.g. on Fedora 32 with gcc 10.3 and non-0 seed I get

$ ./sage -t --long --random-seed=423333338768755345354 src/sage/modules/fg_pid/fgp_module.py
Running doctests with ID 2021-06-23-14-11-38-7a14ecbb.
Git branch: HEAD
Using --optional=build,dochtml,fedora,gap_packages,libsemigroups,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 53.3 --random-seed=423333338768755345354 src/sage/modules/fg_pid/fgp_module.py
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1274, in sage.modules.fg_pid.fgp_module.FGP_Module_class.coordinate_vector
Failed example:
    Q.coordinate_vector(Q.0 - Q.1)
Expected:
    (1, -1)
Got:
    (1, 47)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1294, in sage.modules.fg_pid.fgp_module.FGP_Module_class.coordinate_vector
Failed example:
    Q.coordinate_vector(x)
Expected:
    (0, -1)
Got:
    (0, 11)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1701, in sage.modules.fg_pid.fgp_module.FGP_Module_class.random_element
Failed example:
    Q.random_element()
Expected:
    (1, 5)
Got:
    (3, 8)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1923, in sage.modules.fg_pid.fgp_module.random_fgp_module
Failed example:
    fgp.random_fgp_module(4)
Expected:
    Finitely generated module V/W over Integer Ring with invariants (4)
Got:
    Finitely generated module V/W over Integer Ring with invariants (0)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1951, in sage.modules.fg_pid.fgp_module.random_fgp_morphism_0
Failed example:
    fgp.random_fgp_morphism_0(4)
Expected:
    Morphism from module over Integer Ring with invariants (4,) to module with invariants (4,) that sends the generators to [(0)]
Got:
    Morphism from module over Integer Ring with invariants (0,) to module with invariants (0,) that sends the generators to [(1)]
**********************************************************************
4 items had failures:
   2 of  23 in sage.modules.fg_pid.fgp_module.FGP_Module_class.coordinate_vector
   1 of   4 in sage.modules.fg_pid.fgp_module.FGP_Module_class.random_element
   1 of   3 in sage.modules.fg_pid.fgp_module.random_fgp_module
   1 of   3 in sage.modules.fg_pid.fgp_module.random_fgp_morphism_0
    [397 tests, 5 failures, 1.93 s]
----------------------------------------------------------------------
sage -t --long --warn-long 53.3 --random-seed=423333338768755345354 src/sage/modules/fg_pid/fgp_module.py  # 5 doctests failed
----------------------------------------------------------------------
Total time for all tests: 2.0 seconds

on the same machine, but with Sage built using clang 10, with 0 random seed these tests pass. With random seed as above I get

$ ./sage -t --long --random-seed=423333338768755345354 src/sage/modules/fg_pid/fgp_module.py
Running doctests with ID 2021-06-23-14-10-28-0b9d083c.
Git branch: HEAD
Using --optional=build,dochtml,fedora,gap_packages,libsemigroups,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 53.3 --random-seed=423333338768755345354 src/sage/modules/fg_pid/fgp_module.py
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1701, in sage.modules.fg_pid.fgp_module.FGP_Module_class.random_element
Failed example:
    Q.random_element()
Expected:
    (1, 5)
Got:
    (3, 8)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1923, in sage.modules.fg_pid.fgp_module.random_fgp_module
Failed example:
    fgp.random_fgp_module(4)
Expected:
    Finitely generated module V/W over Integer Ring with invariants (4)
Got:
    Finitely generated module V/W over Integer Ring with invariants (0)
**********************************************************************
File "src/sage/modules/fg_pid/fgp_module.py", line 1951, in sage.modules.fg_pid.fgp_module.random_fgp_morphism_0
Failed example:
    fgp.random_fgp_morphism_0(4)
Expected:
    Morphism from module over Integer Ring with invariants (4,) to module with invariants (4,) that sends the generators to [(0)]
Got:
    Morphism from module over Integer Ring with invariants (0,) to module with invariants (0,) that sends the generators to [(1)]
**********************************************************************
3 items had failures:
   1 of   4 in sage.modules.fg_pid.fgp_module.FGP_Module_class.random_element
   1 of   3 in sage.modules.fg_pid.fgp_module.random_fgp_module
   1 of   3 in sage.modules.fg_pid.fgp_module.random_fgp_morphism_0
    [397 tests, 3 failures, 2.21 s]
----------------------------------------------------------------------
sage -t --long --warn-long 53.3 --random-seed=423333338768755345354 src/sage/modules/fg_pid/fgp_module.py  # 3 doctests failed
----------------------------------------------------------------------
Total time for all tests: 2.3 seconds

I suppose this should become part of #29935

comment:196 Changed 18 months ago by gh-kliem

I believe this will be fixed with #29978.

comment:197 Changed 18 months ago by Frédéric Chapoton

No, #29978 does not fix the problem.

comment:198 Changed 18 months ago by gh-kliem

I see. I got confused.

So there is actually 2 failing doctests in lines 1274 and lines 1294 that should be fixed.

comment:199 in reply to:  198 Changed 18 months ago by Dima Pasechnik

Replying to gh-kliem:

I see. I got confused.

So there is actually 2 failing doctests in lines 1274 and lines 1294 that should be fixed.

#29978 fixes the clang setup from comment:195 fully, but the gcc (gcc version 10.3.1 20210422 (Red Hat 10.3.1-1)) setup still exhibits 2 failures (irrespectively of the random seed).

Maybe it's a gcc 10 bug? Or we have randomness in various Sage packages not fully under control, otherwise this discrepancy looks strange.

Last edited 18 months ago by Dima Pasechnik (previous) (diff)

comment:200 Changed 18 months ago by gh-kliem

I think the fix is quite trivial. Apparently with reduce=False things really are random. There is already one test in line 1`288 that is marked as random because of this. So yes, this could be done in a follow up of #29978.

comment:201 Changed 18 months ago by gh-kliem

See #32048 for a possible fix.

comment:202 Changed 17 months ago by gh-kliem

The problem above and another error not reported before, are responsible for "my" patchbot panke to be dead for now, I just found out.

sage -t --long --random-seed=0 src/sage/modular/local_comp/local_comp.py                                                                                                                                                                                                       
**********************************************************************                                                                                                                                                                                                         
File "src/sage/modular/local_comp/local_comp.py", line 653, in sage.modular.local_comp.local_comp.PrimitiveSupercuspidal.characters                                                                                                                                            
Failed example:                                                                                                                                                                                                                                                                
    set(Pi.characters())                                                                                                                                                                                                                                                       
Expected:                                                                                                                                                                                                                                                                      
    {Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5,                                                                                                                                                
     Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5}                                                                                                                                                          
Got:                                                                                                                                                                                                                                                                           
    {Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5,                                                                                                                                                          
     Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5}  

I don't know why you introduced set in a doctest. It's not stable.

In #29977 I try to fix it by just outputing Pi.characters(). I don't know if that is stable either.

comment:203 Changed 17 months ago by Dima Pasechnik

the Python's insane treatment of sets makes me mentally unstable, this is for sure. Is there any f*cking way in this cr*p of a language to output anything at all in a unique order?

comment:204 Changed 17 months ago by Dima Pasechnik

sorry for screaming...

comment:205 Changed 17 months ago by Samuel Lelièvre

Perhaps sorted could sort this out?

sage: chars = sorted(Pi.characters())
sage: chars

How about using sorted for all occurrences of Pi.characters() in that file?

comment:206 Changed 17 months ago by gh-kliem

Doctests try to sort dictionaries and sets uniquely. However it fails here.

This is really strange.

It apparently depends on the compiler(?):

sage: from IPython.lib.pretty import pprint 
sage: f = Newforms(GammaH(25, [6]), 3, names='j')[0]
sage: Pi = LocalComponent(f, 5)                                                                                                       
sage: pprint(set(Pi.characters()))                                                                                                    
{Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5,
 Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5}

on Debian buster with pari 2.13.1 installed and gcc 8.3.

sage: from IPython.lib.pretty import pprint 
sage: f = Newforms(GammaH(25, [6]), 3, names='j')[0]
sage: Pi = LocalComponent(f, 5)                                                                                                       
sage: pprint(set(Pi.characters()))  
{Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5,
 Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5}

on Ubuntu focal with pari 2.13.1 installed and gcc 9.3.0.

The problems is that the output is not sortable:

sage: f = Newforms(GammaH(25, [6]), 3, names='j')[0]
sage: Pi = LocalComponent(f, 5)        
sage: a = list(Pi.characters())
sage: b = [a[1], a[0]]
sage: sorted(a) == sorted(b)
False
sage: a                                                                                                                             
[Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5,
 Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5]
sage: b                                                                                                                               
[Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> -1/3*j0^2*d, 5 |--> 5,
 Character of unramified extension Q_5(s)* (s^2 + 4*s + 2 = 0), of level 1, mapping s |--> 1/3*j0^2*d - 1/3*j0^3, 5 |--> 5]

Hence ipythons pretty printer fails https://github.com/ipython/ipython/blob/master/IPython/lib/pretty.py.

comment:207 Changed 17 months ago by Samuel Lelièvre

Let's move the discussion to #29977.

Note: See TracTickets for help on using tickets.