Upgrade PARI to git master

[jdemeyer]

Alternative: #16939

Upstream: ​http://boxen.math.washington.edu/home/jdemeyer/spkg/pari-2.8-1315-g031c9d6.tar.gz

[jdemeyer]

New commits:

​006f9b9

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​927e70c	Remove genus2reduction
​3ffdf18	Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​11946d6	Fix documentation
​c92fba3	#17015: upgrade eclib to 20140920
​ec61d5e	#17105 upgrade eclib to 20140921
​0a4f6ec	Merge branches 'ticket/15808' and 'ticket/17015' into HEAD
​8bbd25d	Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​0ad0a2a

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​75d40e2

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​66ba6db

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​a70061e

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​f7db9fd

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​758c15d

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b57b237

Upgrade to PARI git master

[jdemeyer]

This patch changes a French book doctest.

[jdemeyer]

Removed patch

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​c4f948d	Merge with 6.4.beta4
​1068fef	trac #15808 correct name for _reduce + a few pep8 changes
​f35c944	Fix "away from 2" check
​e882752	Upgrade to PARI git master

[zimmerma]

I removed my name in the cc list, since it makes no sense to try to maintain coherence of the french doctests with our book: there are now too many changes which break the examples from the book.

Paul

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​e28c799

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​84b3449

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​105255a

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​daaea9f

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​cbc0c5a

Upgrade to PARI git master

[jdemeyer]

I have removed the controversial patches, it now needs review again.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​660eb66

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​db7dbd6

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​ff7c17f

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​5e8d984

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b2597a7

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​a13b116	Fix PI declaration in complex_plot.pyx
​bb12f77	Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​7528799

Upgrade to PARI git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​0602271	Use cb_pari_err_handle() to handle PARI errors
​fbd37a0	Merge commit 'a13b11636e7cb6b0f5c066b6d7784361fa68091d' into HEAD
​5cd1f00	Upgrade PARI to git master

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​0fa866c	Trac #17588: Remove darwin specific install instruction that are counterproductive.
​b8004f7	Upgrade PARI to git master

[vdelecroix]

Do we change the package name to sage-pari as it was discussed on sage-devel?

[fbissey]

After my long rant (and mostly getting stuff out of my chest) I no longer think it is necessary. Let me explain:

• it is not just giving a name to the tarball. You would want a distinct name for the library and the executable.
• for sanity reason you don't want to carry 2 versions of pari, one for the package and one for the library. So you get to change all the packages that link to pari at least.
• Last but not least. There are not that many packages using pari that are not in sage and a lot of them depend on ancient pari (2.3). So unless someone pulls a list of software using pari from under their beds i think the point on maintaining the stack is moot.

I did some homework on the last point in Gentoo I am not sure what it would hold for debian but I am willing to bet it will be about the same.

[jdemeyer]

By the way: if somebody actually feels like reviewing this, let me know. Then I will update the branch.

[fbissey]

I'll need a little bit of time to get over all the dependency but once I get there I may review this. One question however: is the new patch public_memory_functions.patch something waiting for inclusion upstream? As of this morning it would probably still apply (I now have a live ebuild for pari and this morning master is not *broken*, not like 2 days ago).

[jdemeyer]

Replying to fbissey:

One question however: is the new patch public_memory_functions.patch something waiting for inclusion upstream?

Consider this a "transient" patch. PARI upstream has changed their stack-handling in a non-backwards-compatible way. This patch public_memory_functions.patch is what's needed to use the current Sage hooks for dealing with the PARI stack. A follow-up ticket could use the new PARI stack mechanism.

[jdemeyer]

Replying to fbissey:

I'll need a little bit of time to get over all the dependency

What does "get over all the dependency" mean???

[fbissey]

To be honest I thought there would be more package depending on pari and that more would be impacted. But apart from sage itself that's just lcalc. If we are looking at optional package there may be MaCaulay2.

[jdemeyer]

eclib also depends on pari.

[fbissey]

Replying to jdemeyer:

eclib also depends on pari.

And I noticed that the extent of your fix is adding the standard patching code in spkg-install. I can do some more checking with it but I can believe eclib will be unaffected since John already did a fix for pari 2.8 ​https://github.com/JohnCremona/eclib/commit/652337883464b277f52901c7bcaf9c71001e0c3c

[fbissey]

And if we want to be complete sympow depends on pari but only through gp.

[jdemeyer]

Replying to fbissey:

Replying to jdemeyer:

eclib also depends on pari.

And I noticed that the extent of your fix is adding the standard patching code in spkg-install. I can do some more checking with it but I can believe eclib will be unaffected since John already did a fix for pari 2.8

Yes, I know. In an earlier version of this branch, I had to explicitly add a patch and the "patching loop" in spkg-install. I left the patching loop since it makes sense anyway to have it.

I am currently in the process of upgrading this branch to the latest PARI.

[fbissey]

Can you put an spkg-src this time? "make snapshot" is not enough to reproduce a given tarball, we should know the precise commit at which you snapshot.

[jdemeyer]

Replying to fbissey:

"make snapshot" is not enough to reproduce a given tarball, we should know the precise commit at which you snapshot.

The commit is in build/pkgs/pari/package-version.txt, is that not sufficient?

[fbissey]

Replying to jdemeyer:

Replying to fbissey:

"make snapshot" is not enough to reproduce a given tarball, we should know the precise commit at which you snapshot.

The commit is in build/pkgs/pari/package-version.txt, is that not sufficient?

I guess it is. I am too lazy ;)

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​ade4f75

Upgrade PARI to git master

[jdemeyer]

Replying to fbissey:

And if we want to be complete sympow depends on pari but only through gp.

I didn't know that, but you're right.

[fbissey]

OK now I am starting tests. A technical question about upstream's intention. The current soname for pari 2.7 is libpari-gmp.so.4 the one from git master is libpari-gmp-2.8.so.0 do you know if it is temporary until 2.8 is released (and switch to something like libpari-gmp.so.5) or it is their plan to stick with that soname?

[fbissey]

I think I found the answer to my own question in config directory. They have a different scheme for even numbered release on purpose. Which makes life hard for automated rebuilding in gentoo. I'll have to force things there.

[fbissey]

OK passes a few sanity checks so this is ready for bot testing.

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

​b41dd91

Fix 32-bit doctest

[vbraun]

I also get on some buildbots:

sage -t --long src/sage/modular/modform_hecketriangle/hecke_triangle_group_element.py
**********************************************************************
File "src/sage/modular/modform_hecketriangle/hecke_triangle_group_element.py", line 38, in sage.modular.modform_hecketriangle.hecke_triangle_group_element.coerce_AA
Failed example:
    AA(p)._exact_field()
Expected:
    Number Field in a with defining polynomial y^16 + ... with a in ...
Got:
    Number Field in a with defining polynomial y^16 - 1312*y^14 + 864612*y^12 - 2512720*y^11 - 50780668*y^10 + 360028808*y^9 - 48779642409*y^8 + 290398268512*y^7 + 15186503722504*y^6 - 566858046818416*y^5 + 27431234189939174*y^4 - 226711842252026288*y^3 + 7395502353944441460*y^2 - 39603059247494358200*y + 911555406262192128578 with a in -25.34269035681775? - 10.68732885060488?*I
**********************************************************************
1 item had failures:
   1 of   5 in sage.modular.modform_hecketriangle.hecke_triangle_group_element.coerce_AA
    [709 tests, 1 failure, 115.89 s]

and

sage -t --long src/sage/lfunctions/dokchitser.py
**********************************************************************
File "src/sage/lfunctions/dokchitser.py", line 127, in sage.lfunctions.dokchitser.Dokchitser
Failed example:
    L.taylor_series(1,4)
Expected:
    2.90614059330736e-20 + (-1.64690098288532e-20)*z + 0.759316500288427*z^2 - 0.430302337583362*z^3 + O(z^4)  
Got:
    -1.15498883731660e-19 + (6.54528640417060e-20)*z + 0.759316500288427*z^2 - 0.430302337583362*z^3 + O(z^4)
**********************************************************************

[jdemeyer]

The first one is because of interaction with #16976.

[jdemeyer]

The dokchitser failure is at least related to this ticket, but I don't see what could cause the failure... Maybe there is also some interaction with another ticket?

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. Last 10 new commits:

​9d3bb76	remove a useless TODO
​433b9d1	merge 6.5.beta5
​f63c738	trac #16976: clean code for Hecke group element
​616b048	trac #16976: removed 3 out of 5 @cached_method
​67116d5	addressed the review: renamed some functions, added 2 cached_methods back, improved documentation, fixed doctests
​9fabb44	improve documentation
​6d73481	merge sage-6.6.beta1 in #16976
​7dcee34	remove .gitignore, add further explanations on __init__, change ValueError to TypeError for element checks
​02aab05	Merge commit '7dcee3441862c9af9f1a63e1c3c8d4b957d2d705' into HEAD
​4c8b044	Fix doctest for _exact_field()

[jdemeyer]

Merged with #16976 and fixed first failure.

[jdemeyer]

For the dokchitser failure, there is some annoying global state (which is probably a bug) in the computel.gp script. For example, in the doctest

        sage: E = EllipticCurve('389a')
        sage: L = E.lseries().dokchitser()
        sage: L.num_coeffs ()
        156
        sage: L.derivative(1,E.rank())
        1.51863300057685
        sage: L.taylor_series(1,4)
        2.90760251490292e-20 + (-1.64772944938078e-20)*z + 0.759316500288427*z^2 - 0.430302337583362*z^3 + O(z^4)  # 32-bit
        2.90614059330736e-20 + (-1.64690098288532e-20)*z + 0.759316500288427*z^2 - 0.430302337583362*z^3 + O(z^4)  # 64-bit

removing the L.num_coeffs () test changes the last results.

[vbraun]

Aren't both answers valid (up to numerical precision). Its somewhat annoying for testing purposes, maybe reset the state with other rng seeds.

[vdelecroix]

Indeed

sage: E = EllipticCurve('389a')
sage: L = E.lseries().dokchitser()
sage: L.derivative(1,E.rank())
1.51863300057685
sage: s4_1 = L.taylor_series(1,4)
sage: L.num_coeffs ()
156
sage: s4_2 = L.taylor_series(1,4)
sage: s4_1 == s4_2
False
sage: s4_1 - s4_2
-2.90886971068887e-20 + (1.64844756535569e-20)*z + O(z^4)

[jdemeyer]

AFAIK, it's not a rng issue. It's just that the result depends on seemingly unrelated things (like whether or not L.num_coeffs() has been called).

Volker, do have an idea which elliptic-curve/L-function related ticket could have caused this?

[vbraun]

All doctests pass without the pari update, so its this ticket.

[jdemeyer]

Replying to vbraun:

All doctests pass without the pari update, so its this ticket.

All doctests also pass when just this ticket is applied, so it must be some interaction between this ticket and another unspecified ticket.

[vdelecroix]

Sided note: why is dokchister included as such in Sage source files? Wouldn't it be better as an external gp library (let say $SAGE_ROOT/local/pari/)? Morever, we ship version 1.3 and current version is 1.3.3 (see ​http://www.maths.bris.ac.uk/~matyd/computel/index.html). Though the last version does not solve the issue at all (but at least the src/ext/pari/dokchister/testall script is not emitting any warning anymore).

[jdemeyer]

Replying to vdelecroix:

Sided note: why is dokchister included as such in Sage source files?

Historical reasons. There are various such scripts in src/ext.

Wouldn't it be better as an external gp library (let say $SAGE_ROOT/local/pari/)? Morever, we ship version 1.3 and current version is 1.3.3 (see ​http://www.maths.bris.ac.uk/~matyd/computel/index.html). Though the last version does not solve the issue at all (but at least the src/ext/pari/dokchister/testall script is not emitting any warning anymore).

Perhaps, but let's leave that for a different ticket.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​73e4040

Fix for LaTeX doc

[jdemeyer]

I cannot reproduce the src/sage/lfunctions/dokchitser.py failure, even with 6.6.beta2. Can you confirm that the problem still exists?

[vbraun]

Yep, still happens (this time on UW redhawk)

[fbissey]

What kind of machine is redhawk?

[jdemeyer]

I guess we could easily solve this with some # tol but I'm very curious to know why this is happening. It would be the first time that the result of PARI was different on different 64-bit systems (it is well known that PARI often has different results for 32-bit vs. 64-bit, but all 32-bit and all 64-bit systems should be the same).

[zimmerma]

but all 32-bit and all 64-bit systems should be the same).

I would not bet on this. It suffices that some processor provides a fused multiply-add and that Pari uses it to get different results.

Paul

[jdemeyer]

Replying to zimmerma:

but all 32-bit and all 64-bit systems should be the same).

I would not bet on this. It suffices that some processor provides a fused multiply-add and that Pari uses it to get different results.

As far as I know, PARI doesn't use native floating-point operations. They use custom arbitrary-precision arithmetic which shouldn't depend on the system.

The reason there is a difference between 32-bit and 64-bit is because their precision is measured in words: some computation which would use 96 bits on a 32-bit system would use 128 bits on a 64-bit system.

[jdemeyer]

I confirm that PARI indeed gives a different result on redhawk. Upgrading Dokchitser's GP scripts seems to fix the issue, so that is what I will do.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​32e0b06

Upgrade Dokchitser's GP scripts

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​504f6fe

Upgrade Dokchitser's GP scripts

[jdemeyer]

Note: the file computel.gp is exactly Dokchitser's computel with 2 minor changes: DOS newlines converted to Unix and trailing whitespace removed.

[fbissey]

I am happy to put it back to positive review.

[jdemeyer]

Thanks, let's hope it can be merged this time.

[vbraun]

I now get this on linux 32-bit:

sage -t --long src/doc/en/reference/coercion/index.rst
**********************************************************************
File "src/doc/en/reference/coercion/index.rst", line 65, in doc.en.reference.coercion.index
Failed example:
    f = EllipticCurve('37a').lseries().taylor_series(10); f
Expected:
    0.997997869801216 + 0.00140712894524925*z - 0.000498127610960097*z^2 + 0.000118835596665956*z^3 - 0.0000215906522442707*z^4 + (3.20363155418419e-6)*z^5 + O(z^6)
Got:
    0.990010459847588 + 0.0191338632530789*z - 0.0197489006172923*z^2 + 0.0137240085327618*z^3 - 0.00703880791607153*z^4 + 0.00280906165766519*z^5 + O(z^6)
**********************************************************************
1 item had failures:
   1 of  99 in doc.en.reference.coercion.index
    [84 tests, 1 failure, 0.98 s]

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​451fb95

Fix another 32-bit doctest

[jdemeyer]

Fixed and tested on arando (32-bit).

[tmonteil]

Doctests should help tracking bugs, not hide them (last commit). Last digit noise is ok but 0.990010459847588 should not be considered as a correct 32-bits approximation of 0.997997869801216, this is too much noise and there should be a bug somewhere. Since the ticket is closed, there is a followup at #17903.