Opened 6 years ago

Closed 4 years ago

Last modified 4 years ago

#19706 closed enhancement (fixed)

Upgrade GMP

Reported by: jdemeyer Owned by:
Priority: major Milestone: sage-8.1
Component: packages: optional Keywords:
Cc: jpflori Merged in:
Authors: Jeroen Demeyer Reviewers: François Bissey
Report Upstream: N/A Work issues:
Branch: 171e356 (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Change History (22)

comment:1 Changed 6 years ago by jdemeyer

  • Description modified (diff)

comment:2 Changed 6 years ago by jdemeyer

  • Branch set to u/jdemeyer/upgrade_gmp

comment:3 Changed 6 years ago by jdemeyer

  • Commit set to 94ac0168fcb2c77883bd37b121f047ccd94d9e1a
  • Status changed from new to needs_review

New commits:

94ac016Upgrade GMP to version 6.1.0

comment:4 Changed 6 years ago by fbissey

I haven't tried 6.1.0 yet but with 6.0.0a there is a doctest failure in sage/libs/libecm.pyx (I contacted Paul Zimmerman about it and he basically said he prefered the convention used for the new result, I can fetch the emails if interested). Also it caused problems with maxima, or rather the underlying ecl. Upgrading ecl to 16.0.0 fixed that but brings some formatting noise (output format changed from 1.2E-3 to 0.12E-2) see: https://github.com/cschwan/sage-on-gentoo/issues/356

comment:5 Changed 6 years ago by jdemeyer

  • Status changed from needs_review to needs_work

comment:6 follow-up: Changed 6 years ago by jdemeyer

There is also this error, which is due to error handling in GMP which does not exist in MPIR:

**********************************************************************
File "src/sage/ext/memory.pyx", line 9, in sage.ext.memory
Failed example:
    2^(2^63-2)
Expected:
    Traceback (most recent call last):
    ...
    MemoryError: failed to allocate 1152921504606847008 bytes  
Got:
    gmp: overflow in mpz type
    Traceback (most recent call last):
      File "/usr/local/src/sage-config/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-config/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.ext.memory[0]>", line 1, in <module>
        Integer(2)**(Integer(2)**Integer(63)-Integer(2))
      File "sage/rings/integer.pyx", line 1921, in sage.rings.integer.Integer.__pow__ (build/cythonized/sage/rings/integer.c:13350)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 98, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1113)
        raise RuntimeError(msg)
    RuntimeError: Aborted
**********************************************************************

comment:7 Changed 6 years ago by jdemeyer

I get these failures:

----------------------------------------------------------------------
sage -t --long src/sage/structure/sage_object.pyx  # Killed due to abort
sage -t --long src/sage/interfaces/maxima.py  # 2 doctests failed
sage -t --long src/sage/libs/libecm.pyx  # 1 doctest failed
sage -t --long src/doc/en/constructions/algebraic_geometry.rst  # 1 doctest failed
sage -t --long src/sage/ext/memory.pyx  # 1 doctest failed
----------------------------------------------------------------------

comment:8 in reply to: ↑ 6 ; follow-up: Changed 6 years ago by fbissey

Replying to jdemeyer:

There is also this error, which is due to error handling in GMP which does not exist in MPIR:

**********************************************************************
File "src/sage/ext/memory.pyx", line 9, in sage.ext.memory
Failed example:
    2^(2^63-2)
Expected:
    Traceback (most recent call last):
    ...
    MemoryError: failed to allocate 1152921504606847008 bytes  
Got:
    gmp: overflow in mpz type
    Traceback (most recent call last):
      File "/usr/local/src/sage-config/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/local/src/sage-config/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.ext.memory[0]>", line 1, in <module>
        Integer(2)**(Integer(2)**Integer(63)-Integer(2))
      File "sage/rings/integer.pyx", line 1921, in sage.rings.integer.Integer.__pow__ (build/cythonized/sage/rings/integer.c:13350)
        sig_on()
      File "sage/ext/interrupt/interrupt.pyx", line 98, in sage.ext.interrupt.interrupt.sig_raise_exception (build/cythonized/sage/ext/interrupt/interrupt.c:1113)
        raise RuntimeError(msg)
    RuntimeError: Aborted
**********************************************************************

I didn't mention that one because it is an acknowledged issue for a long time. The failure in sage/libs/libecm.pyx and sage/interfaces/maxima.py I was expecting. The other two are new I think, I'll have to check carefully. The only way to get out of the one with maxima is upgrading ecl. For ecm we may have to change the doctest, I'll dig my emails to Paul about it.

comment:9 Changed 6 years ago by fbissey

My discussion with Paul is in French, roughly translated it goes: "It is linked to a change of behavior of the mpz_invert between 5.1.3 and 6.0.0 when the modulo (last argument) is 1

zimmerma@coing:/tmp$ cat e.c
#include <stdio.h>
#include "gmp.h"

main()
{
 mpz_t a, b;
 int ret;
 printf ("GMP %s\n", gmp_version);
 mpz_init_set_ui (a, 1);
 mpz_init_set_ui (b, 1);
 ret = mpz_invert (a, a, b);
 gmp_printf ("ret=%d a=%Zd\n", ret, a);
}

zimmerma@coing:/tmp$ gcc e.c /usr/local/gmp-5.1.3/lib/libgmp.a
zimmerma@coing:/tmp$ ./a.out 
GMP 5.1.3
ret=0 a=1

zimmerma@coing:/tmp$ gcc e.c /usr/local/gmp-6.0.0/lib/libgmp.a
zimmerma@coing:/tmp$ ./a.out 
GMP 6.0.0
ret=1 a=0

GMP 6.0.0's behavior feels more logical to me "

comment:10 Changed 6 years ago by fbissey

I cannot reproduce the last two failures (in sage_object.pyx and algebraic_geometry.rst) in sage-on-gentoo after moving to gmp-6.1.0 on my main machine. Could you provide more details?

comment:11 Changed 6 years ago by jpflori

  • Cc jpflori added

comment:12 Changed 4 years ago by jdemeyer

  • Description modified (diff)
  • Milestone changed from sage-7.0 to sage-8.1

comment:13 Changed 4 years ago by git

  • Commit changed from 94ac0168fcb2c77883bd37b121f047ccd94d9e1a to 171e35673ca2e6bfd95a86180d60d8efe3513f00

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

171e356Upgrade GMP to version 6.1.2

comment:14 in reply to: ↑ 8 Changed 4 years ago by jdemeyer

Replying to fbissey:

I didn't mention that one because it is an acknowledged issue for a long time.

I looked this up and this seems to be since GMP-4.2.3:

Changes between GMP version 4.2.2 and 4.2.3
[...]
  Features:
  * The allocation functions now detect overflow of the mpz_t type.  This means
    that overflow will now cause an abort, except when the allocation
    computation itself overflows.  (Such overflow can probably only happen in
    powering functions; we will detect powering overflow in the future.)

comment:15 Changed 4 years ago by jdemeyer

I asked about this error checking on the GMP mailing list: https://gmplib.org/list-archives/gmp-discuss/2017-September/006144.html

comment:16 Changed 4 years ago by jdemeyer

  • Status changed from needs_work to needs_review

Builds fine. Running doctests now...

comment:17 Changed 4 years ago by jdemeyer

All doctests pass apart from the known issues

**********************************************************************
File "src/sage/rings/integer.pyx", line 6099, in sage.rings.integer.Integer._shift_helper
Failed example:
    1 << (2^60)
Expected:
    Traceback (most recent call last):
    ...
    MemoryError: failed to allocate ... bytes   
Got:
    gmp: overflow in mpz type
    Traceback (most recent call last):
      File "/home/jdemeyer/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 515, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/jdemeyer/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 885, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.rings.integer.Integer._shift_helper[8]>", line 1, in <module>
        Integer(1) << (Integer(2)**Integer(60))
      File "sage/rings/integer.pyx", line 6177, in sage.rings.integer.Integer.__lshift__ (build/cythonized/sage/rings/integer.c:39270)
        return (<Integer>x)._shift_helper(y, 1)
      File "sage/rings/integer.pyx", line 6138, in sage.rings.integer.Integer._shift_helper (build/cythonized/sage/rings/integer.c:39014)
        sig_on()
    RuntimeError: Aborted
**********************************************************************

and

sage -t --warn-long 61.3 src/sage/ext/memory.pyx
**********************************************************************
File "src/sage/ext/memory.pyx", line 9, in sage.ext.memory
Failed example:
    2^(2^63-2)
Expected:
    Traceback (most recent call last):
    ...
    MemoryError: failed to allocate 1152921504606847008 bytes  
Got:
    gmp: overflow in mpz type
    Traceback (most recent call last):
      File "/home/jdemeyer/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 515, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/jdemeyer/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 885, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.ext.memory[0]>", line 1, in <module>
        Integer(2)**(Integer(2)**Integer(63)-Integer(2))
      File "sage/rings/integer.pyx", line 2067, in sage.rings.integer.Integer.__pow__ (build/cythonized/sage/rings/integer.c:14173)
        sig_on()
    RuntimeError: Aborted
**********************************************************************

comment:18 Changed 4 years ago by fbissey

  • Reviewers set to François Bissey
  • Status changed from needs_review to positive_review

Good enough for me.

comment:19 Changed 4 years ago by jdemeyer

I created #23913 for the longstanding issue of the GMP errors.

comment:20 Changed 4 years ago by vbraun

  • Branch changed from u/jdemeyer/upgrade_gmp to 171e35673ca2e6bfd95a86180d60d8efe3513f00
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:21 Changed 4 years ago by mderickx

  • Commit 171e35673ca2e6bfd95a86180d60d8efe3513f00 deleted

How do I pull the branch 171e35673ca2e6bfd95a86180d60d8efe3513f00 ?

comment:22 Changed 4 years ago by mderickx

Sorry, apparently it's a commit not a branch, and for some reason git pull does not work but git fetch does.

Last edited 4 years ago by mderickx (previous) (diff)
Note: See TracTickets for help on using tickets.