Opened 7 years ago

Closed 6 years ago

#9969 closed enhancement (fixed)

Update extension code for mpmath-0.17

Reported by: fredrik.johansson Owned by:
Priority: blocker Milestone: sage-4.7
Component: packages: standard Keywords:
Cc: Merged in: sage-4.7.alpha5
Authors: Fredrik Johansson Reviewers: François Bissey
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jdemeyer)

The patch adds fast Cython version of some more mpmath functions to sage.libs.mpmath.

This ticket contains a new spkg for mpmath-0.17 and two patches that need to be applied to sage itself. First mpmath_update_fixed_4.6.1.patch, then truediv_fix.patch.

It can be tested with:

sage: import mpmath
sage: mpmath.runtests()
sage: mpmath.doctests()

New SPKG: http://boxen.math.washington.edu/home/jdemeyer/spkg/mpmath-0.17.spkg

Apply:

Attachments (5)

14600.patch (56.8 KB) - added by fredrik.johansson 7 years ago.
updated mpmath Cython extension code
docstring-fixes.patch (21.6 KB) - added by fredrik.johansson 6 years ago.
mpmath_update_fixed_4.6.1.patch (60.7 KB) - added by fredrik.johansson 6 years ago.
Fixed patch that should apply in sage-4.6.1
truediv_fix.patch (924 bytes) - added by fredrik.johansson 6 years ago.
Patch that fixes some mpmath test failures due to use of the truediv operator
mpmath-deps.patch (565 bytes) - added by fbissey 6 years ago.
patch to deps to make mpmath depend on matplotlib

Download all attachments as: .zip

Change History (46)

Changed 7 years ago by fredrik.johansson

updated mpmath Cython extension code

comment:1 Changed 6 years ago by fredrik.johansson

  • Status changed from new to needs_review

Here is an spkg: http://sage.math.washington.edu/home/fredrik/mpmath-0.16.spkg

It should work fine together with the patch. Sorry for the very long delay!

comment:2 Changed 6 years ago by was

  • Status changed from needs_review to needs_work

REPORT:

  • All tests passon sage.math.
  • Your code needs "::\n\n" before any sage sessions for sphinx to format it right.

See the template here and note rightafterwords: http://sagemath.blogspot.com/ right now if somebody introspects one of your functions in the notebook, sphinx will butcher it.

  • You define "isqrt" but sage integers already have an isqrt method that is exactly the same. Clarify in the documentation the distinction.

Bug me again by email or chat when you've posted another patch (or patch on top of the above) that fixes the above two little things.

comment:3 Changed 6 years ago by was

Oh, could you have at least a short comment explaining what every single function does. E.g.,

cdef cy_exp_basecase(mpz_t y, mpz_t x, int prec):
     # <what this function does>

Changed 6 years ago by fredrik.johansson

comment:4 Changed 6 years ago by fredrik.johansson

  • Owner changed from tbd to (none)

Added second patch with docstring fixes.

comment:5 Changed 6 years ago by fredrik.johansson

  • Authors set to fredrik.johansson
  • Status changed from needs_work to needs_review

comment:6 Changed 6 years ago by fbissey

If I understand correctly you need to apply 14600.patch first, then docstring-fixes.patch? This goes in sage-on-gentoo tonight for testing.

comment:7 Changed 6 years ago by fbissey

I forgot, for it to go in sage proper a new spkg will be needed. Are you working on that?

comment:8 Changed 6 years ago by fredrik.johansson

Yes, the patches should be applied in that order.

The spkg was linked in a post above: http://sage.math.washington.edu/home/fredrik/mpmath-0.16.spkg

comment:9 Changed 6 years ago by fbissey

  • Description modified (diff)
  • Milestone set to sage-4.7

I should get a reader since I am obviously blind :) I say we try to get it formally in 4.7, I rewrite the description a little bit.

comment:10 Changed 6 years ago by fbissey

I just saw your post about mpmath-0.17 release, are you planing to update this ticket for mpmath 0.17?

comment:11 follow-up: Changed 6 years ago by fredrik.johansson

Yes, I'll create an spkg for 0.17 immediately to replace the 0.16 one.

Nothing should have changed in the way of Sage compatibility, so the patches should still be fine.

comment:12 Changed 6 years ago by fredrik.johansson

  • Description modified (diff)
  • Summary changed from Update extension code for mpmath-0.16 to Update extension code for mpmath-0.17

comment:13 Changed 6 years ago by fredrik.johansson

By the way, I noticed (too late) that there are two failing doctests if you run mpmath.doctests() as per the instructions. These failures a trivial (output formatting) and can be ignored.

comment:14 in reply to: ↑ 11 Changed 6 years ago by fbissey

Replying to fredrik.johansson:

Yes, I'll create an spkg for 0.17 immediately to replace the 0.16 one.

Nothing should have changed in the way of Sage compatibility, so the patches should still be fine.

Good to know, that means I won't have to change anything to support mpmath-0.17 in sage-on-gentoo when 4.6.2 is released.

Changed 6 years ago by fredrik.johansson

Fixed patch that should apply in sage-4.6.1

Changed 6 years ago by fredrik.johansson

Patch that fixes some mpmath test failures due to use of the truediv operator

comment:15 Changed 6 years ago by fredrik.johansson

  • Description modified (diff)

comment:16 Changed 6 years ago by fbissey

The two new patches together are smaller than the two previous patches. Is docstring-fixes.patch really not needed anymore?

comment:17 follow-up: Changed 6 years ago by fredrik.johansson

Most of docstring-fixes.patch consisted of reformatting existing docstrings, and there's a lot of air in the diff format, so I think the numbers add up.

comment:18 in reply to: ↑ 17 Changed 6 years ago by fbissey

Replying to fredrik.johansson:

Most of docstring-fixes.patch consisted of reformatting existing docstrings, and there's a lot of air in the diff format, so I think the numbers add up.

I thought it could be the case. I will try to review this quickly so we can have it included in 4.7.alpha.

comment:19 Changed 6 years ago by fbissey

tests are not looking good to me.

Python 2.6.6 (r266:84292, Jan 28 2011, 01:59:55) 
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mpmath
>>> mpmath.runtests()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/site-packages/mpmath/__init__.py", line 413, in runtests
    from .tests import runtests as tests
ImportError: No module named tests

I get pretty much the same thing from sage.

By the way that looks more like a procedure to do a spkg-check than a doctest.

comment:20 Changed 6 years ago by fredrik.johansson

Hmm... it doesn't seem like mpmath installed correctly for you.

Is "/usr/lib64/python2.6/site-packages" actually the location of your Sage site-packages library, or is this a standalone installation of mpmath 0.17?

comment:21 Changed 6 years ago by fbissey

sorry for the noise I shouldn't post past 11pm. There is an issue in Gentoo and sage-on-gentoo but there shouldn't be any in a pure sage environment. It may take a little bit of time testing this on various hardware.

comment:22 Changed 6 years ago by fbissey

Something I have noticed in the gentoo ebuild. There is now an option to use matplotlib. Since matplotlib is shipped in sage mpmath may have to depend on it or configured not to use it.

Usually people choose the first option in this case, unless there is major trouble. If you want to do that, the deps file in spkg/standard will need to be updated.

comment:23 Changed 6 years ago by fbissey

I got a number of failing mpmath.doctests on my x86 box. A formatting issue really. Does any of the code interacts with ecl or maxima? The install I tested it on is based on 4.6.2.alpha4 with ecl-11.1.1 (#10766) and maxima-5.23.2 (#10773) and a couple of other patch that should be orthogonal (ppl #10039 and associated patches).

almosteq 0.004
findroot **********************************************************************
File "/home/francois/Work/sandbox/sage-4.6.2.alpha4/local/lib/python2.6/site-packages/mpmath/calculus/optimization.py", line 835, in NoName
Failed example:
    findroot(f, -10, solver='anewton', verbose=True)
Expected:
    x:     -9.88888888888888888889
    error: 0.111111111111111111111
    converging slowly
    x:     -9.77890011223344556678
    error: 0.10998877665544332211
    converging slowly
    x:     -9.67002233332199662166
    error: 0.108877778911448945119
    converging slowly
    accelerating convergence
    x:     -9.5622443299551077669
    error: 0.107778003366888854764
    converging slowly
    x:     0.99999999999999999214
    error: 10.562244329955107759
    x:     1.0
    error: 7.8598304758094664213e-18
    1.0
Got:
    ('x:    ', -9.88888888888888888889)
    ('error:', 0.111111111111111111111)
    converging slowly
    ('x:    ', -9.77890011223344556678)
    ('error:', 0.10998877665544332211)
    converging slowly
    ('x:    ', -9.67002233332199662166)
    ('error:', 0.108877778911448945119)
    converging slowly
    accelerating convergence
    ('x:    ', -9.5622443299551077669)
    ('error:', 0.107778003366888854764)
    converging slowly
    ('x:    ', 0.99999999999999999214)
    ('error:', 10.562244329955107759)
    ('x:    ', 1.0)
    ('error:', 7.8598304758094664213e-18)
    1.0
0.132
__name__ 0.001

and

superfac 0.05
secondzeta **********************************************************************
File "/home/francois/Work/sandbox/sage-4.6.2.alpha4/local/lib/python2.6/site-packages/mpmath/functions/zeta.py", line 983, in NoName
Failed example:
    secondzeta(0.5 + 40j, error=True, verbose=True)
Expected:
    main term = (-30190318549.138656312556 - 13964804384.624622876523j)
        computed using 19 zeros of zeta
    prime term = (132717176.89212754625045 + 188980555.17563978290601j)
        computed using 9 values of the von Mangoldt function
    exponential term = (542447428666.07179812536 + 362434922978.80192435203j)
    singular term = (512124392939.98154322355 + 348281138038.65531023921j)
    ((0.059471043 + 0.3463514534j), 1.455191523e-11)
Got:
    ('main term =', (-30190318549.138656312556 - 13964804384.624622876523j))
    ('    computed using', 19, 'zeros of zeta')
    ('prime term =', (132717176.89212754625045 + 188980555.17563978290601j))
    ('    computed using', 9, 'values of the von Mangoldt function')
    ('exponential term =', (542447428666.07179812536 + 362434922978.80192435203j))
    ('singular term =', (512124392939.98154322355 + 348281138038.65531023921j))
    ((0.059471043 + 0.3463514534j), 1.455191523e-11)
**********************************************************************
File "/home/francois/Work/sandbox/sage-4.6.2.alpha4/local/lib/python2.6/site-packages/mpmath/functions/zeta.py", line 992, in NoName
Failed example:
    secondzeta(0.5 + 40j, a=0.04, error=True, verbose=True)
Expected:
    main term = (-151962888.19606243907725 - 217930683.90210294051982j)
        computed using 9 zeros of zeta
    prime term = (2476659342.3038722372461 + 28711581821.921627163136j)
        computed using 37 values of the von Mangoldt function
    exponential term = (178506047114.7838188264 + 819674143244.45677330576j)
    singular term = (175877424884.22441310708 + 790744630738.28669174871j)
    ((0.059471043 + 0.3463514534j), 1.455191523e-11)
Got:
    ('main term =', (-151962888.19606243907725 - 217930683.90210294051982j))
    ('    computed using', 9, 'zeros of zeta')
    ('prime term =', (2476659342.3038722372461 + 28711581821.921627163136j))
    ('    computed using', 37, 'values of the von Mangoldt function')
    ('exponential term =', (178506047114.7838188264 + 819674143244.45677330576j))
    ('singular term =', (175877424884.22441310708 + 790744630738.28669174871j))
    ((0.059471043 + 0.3463514534j), 1.455191523e-11)
34.812
inf 0.002

I am preparing a run on amd64 with a fresh 4.6.2.rc0 without any patches.

comment:24 follow-up: Changed 6 years ago by fredrik.johansson

These two failures were noted in a previous comment. They can safely be ignored.

comment:25 in reply to: ↑ 24 Changed 6 years ago by fbissey

Replying to fredrik.johansson:

These two failures were noted in a previous comment. They can safely be ignored.

I didn't realise they were the same one. I guess I expected they were fixed but that would require a new version of mpmath, not just sage patch I imagine. Once I have tested this at least on amd64 I can give a positive review.

comment:26 Changed 6 years ago by fbissey

OK so I tested this on linux-amd64 and OS X (10.5). and the tests perform ok. Although there is some noise on OS X because the wrong version of freetype is detected. I am not sure that the spkg fault or not.

test_visualization
    axes                      dyld: Library not loaded: /usr/X11/lib/libfreetype.6.dylib
  Referenced from: /usr/X11/bin/fc-list
  Reason: Incompatible library version: fc-list requires version 13.0.0 or later, but libfreetype.6.dylib provides version 10.0.0
dyld: Library not loaded: /usr/X11/lib/libfreetype.6.dylib
  Referenced from: /usr/X11/bin/fc-list
  Reason: Incompatible library version: fc-list requires version 13.0.0 or later, but libfreetype.6.dylib provides version 10.0.0
ok        6.5380421 s

finished tests in 17.99 seconds

On further inspection this test is run because matplotlib is present and I assume that this message originates from matplotlib. So the problem is probably in matplotlib.

The last thing is whether we make matplotlib a dependency of mpmath or not. I will post a patch making this the default. Voice any opposition then.

Changed 6 years ago by fbissey

patch to deps to make mpmath depend on matplotlib

comment:27 Changed 6 years ago by fbissey

Fredrik, a question about mpmath behavior with regards to matplotlib. Is mpmath detecting matplotlib at compilation time or its presence at runtime?

comment:28 Changed 6 years ago by fredrik.johansson

Runtime. In fact, it only attempts to import matplotlib from inside the plotting functions (which are pure Python).

comment:29 Changed 6 years ago by fbissey

  • Status changed from needs_review to positive_review

Thanks, that means we don't need to depend on it. I am giving this a positive review. Hopefully it will be picked up for 4.7.

comment:30 Changed 6 years ago by fbissey

  • Description modified (diff)

comment:31 Changed 6 years ago by jdemeyer

  • Authors changed from fredrik.johansson to Fredrik Johansson
  • Reviewers set to François Bissey

comment:32 Changed 6 years ago by jdemeyer

  • Status changed from positive_review to needs_work

Please change the commit message (using hg qrefresh -e) such that the ticket number appears on the first line.

comment:33 Changed 6 years ago by jdemeyer

  • Status changed from needs_work to positive_review

comment:34 Changed 6 years ago by jdemeyer

  • Merged in set to sage-4.7.alpha5
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:35 Changed 6 years ago by fbissey

Sorry I meant to do the refresh but I was unable to get it working.

comment:36 Changed 6 years ago by jdemeyer

  • Resolution fixed deleted
  • Status changed from closed to new

The files SPKG.txt and spkg-install should be put under hg control. Also, SPKG.txt should not be executable. Please fix these trivial issues.

comment:37 Changed 6 years ago by jdemeyer

  • Priority changed from major to blocker

comment:38 follow-up: Changed 6 years ago by fredrik.johansson

I can fix the permissions in the spkg. What is involved to put SPKG.txt and spkg-install under hg control?

comment:39 in reply to: ↑ 38 Changed 6 years ago by jdemeyer

  • Status changed from new to needs_review

Replying to fredrik.johansson:

I can fix the permissions in the spkg. What is involved to put SPKG.txt and spkg-install under hg control?

Never mind. I did it myself: http://boxen.math.washington.edu/home/jdemeyer/spkg/mpmath-0.17.spkg

comment:40 Changed 6 years ago by jdemeyer

  • Description modified (diff)

comment:41 Changed 6 years ago by jdemeyer

  • Resolution set to fixed
  • Status changed from needs_review to closed
Note: See TracTickets for help on using tickets.