Opened 8 years ago

Closed 8 years ago

#16624 closed enhancement (fixed)

upgrade sympy to version 0.7.5

Reported by: Ralf Stephan Owned by:
Priority: major Milestone: sage-6.4
Component: packages: standard Keywords: upgrade, mpmath
Cc: François Bissey Merged in:
Authors: Ralf Stephan, Sergey B Kirpichev Reviewers: Volker Braun
Report Upstream: Reported upstream. Developers acknowledge bug. Work issues:
Branch: e6c24d3 (Commits, GitHub, GitLab) Commit: e6c24d3106cba8af3e9f662aa0b38725d656e520
Dependencies: Stopgaps:

Status badges

Attachments (1)

0001-Update-the-sympy-package.patch.gz (524.6 KB) - added by Sergey B Kirpichev 8 years ago.

Download all attachments as: .zip

Change History (28)

comment:1 Changed 8 years ago by Ralf Stephan

Branch: u/rws/upgrade_sympy_to_version_0_7_5

comment:2 Changed 8 years ago by Ralf Stephan

Authors: Ralf Stephan
Commit: 3cc6ca3d31c872ecaea8bd7985c8b20ed6c0e03e

No idea what's the matter here:

Traceback (most recent call last):
  File "/home/ralf/sage/src/bin/sage-runtests", line 87, in <module>
    err = DC.run()
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/control.py", line 904, in run
    self.run_doctests()
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/control.py", line 674, in run_doctests
    self.dispatcher = DocTestDispatcher(self)
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1335, in __init__
    init_sage()
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 113, in init_sage
    from sympy.printing.pretty.stringpict import stringPict
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/__init__.py", line 32, in <module>
    from .core import *
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/core/__init__.py", line 8, in <module>
    from .expr import Expr, AtomicExpr
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/core/expr.py", line 7, in <module>
    from .evalf import EvalfMixin, pure_complex
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/core/evalf.py", line 9, in <module>
    import sympy.mpmath.libmp as libmp
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/__init__.py", line 6, in <module>
    from .ctx_mp import MPContext
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/ctx_mp.py", line 47, in <module>
    from sage.libs.mpmath.ext_main import Context as BaseMPContext
  File "ext_main.pyx", line 38, in init sage.libs.mpmath.ext_main (build/cythonized/sage/libs/mpmath/ext_main.c:28725)
  File "/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/__init__.py", line 6, in <module>
    from .ctx_mp import MPContext
  File "/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/ctx_mp.py", line 49, in <module>
    import sage.libs.mpmath.ext_main as _mpf_module
AttributeError: 'module' object has no attribute 'ext_main'
make: *** [ptestlong] Error 1

New commits:

3cc6ca316624: sympy version 0.7.5

comment:3 Changed 8 years ago by git

Commit: 3cc6ca3d31c872ecaea8bd7985c8b20ed6c0e03ec280b72b302616eef2d5e0b94848fe8a02af091b

Branch pushed to git repo; I updated commit sha1. New commits:

c280b7216624: fix import issue

comment:4 Changed 8 years ago by Ralf Stephan

Even with sympy-0.7.4 in a fresh Sage I get

sage:     import sage.libs.mpmath.ext_main as _mpf_module
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-06f00c8c1fbc> in <module>()
----> 1 import sage.libs.mpmath.ext_main as _mpf_module

/home/ralf/sage/ext_main.pyx in init sage.libs.mpmath.ext_main (build/cythonized/sage/libs/mpmath/ext_main.c:28725)()

/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/__init__.py in <module>()
      4 
      5 from .ctx_fp import FPContext
----> 6 from .ctx_mp import MPContext
      7 from .ctx_iv import MPIntervalContext
      8 

/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/ctx_mp.py in <module>()
     47     from sage.libs.mpmath.ext_main import Context as BaseMPContext
     48     # pickle hack
---> 49     import sage.libs.mpmath.ext_main as _mpf_module
     50 else:
     51     from .ctx_mp_python import PythonMPContext as BaseMPContext

AttributeError: 'module' object has no attribute 'ext_main'

This and the 0.7.5 failure both can be cured for example with

--- mpmath/libmp/backend.py~    2013-08-08 14:49:58.000000000 +0200
+++ mpmath/libmp/backend.py     2014-07-11 08:54:28.759388954 +0200
@@ -71,6 +71,7 @@
     try:
         import sage.all
         import sage.libs.mpmath.utils as _sage_utils
+        import sage.libs.mpmath.ext_main as _mpf_module
         sage = sage.all
         sage_utils = _sage_utils
         BACKEND = 'sage'

So it seems ext_main has to be imported in the right place and there was somewhere an import in sympy-0.7.4 that made that version work but that was changed in 0.7.5. It does not work to simply catch the error and import again, while it works when in a fresh Sage with 0.7.4.

comment:5 Changed 8 years ago by For batch modifications

Milestone: sage-6.3sage-6.4

comment:6 Changed 8 years ago by François Bissey

Cc: François Bissey added

comment:7 Changed 8 years ago by François Bissey

The whole thing seems to go in circle. I am wondering if the real problem is that first a function is imported from ext_main

/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/ctx_mp.py in <module>()
     47     from sage.libs.mpmath.ext_main import Context as BaseMPContext

and then you want to import the whole ext_main

     48     # pickle hack
---> 49     import sage.libs.mpmath.ext_main as _mpf_module

I wonder what they mean by pickle hack? Presumably if we add the import in backend.py the import in ctx_mp.py becomes redundant?

comment:8 Changed 8 years ago by Ralf Stephan

I tried this patch but the results are the same:

diff --git a/build/pkgs/sympy/patches/import-extmain.patch b/build/pkgs/sympy/patches/import-extmain.patch
new file mode 100644
index 0000000..af222c1
--- /dev/null
+++ b/build/pkgs/sympy/patches/import-extmain.patch
@@ -0,0 +1,15 @@
+--- sympy/mpmath/ctx_mp.py~   2014-02-22 20:13:32.000000000 +0100
++++ sympy/mpmath/ctx_mp.py    2014-09-11 18:09:11.877599866 +0200
+@@ -44,9 +44,9 @@
+                          r'(?P<im>[\+\-]?\d*\.?\d*(e[\+\-]?\d+)?j)?\)?$')
+ 
+ if BACKEND == 'sage':
+-    from sage.libs.mpmath.ext_main import Context as BaseMPContext
+     # pickle hack
+-    import sage.libs.mpmath.ext_main as _mpf_module
++    from sage.libs.mpmath import ext_main as _mpf_module
++    from sage.libs.mpmath.ext_main import Context as BaseMPContext
+ else:
+     from .ctx_mp_python import PythonMPContext as BaseMPContext
+     from . import ctx_mp_python as _mpf_module
+

To be specific, at the moment it's not possible to import anything from sympy. What happens is that at the first attempt the following is raised:

----> 1 from sympy import sympify

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/__init__.py in <module>()
     33 from .logic import *
     34 from .assumptions import *
---> 35 from .polys import *
     36 from .series import *
     37 from .functions import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/__init__.py in <module>()
      3 __all__ = []
      4 
----> 5 from . import polytools
      6 __all__.extend(polytools.__all__)
      7 from .polytools import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/polytools.py in <module>()                                                                                      
     50 import sympy.mpmath
     51 
---> 52 from sympy.polys.domains import FF, QQ
     53 from sympy.polys.constructor import construct_domain
     54 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/__init__.py in <module>()
     76 QQ_gmpy = GMPYRationalField
     77 
---> 78 RR = RealField()
     79 CC = ComplexField()
     80 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/realfield.py in __init__(self, prec, dps, tol)
     52 
     53         self.dtype = context.mpf
---> 54         self.zero = self.dtype(0)
     55         self.one = self.dtype(1)
     56 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/ctx_mp_python.py in __new__(cls, val, **kwargs)
     75         else:
     76             v = new(cls)
---> 77             v._mpf_ = mpf_pos(cls.mpf_convert_arg(val, prec, rounding), prec, rounding)
     78             return v
     79 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/mpelements.py in _set_mpf(self, val)
     23         tol = self.context.tol
     24 
---> 25         if mpf_lt(mpf_abs(val, prec, rounding), tol):
     26             self.__mpf__ = fzero
     27         else:

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/libmp/libmpf.py in mpf_abs(s, prec, rnd)
    662             return (0, man, exp, bc)
    663         return s
--> 664     return normalize1(0, man, exp, bc, prec, rnd)
    665 
    666 def mpf_sign(s):

TypeError: Argument 'man' has incorrect type (expected sage.rings.integer.Integer, got long)

and at any further attempt this one:

----> 1 from sympy.solvers.diophantine import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/__init__.py in <module>()
     33 from .logic import *
     34 from .assumptions import *
---> 35 from .polys import *
     36 from .series import *
     37 from .functions import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/__init__.py in <module>()
      3 __all__ = []
      4 
----> 5 from . import polytools
      6 __all__.extend(polytools.__all__)
      7 from .polytools import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/polytools.py in <module>()                                                                                      
     50 import sympy.mpmath
     51 
---> 52 from sympy.polys.domains import FF, QQ
     53 from sympy.polys.constructor import construct_domain
     54 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/__init__.py in <module>()
      3 __all__ = []
      4 
----> 5 from . import domain
      6 __all__.extend(domain.__all__)
      7 from .domain import *

ImportError: cannot import name domain

comment:9 Changed 8 years ago by Ralf Stephan

Description: modified (diff)
Report Upstream: N/AReported upstream. No feedback yet.
Status: newneeds_info

Changed 8 years ago by Sergey B Kirpichev

comment:10 Changed 8 years ago by Sergey B Kirpichev

If you include patch to unbundle mpmath (see attached patch) - it will solve your problem.

comment:11 Changed 8 years ago by Sergey B Kirpichev

Report Upstream: Reported upstream. No feedback yet.Reported upstream. Developers acknowledge bug.
Status: needs_infoneeds_review

comment:12 in reply to:  10 Changed 8 years ago by François Bissey

Replying to skirpichev:

If you include patch to unbundle mpmath (see attached patch) - it will solve your problem.

I am a bit surprised by this assertion. Gentoo and by extension sage-on-gentoo unbundle mpmath (I was the person who introduced unbundling of mpmath from sympy on Gentoo). Yet I can see this bug with sympy 0.7.4.1 and mpmath 0.19. Your patch must do something else apart from unbundling.

comment:13 Changed 8 years ago by Sergey B Kirpichev

I am a bit surprised by this assertion.

It does work for me. Can you reproduce the bug with this patch?

Gentoo and by extension sage-on-gentoo unbundle mpmath

This: http://dev.gentoo.org/~bicatali/distfiles/sympy-0.7.4.1-system-mpmath.patch.gz ?

Perhaps, it's wrong patch and it just break sympy. Can you post the traceback for failure in this version?

Your patch must do something else apart from unbundling.

Sure. "Something else" - SPKG.txt changes (e.g. Homepage field, changelog and so on). But the only real change is unbundling mpmath.

Last edited 8 years ago by Sergey B Kirpichev (previous) (diff)

comment:14 Changed 8 years ago by François Bissey

Your big patch is unreadable - too big. most of it is the removal of mpmath sources from sympy. This could be achieved in spkg-src by doing a "rm -rf" in the appropriate place. Once that's out of the way we can look at the patch to do the rest and it won't be 2.1MB and we'll be able to see something.

comment:15 Changed 8 years ago by Ralf Stephan

Authors: Ralf StephanRalf Stephan, Sergey B Kirpichev
Branch: u/rws/upgrade_sympy_to_version_0_7_5public/sympy075
Commit: c280b72b302616eef2d5e0b94848fe8a02af091b
Keywords: upgrade mpmath added

It looks like the only file not being deleted is mpmath/tests/__init__.py.

comment:16 Changed 8 years ago by git

Commit: 63650730577e933ba984669ee1fd1f5eba3a2fef

Branch pushed to git repo; I updated commit sha1. New commits:

6365073shortened patch

comment:17 Changed 8 years ago by Ralf Stephan

More can be deleted (mpmath docs). This passes make ptestlong.

comment:18 Changed 8 years ago by git

Commit: 63650730577e933ba984669ee1fd1f5eba3a2fef04d8d622e6803810d5a64826e57f4cc7f74b83eb

Branch pushed to git repo; I updated commit sha1. New commits:

04d8d62remove doc/src/modules/mpmath from patch; remove files via spkg-install

comment:19 Changed 8 years ago by Ralf Stephan

 build/pkgs/sympy/patches/01_no-mpmath.patch | 841 +++++++++++++++++++++++++++++++++++

Now that's more like it. Please review.

comment:20 Changed 8 years ago by Sergey B Kirpichev

Your big patch is unreadable - too big.

It's upstream patch. Does it harm your patch utility, what's wrong?)

I don't see points to modify it, unless you want break this patch just like gentoo one...

remove files via spkg-install

remove sympy/mpmath/tests/init.py too

comment:21 Changed 8 years ago by git

Commit: 04d8d622e6803810d5a64826e57f4cc7f74b83eb4c9f288dd8b571fdda9dc31906a552483b9e2afd

Branch pushed to git repo; I updated commit sha1. New commits:

4c9f28816624: remove mpmath/tests/__init__.py too

comment:22 Changed 8 years ago by Volker Braun

Status: needs_reviewpositive_review

lgtm

comment:23 Changed 8 years ago by Volker Braun

Reviewers: Volker Braun

comment:24 Changed 8 years ago by Jeroen Demeyer

Status: positive_reviewneeds_work

When applying patches, please use the code from http://www.sagemath.org/doc/developer/packaging.html#patching-sources

Otherwise patching will break needlessly if there no patches.

comment:25 Changed 8 years ago by git

Commit: 4c9f288dd8b571fdda9dc31906a552483b9e2afde6c24d3106cba8af3e9f662aa0b38725d656e520

Branch pushed to git repo; I updated commit sha1. New commits:

60f9af7Merge branch 'develop' into t/16624/public/sympy075
e6c24d316624: use standard patch script in spkg-install

comment:26 Changed 8 years ago by Ralf Stephan

Status: needs_workpositive_review

comment:27 Changed 8 years ago by Volker Braun

Branch: public/sympy075e6c24d3106cba8af3e9f662aa0b38725d656e520
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.