#13973 closed defect (fixed)
Upgrade Maxima to 5.33.0
Reported by:  dimpase  Owned by:  tbd 

Priority:  major  Milestone:  sage6.3 
Component:  packages: standard  Keywords:  sd48 
Cc:  Snark  Merged in:  
Authors:  Peter Bruin  Reviewers:  Marc Mezzarobba 
Report Upstream:  N/A  Work issues:  
Branch:  a130eed (Commits)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by pbruin)
This is a continuation of #13364, and aims at including more upstream bug fixes, which e.g. fix an
issue reported on sagesupport,
which was reported as Maxima bug 2535, and marked there as closed in post5.29.1.
Fixes #11894, #13526, #13712, #14209, #14306, #15386 and possibly other Maximarelated bugs.
Ought to fix #13733 (but loading the abs_integrate package currently causes this to fail).
Ought to fix #8728 (but Maxima asks for more constraints which follow from the existing ones).
Does not fix #14591, #14677, #15033.
tarball: http://homepages.warwick.ac.uk/staff/P.Bruin/sage/maxima5.33.0.tar.bz2
Note about the existing maxima_bug_2526.patch: the bug was marked "closed" in the Maxima bug tracking system, but the commit somehow disappeared from the Maxima repository and the bug has been reopened. Hence this patch is still needed to fix it.
Change History (50)
comment:1 Changed 20 months ago by kcrisman
 Dependencies changed from 13364 to #13364
comment:2 Changed 20 months ago by kcrisman
comment:3 followup: ↓ 4 Changed 20 months ago by zimmerma
while we are at it, it seems strange to me that sage/interfaces/maxima_abstract.py contains a doctest for the maxima version used:
File "/localdisk/tmp/sage5.5/devel/sage12615b/sage/interfaces/maxima_abstract.py", line 422: sage: maxima.version() Expected: '5.26.0' Got: '5.29.1'
What really matters is the functionality provided, not the actual version?
Does Sage check the actual version of each component?
Paul
comment:4 in reply to: ↑ 3 Changed 20 months ago by dimpase
Replying to zimmerma:
while we are at it, it seems strange to me that sage/interfaces/maxima_abstract.py contains a doctest for the maxima version used:
File "/localdisk/tmp/sage5.5/devel/sage12615b/sage/interfaces/maxima_abstract.py", line 422: sage: maxima.version() Expected: '5.26.0' Got: '5.29.1'
seems that http://trac.sagemath.org/sage_trac/attachment/ticket/13364/doctests.2.patch
was not installed into the Sage installation you work with.
What really matters is the functionality provided, not the actual version?
Does Sage check the actual version of each component?
Sage provides versions at least for some other components, e.g. gap.version()
Sometimes it might be useful; AFAIK some optional packages check this.
Dima
Paul
comment:5 followup: ↓ 6 Changed 20 months ago by zimmerma
Dima, I don't object about providing a function maxima_version(), but checking it returns a particular value.
Paul
comment:6 in reply to: ↑ 5 Changed 20 months ago by dimpase
Replying to zimmerma:
Dima, I don't object about providing a function maxima_version(), but checking it returns a particular value.
well, one would like to test Sage functions... How else would you check that it works and returns something meaningful?
Paul
comment:7 Changed 20 months ago by zimmerma
we could for example check that the version is of the form x.y.z, or is greater than or equal to a given number.
Paul
comment:8 Changed 17 months ago by zimmerma
see #14306 (issue fixed in Maxima git)
Paul
comment:9 Changed 17 months ago by kcrisman
Also note that Maxima is now at 5.30.0.
comment:10 Changed 17 months ago by fbissey
Yes it is at 5.30.0. No check from sageongentoo on whether it works so far because the maxima ecl library building is currently broken for that particular version  an adsf upgrade problem in gentoo.
I second checking the form of the version returned rather than the value. That would make my life easier when R, maxima are upgraded in sog independently of sage itself (and without side effects at leat in doctests).
comment:11 Changed 16 months ago by Snark
 Cc Snark added
comment:12 Changed 16 months ago by jdemeyer
I'll try making a new Maxima spkg just to see what happpens...
comment:13 Changed 16 months ago by jdemeyer
 Description modified (diff)
 Milestone changed from sage5.10 to sage5.11
comment:14 Changed 16 months ago by jdemeyer
 Description modified (diff)
comment:15 Changed 16 months ago by jdemeyer
This looks problematic:
sage t devel/sage/sage/calculus/calculus.py ********************************************************************** File "devel/sage/sage/calculus/calculus.py", line 145, in sage.calculus.calculus Failed example: e^M Exception raised: Traceback (most recent call last): File "/mazur/release/merger/sage5.10.rc2/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 466, in _run self.execute(example, compiled, test.globs) File "/mazur/release/merger/sage5.10.rc2/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 825, in execute exec compiled in globs File "<doctest sage.calculus.calculus[32]>", line 1, in <module> e**M File "constants_c.pyx", line 275, in sage.symbolic.constants_c.E.__pow__ (sage/symbolic/constants_c.cpp:2779) File "matrix_symbolic_dense.pyx", line 387, in sage.matrix.matrix_symbolic_dense.Matrix_symbolic_dense.exp (sage/matrix/matrix_symbolic_dense.c:3433) File "/mazur/release/merger/sage5.10.rc2/local/lib/python2.7/sitepackages/sage/interfaces/interface.py", line 199, in __call__ return cls(self, x, name=name) File "/mazur/release/merger/sage5.10.rc2/local/lib/python2.7/sitepackages/sage/interfaces/interface.py", line 626, in __init__ raise TypeError, x TypeError: ECL says: Error executing code in Maxima: Unable to find the spectral representation **********************************************************************
comment:16 Changed 16 months ago by jdemeyer
Also
sage t devel/sage/sage/interfaces/maxima.py # Time out
due to a timeout of
sage: maxima('limit(x^a,x,0)')
This is fixable, it is because of a slightly different output format of assumptions.
comment:17 Changed 16 months ago by jdemeyer
 Report Upstream changed from Fixed upstream, but not in a stable release. to N/A
comment:18 Changed 16 months ago by kcrisman
That is awesome, thanks for checking this, Jeroen.
comment:19 Changed 15 months ago by kcrisman
This is fixable, it is because of a slightly different output format of assumptions.
Jeroen, while it's true that the error is slightly different in format, I don't think this is the problem (or at least not the whole one). The other things are still passing doctests, and we never even get to the
if i > 0: v = self._expect.before
branch, as one can verify by inserting print statements.
This looks problematic:
This is #2400 again, to some extent. Not sure why this doesn't work any more. In Maxima 5.29.1, this is fine.
Maxima 5.29.1 http://maxima.sourceforge.net using Lisp ECL 12.12.1 Distributed under the GNU Public License. See the file COPYING. Dedicated to the memory of William Schelter. The function bug_report() provides bug reporting information. (%i1) m: matrix([%i*%pi]); (%o1) [ %i %pi ] (%i4) matrixexp(m), keepfloat:true; (%o4)  1 (%i5) matrixexp(m),keepfloat:false; (%o5)  1
which is good. In 5.30, though, we have
(%i1) m: matrix([%i*%pi]); (%o1) [ %i %pi ] (%i6) matrixexp(m),keepfloat:true; Unable to find the spectral representation  an error. To debug this try: debugmode(true); (%i7) matrixexp(m),keepfloat:false; Unable to find the spectral representation  an error. To debug this try: debugmode(true);
comment:20 Changed 15 months ago by kcrisman
 Keywords sd48 added
comment:21 Changed 15 months ago by kcrisman
Note that the spkg changes are not committed.
comment:22 Changed 15 months ago by kcrisman
 Dependencies #13364 deleted
 Report Upstream changed from N/A to Reported upstream. Developers acknowledge bug.
Thanks to Barton Willis for this workaround.
This bug is due to circa January 2013 changes to matrix inversion. A workaround is to set ratmx to true: Maxima branch_5_30_base_98_g29f9239_dirty http://maxima.sourceforge.net using Lisp Clozure Common Lisp Version 1.9r15764 (WindowsX8632) (%i1) matrixexp(matrix([%i*%pi])); Unable to find the spectral representation (%i2) matrixexp(matrix([%i*%pi])), ratmx=true; (%o2)  1
I've reported this here. I don't know if doing this ratmx thing universally is good, though:
if FALSE will cause determinant and matrix addition, subtraction, and multiplication to be performed in the representation of the matrix elements and will cause the result of matrix inversion to be left in general representation. If it is TRUE, the 4 operations mentioned above will be performed in CRE form and the result of matrix inverse will be in CRE form. Note that this may cause the elements to be expanded (depending on the setting of RATFAC) which might not always be desired.
comment:23 Changed 15 months ago by kcrisman
 Description modified (diff)
comment:24 Changed 13 months ago by jdemeyer
 Milestone changed from sage5.11 to sage5.12
comment:25 Changed 13 months ago by kcrisman
5.31 has been released.
comment:26 Changed 8 months ago by vbraun_spam
 Milestone changed from sage6.1 to sage6.2
comment:27 Changed 5 months ago by vbraun_spam
 Milestone changed from sage6.2 to sage6.3
comment:28 Changed 4 months ago by pbruin
Maxima is now at version 5.33.0. The bug described in comments 15, 19 and 22 still exists.
comment:29 followup: ↓ 30 Changed 4 months ago by pbruin
I found a patch that I think has the same effect as the workaround in comment:22. I don't have a sourceforge account, so here it is in case there are any Maxima experts here who want to check it and (if it is any good) maybe forward it to the Maxima bug tracking system:

share/linearalgebra/matrixexp.lisp
a b 138 138 (print `(ratvars = ,$ratvars gcd = '$gcd algebraic = ,$algebraic)) 139 139 (print `(ratfac = ,$ratfac)) 140 140 (merror "Unable to find the spectrum"))) 141 142 (setq res ($fullratsimp ( ncpower (sub (mult z ($ident n)) mat) 1) z))141 142 (setq res ($fullratsimp ($invert (sub (mult z ($ident n)) mat) '$crering) z)) 143 143 (setq m (length sp)) 144 144 (dotimes (i m) 145 145 (setq zi (nth i sp))
I'll try to see if upgrading to 5.33.0 gives any further unexpected problems.
comment:30 in reply to: ↑ 29 Changed 4 months ago by kcrisman
I found a patch that I think has the same effect as the workaround in comment:22. I don't have a sourceforge account, so here it is in case there are any Maxima experts here who want to check it and (if it is any good) maybe forward it to the Maxima bug tracking system:
I have added it to the bug report in question.
I'll try to see if upgrading to 5.33.0 gives any further unexpected problems.
Let's hope not!
comment:31 Changed 4 months ago by pbruin
Thanks! No major problems so far; there is one integration doctest that does not finish, but Maxima itself returns the answer quickly, so the timeout has to occur in the interface.
comment:32 followup: ↓ 33 Changed 4 months ago by pbruin
The problematic doctest is the following one from sage/misc/functional.py:
sage: f = exp(x) * sinh(sqrt(x)) sage: t = integrate(f, x, 0, Infinity)
With an unpatched Sage 6.2 this does finish, but takes a very long time (23 minutes). The same situation occurs for
sage: maxima('integrate(exp(x)*sinh(sqrt(x)), x, 0, inf);')
This is surprising given that the following takes less than a second in Maxima (both the one that comes with Sage and a standalone copy, and both versions 5.29.1 and 5.33.0):
(%i1) integrate(exp(x)*sinh(sqrt(x)), x, 0, inf); (%o1) 1/4 1 %e (2 gamma_incomplete(1, 1)  gamma_incomplete(, 1)  sqrt(%pi)  2) 2   4 1/4 1 1/4 %e gamma_incomplete(, 1) %e gamma_incomplete(1, 1) 2 +    2 4 1/4 1/4 %e sqrt(%pi) %e +    4 2
It would seem to be an interface problem that already exists in some form in Sage 6.2 and manifests itself more relentlessly with the Maxima 5.33.0 spkg I am making.
comment:33 in reply to: ↑ 32 Changed 4 months ago by dimpase
Replying to pbruin:
The problematic doctest is the following one from sage/misc/functional.py:
sage: f = exp(x) * sinh(sqrt(x)) sage: t = integrate(f, x, 0, Infinity)With an unpatched Sage 6.2 this does finish, but takes a very long time (23 minutes). The same situation occurs for
sage: maxima('integrate(exp(x)*sinh(sqrt(x)), x, 0, inf);')This is surprising given that the following takes less than a second in Maxima (both the one that comes with Sage and a standalone copy, and both versions 5.29.1 and 5.33.0):
I am not at all convinced it is the interface problem. It is more likely the Maxima packages loaded by default in Sage library (see the corr. Sage source files) and not loaded in your test run. When I only load(abs_integrate) i Sage's 6.2 Maxima, or don't load anything at all, I get an almost instant answer.
When I load the whole bunch from lines 1424 of sage/interfaces/maxima_lib.py
init_code = ['display2d : false', 'domain : complex', 'keepfloat : true', 'load(to_poly_solve)', 'load(simplify_sum)', 'load(abs_integrate)']
it goes much slower.
comment:34 Changed 4 months ago by pbruin
Thanks! In fact it seems to be the domain : complex setting that is causing the slowness.
comment:35 followup: ↓ 36 Changed 4 months ago by pbruin
How about the minimal workaround of temporarily setting domain: real in this doctest?
comment:36 in reply to: ↑ 35 Changed 4 months ago by dimpase
Replying to pbruin:
How about the minimal workaround of temporarily setting domain: real in this doctest?
sure, this is not the 1st time something like this is needed.
comment:37 Changed 4 months ago by pbruin
 Branch set to u/pbruin/13973maxima_update
 Commit set to 13d48c244fa349d456ecd74cc426931433266102
 Description modified (diff)
 Report Upstream changed from Reported upstream. Developers acknowledge bug. to Completely fixed; Fix reported upstream
 Status changed from new to needs_review
 Summary changed from Upgrade Maxima past the release 5.29.1 to Upgrade Maxima to 5.33.0
comment:38 followup: ↓ 49 Changed 4 months ago by pbruin
 Description modified (diff)
Note about the patches: maxima_bug_2526.patch, which fixes http://sourceforge.net/p/maxima/bugs/2526/, is still necessary even though the bug is marked "closed" in the Maxima bug tracking system; the commit somehow didn't make it into the Maxima repository.
comment:39 Changed 4 months ago by pbruin
 Description modified (diff)
comment:40 Changed 4 months ago by pbruin
 Description modified (diff)
comment:41 Changed 4 months ago by mmezzarobba
 Status changed from needs_review to needs_work
sage t long src/sage/interfaces/maxima_lib.py ********************************************************************** File "src/sage/interfaces/maxima_lib.py", line 702, in sage.interfaces.maxima_lib.MaximaLib.sr_integral Failed example: integrate(cos(x + abs(x)), x) Expected: 1/4*(2*x  sin(2*x))*realpart(sgn(x)) + 1/2*x + 1/4*sin(2*x) Got: 1/4*(2*x  sin(2*x))*real_part(sgn(x)) + 1/2*x + 1/4*sin(2*x) **********************************************************************
This is with Volker's current develop branch on github (d409685), which includes a few dozens new tickets compared to 6.3.beta1.
comment:42 Changed 4 months ago by git
 Commit changed from 13d48c244fa349d456ecd74cc426931433266102 to a130eed39a9e2e86301e94d9e16932fd38be64df
Branch pushed to git repo; I updated commit sha1. New commits:
a130eed  Trac 13973: realpart should be real_part 
comment:43 followup: ↓ 44 Changed 4 months ago by pbruin
 Status changed from needs_work to needs_review
Thanks, that must be a consequence of #16224.
comment:44 in reply to: ↑ 43 Changed 4 months ago by kcrisman
Thanks, that must be a consequence of #16224.
So... did we have an incorrect translation of Maxima's realpart all along? Interesting.
comment:45 followup: ↓ 46 Changed 4 months ago by mmezzarobba
Is there a reason for no longer using z... in the following test?
@@ 9231,7 +9231,7 @@ cdef class Expression(CommutativeRingElement): sage: from sage.calculus.calculus import maxima sage: sol = maxima(cos(x)==0).to_poly_solve(x) sage: sol.sage()  [[x == 1/2*pi + 2*pi*z...], [x == 1/2*pi + 2*pi*z...]] + [[x == 1/2*pi + pi*z82]] If a returned unsolved expression has a denominator, but the original one did not, this may also be true::
The same comment may apply to z[09]+ in
src/sage/interfaces/maxima_lib.py: [[x == pi*z54]] src/sage/symbolic/relation.py: [[x == 1/4*pi + pi*z80, y == 1/4*pi  pi*z80]]
The bugs listed in the description indeed appear to be fixed with the update, except #13733, for which I get:
sage: integral(log(cot(x)1),x,0,pi/4) ... RuntimeError: ECL says: Error executing code in Maxima: PSLOG: internal error.
Otherwise looks good to me.
comment:46 in reply to: ↑ 45 ; followup: ↓ 47 Changed 4 months ago by pbruin
 Description modified (diff)
Replying to mmezzarobba:
Is there a reason for no longer using z... in the following test?
Mostly readability (since the tests are also part of the documentation); with the ... it is not clear that it is just a "serial number" and not part of a larger expression that has been left out. Of course it is somewhat random, so it may change once in a while with Maxima upgrades or changes in other doctests in the same file. I personally think displaying a complete answer is worth the trouble of having to change it every now and then, but if you insist I can change all such examples to ....
The bugs listed in the description indeed appear to be fixed with the update, except #13733, for which I get:
sage: integral(log(cot(x)1),x,0,pi/4) ... RuntimeError: ECL says: Error executing code in Maxima: PSLOG: internal error.
You're right; I tested this inside Maxima (sage maxima), but Sage loads additional packages. In this case the failure is caused by loading abs_integrate. Using 5.33.0:
$ sage maxima ... (%i1) display2d: false$ (%i2) integrate(log(cot(x)1),x,0,%pi/4); (%o2) (%i*li[2]((%i+1)/2)%i*li[2]((%i1)/2))/2 (%i*(2*li[2](%i+1)2*li[2](1%i))+%pi*log(2))/4 (%i3) load(abs_integrate)$ (%i4) integrate(log(cot(x)1),x,0,%pi/4); log: encountered log(0). log: encountered log(0). log: encountered log(0). log: encountered log(0). log: encountered log(0). log: encountered log(0). log: encountered log(0). log: encountered log(0). log: encountered log(0). PSLOG: internal error.  an error. To debug this try: debugmode(true);
comment:47 in reply to: ↑ 46 Changed 4 months ago by mmezzarobba
 Report Upstream changed from Completely fixed; Fix reported upstream to N/A
 Reviewers set to Marc Mezzarobba
 Status changed from needs_review to positive_review
Replying to pbruin:
Replying to mmezzarobba:
Is there a reason for no longer using z... in the following test?
Mostly readability (since the tests are also part of the documentation); with the ... it is not clear that it is just a "serial number" and not part of a larger expression that has been left out. Of course it is somewhat random, so it may change once in a while with Maxima upgrades or changes in other doctests in the same file. I personally think displaying a complete answer is worth the trouble of having to change it every now and then, but if you insist I can change all such examples to ....
No, that's fine with me, I just wanted to make sure that it was intentional.
comment:48 Changed 4 months ago by vbraun
 Branch changed from u/pbruin/13973maxima_update to a130eed39a9e2e86301e94d9e16932fd38be64df
 Resolution set to fixed
 Status changed from positive_review to closed
comment:49 in reply to: ↑ 38 Changed 4 months ago by kcrisman
 Commit a130eed39a9e2e86301e94d9e16932fd38be64df deleted
Note about the patches: maxima_bug_2526.patch, which fixes http://sourceforge.net/p/maxima/bugs/2526/, is still necessary even though the bug is marked "closed" in the Maxima bug tracking system; the commit somehow didn't make it into the Maxima repository.
Okay, this is finally in Maxima, so the next Maxima upgrade will get to remove this patch.
comment:50 Changed 4 months ago by kcrisman
#14306 needs review.
So for instance going at least as far as the fix for #13733 would be good.