Opened 22 months ago

Closed 5 months ago

Last modified 5 months ago

#13973 closed defect (fixed)

Upgrade Maxima to 5.33.0

Reported by: dimpase Owned by: tbd
Priority: major Milestone: sage-6.3
Component: packages: standard Keywords: sd48
Cc: Snark Merged in:
Authors: Peter Bruin Reviewers: Marc Mezzarobba
Report Upstream: N/A Work issues:
Branch: a130eed (Commits) Commit:
Dependencies: Stopgaps:

Description (last modified by pbruin)

This is a continuation of #13364, and aims at including more upstream bug fixes, which e.g. fix an
issue reported on sage-support,
which was reported as Maxima bug 2535, and marked there as closed in post-5.29.1.

Fixes #11894, #13526, #13712, #14209, #14306, #15386 and possibly other Maxima-related bugs.

Ought to fix #13733 (but loading the abs_integrate package currently causes this to fail).

Ought to fix #8728 (but Maxima asks for more constraints which follow from the existing ones).

Does not fix #14591, #14677, #15033.

tarball: http://homepages.warwick.ac.uk/staff/P.Bruin/sage/maxima-5.33.0.tar.bz2

Note about the existing maxima_bug_2526.patch: ​the bug was marked "closed" in the Maxima bug tracking system, but the commit somehow disappeared from the Maxima repository and the bug has been reopened. Hence this patch is still needed to fix it.

Change History (50)

comment:1 Changed 22 months ago by kcrisman

  • Dependencies changed from 13364 to #13364

comment:2 Changed 22 months ago by kcrisman

So for instance going at least as far as the fix for #13733 would be good.

comment:3 follow-up: Changed 22 months ago by zimmerma

while we are at it, it seems strange to me that sage/interfaces/maxima_abstract.py contains a doctest for the maxima version used:

File "/localdisk/tmp/sage-5.5/devel/sage-12615b/sage/interfaces/maxima_abstract.py", line 422:
    sage: maxima.version()
Expected:
    '5.26.0'
Got:
    '5.29.1'

What really matters is the functionality provided, not the actual version?
Does Sage check the actual version of each component?

Paul

comment:4 in reply to: ↑ 3 Changed 22 months ago by dimpase

Replying to zimmerma:

while we are at it, it seems strange to me that sage/interfaces/maxima_abstract.py contains a doctest for the maxima version used:

File "/localdisk/tmp/sage-5.5/devel/sage-12615b/sage/interfaces/maxima_abstract.py", line 422:
    sage: maxima.version()
Expected:
    '5.26.0'
Got:
    '5.29.1'

seems that http://trac.sagemath.org/sage_trac/attachment/ticket/13364/doctests.2.patch
was not installed into the Sage installation you work with.

What really matters is the functionality provided, not the actual version?
Does Sage check the actual version of each component?

Sage provides versions at least for some other components, e.g. gap.version()

Sometimes it might be useful; AFAIK some optional packages check this.

Dima

Paul

comment:5 follow-up: Changed 22 months ago by zimmerma

Dima, I don't object about providing a function maxima_version(), but checking it returns a particular value.

Paul

comment:6 in reply to: ↑ 5 Changed 21 months ago by dimpase

Replying to zimmerma:

Dima, I don't object about providing a function maxima_version(), but checking it returns a particular value.

well, one would like to test Sage functions... How else would you check that it works and returns something meaningful?

Paul

comment:7 Changed 21 months ago by zimmerma

we could for example check that the version is of the form x.y.z, or is greater than or equal to a given number.

Paul

comment:8 Changed 18 months ago by zimmerma

see #14306 (issue fixed in Maxima git)

Paul

comment:9 Changed 18 months ago by kcrisman

Also note that Maxima is now at 5.30.0.

comment:10 Changed 18 months ago by fbissey

Yes it is at 5.30.0. No check from sage-on-gentoo on whether it works so far because the maxima ecl library building is currently broken for that particular version - an adsf upgrade problem in gentoo.

I second checking the form of the version returned rather than the value. That would make my life easier when R, maxima are upgraded in s-o-g independently of sage itself (and without side effects at leat in doctests).

comment:11 Changed 18 months ago by Snark

  • Cc Snark added

comment:12 Changed 17 months ago by jdemeyer

I'll try making a new Maxima spkg just to see what happpens...

comment:13 Changed 17 months ago by jdemeyer

  • Authors set to Jeroen Demeyer
  • Description modified (diff)
  • Milestone changed from sage-5.10 to sage-5.11

comment:14 Changed 17 months ago by jdemeyer

  • Description modified (diff)

comment:15 Changed 17 months ago by jdemeyer

This looks problematic:

sage -t devel/sage/sage/calculus/calculus.py
**********************************************************************
File "devel/sage/sage/calculus/calculus.py", line 145, in sage.calculus.calculus
Failed example:
    e^M
Exception raised:
    Traceback (most recent call last):
      File "/mazur/release/merger/sage-5.10.rc2/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 466, in _run
        self.execute(example, compiled, test.globs)
      File "/mazur/release/merger/sage-5.10.rc2/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 825, in execute
        exec compiled in globs
      File "<doctest sage.calculus.calculus[32]>", line 1, in <module>
        e**M
      File "constants_c.pyx", line 275, in sage.symbolic.constants_c.E.__pow__ (sage/symbolic/constants_c.cpp:2779)
      File "matrix_symbolic_dense.pyx", line 387, in sage.matrix.matrix_symbolic_dense.Matrix_symbolic_dense.exp (sage/matrix/matrix_symbolic_dense.c:3433)
      File "/mazur/release/merger/sage-5.10.rc2/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 199, in __call__
        return cls(self, x, name=name)
      File "/mazur/release/merger/sage-5.10.rc2/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 626, in __init__
        raise TypeError, x
    TypeError: ECL says: Error executing code in Maxima: Unable to find the spectral representation
**********************************************************************

comment:16 Changed 17 months ago by jdemeyer

Also

sage -t devel/sage/sage/interfaces/maxima.py  # Time out

due to a timeout of

sage: maxima('limit(x^a,x,0)')

This is fixable, it is because of a slightly different output format of assumptions.

Last edited 17 months ago by jdemeyer (previous) (diff)

comment:17 Changed 17 months ago by jdemeyer

  • Report Upstream changed from Fixed upstream, but not in a stable release. to N/A

comment:18 Changed 17 months ago by kcrisman

That is awesome, thanks for checking this, Jeroen.

comment:19 Changed 17 months ago by kcrisman

This is fixable, it is because of a slightly different output format of assumptions.

Jeroen, while it's true that the error is slightly different in format, I don't think this is the problem (or at least not the whole one). The other things are still passing doctests, and we never even get to the

           if i > 0:
                v = self._expect.before

branch, as one can verify by inserting print statements.

This looks problematic:

This is #2400 again, to some extent. Not sure why this doesn't work any more. In Maxima 5.29.1, this is fine.

Maxima 5.29.1 http://maxima.sourceforge.net
using Lisp ECL 12.12.1
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) m: matrix([%i*%pi]);
(%o1)                             [ %i %pi ]
(%i4) matrixexp(m), keepfloat:true;
(%o4)                                 - 1
(%i5) matrixexp(m),keepfloat:false;
(%o5)                                 - 1

which is good. In 5.30, though, we have

(%i1) m: matrix([%i*%pi]);
(%o1)                             [ %i %pi ]
(%i6) matrixexp(m),keepfloat:true;

Unable to find the spectral representation
 -- an error. To debug this try: debugmode(true);
(%i7) matrixexp(m),keepfloat:false;

Unable to find the spectral representation
 -- an error. To debug this try: debugmode(true);

comment:20 Changed 17 months ago by kcrisman

  • Keywords sd48 added

comment:21 Changed 17 months ago by kcrisman

Note that the spkg changes are not committed.

comment:22 Changed 17 months ago by kcrisman

  • Dependencies #13364 deleted
  • Report Upstream changed from N/A to Reported upstream. Developers acknowledge bug.

Thanks to Barton Willis for this workaround.

This bug is due to circa January 2013 changes to matrix inversion. A workaround is to set ratmx to true:

Maxima branch_5_30_base_98_g29f9239_dirty http://maxima.sourceforge.net
using Lisp Clozure Common Lisp Version 1.9-r15764  (WindowsX8632)

(%i1) matrixexp(matrix([%i*%pi]));

Unable to find the spectral representation

(%i2) matrixexp(matrix([%i*%pi])), ratmx=true;
(%o2)                                 - 1

I've reported this here. I don't know if doing this ratmx thing universally is good, though:

 if FALSE will cause determinant and matrix addition, subtraction, and multiplication to be performed in the representation of the matrix elements and will cause the result of matrix inversion to be left in general representation. If it is TRUE, the 4 operations mentioned above will be performed in CRE form and the result of matrix inverse will be in CRE form. Note that this may cause the elements to be expanded (depending on the setting of RATFAC) which might not always be desired.

comment:23 Changed 16 months ago by kcrisman

  • Description modified (diff)

comment:24 Changed 15 months ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:25 Changed 14 months ago by kcrisman

5.31 has been released.

comment:26 Changed 9 months ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:27 Changed 6 months ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:28 Changed 6 months ago by pbruin

Maxima is now at version 5.33.0. The bug described in comments 15, 19 and 22 still exists.

comment:29 follow-up: Changed 6 months ago by pbruin

I found a patch that I think has the same effect as the workaround in comment:22. I don't have a sourceforge account, so here it is in case there are any Maxima experts here who want to check it and (if it is any good) maybe forward it to the Maxima bug tracking system:

  • share/linearalgebra/matrixexp.lisp

    a b  
    138138          (print `(ratvars = ,$ratvars gcd = '$gcd algebraic = ,$algebraic)) 
    139139          (print `(ratfac = ,$ratfac)) 
    140140          (merror "Unable to find the spectrum"))) 
    141     
    142     (setq res ($fullratsimp (ncpower (sub (mult z ($ident n)) mat) -1) z)) 
     141 
     142    (setq res ($fullratsimp ($invert (sub (mult z ($ident n)) mat) '$crering) z)) 
    143143    (setq m (length sp)) 
    144144    (dotimes (i m) 
    145145      (setq zi (nth i sp)) 

I'll try to see if upgrading to 5.33.0 gives any further unexpected problems.

comment:30 in reply to: ↑ 29 Changed 6 months ago by kcrisman

I found a patch that I think has the same effect as the workaround in comment:22. I don't have a sourceforge account, so here it is in case there are any Maxima experts here who want to check it and (if it is any good) maybe forward it to the Maxima bug tracking system:

I have added it to the bug report in question.

I'll try to see if upgrading to 5.33.0 gives any further unexpected problems.

Let's hope not!

comment:31 Changed 6 months ago by pbruin

Thanks! No major problems so far; there is one integration doctest that does not finish, but Maxima itself returns the answer quickly, so the timeout has to occur in the interface.

comment:32 follow-up: Changed 6 months ago by pbruin

The problematic doctest is the following one from sage/misc/functional.py:

sage: f = exp(-x) * sinh(sqrt(x))
sage: t = integrate(f, x, 0, Infinity)

With an unpatched Sage 6.2 this does finish, but takes a very long time (2-3 minutes). The same situation occurs for

sage: maxima('integrate(exp(-x)*sinh(sqrt(x)), x, 0, inf);')

This is surprising given that the following takes less than a second in Maxima (both the one that comes with Sage and a standalone copy, and both versions 5.29.1 and 5.33.0):

(%i1) integrate(exp(-x)*sinh(sqrt(x)), x, 0, inf); 
(%o1) 
    1/4                                              1
  %e    (2 gamma_incomplete(1, 1) - gamma_incomplete(-, 1) - sqrt(%pi) - 2)
                                                     2
- -------------------------------------------------------------------------
                                      4
                                    1/4                  1
     1/4                          %e    gamma_incomplete(-, 1)
   %e    gamma_incomplete(1, 1)                          2
 + ---------------------------- - ----------------------------
                2                              4
     1/4               1/4
   %e    sqrt(%pi)   %e
 + --------------- - -----
          4            2

It would seem to be an interface problem that already exists in some form in Sage 6.2 and manifests itself more relentlessly with the Maxima 5.33.0 spkg I am making.

comment:33 in reply to: ↑ 32 Changed 6 months ago by dimpase

Replying to pbruin:

The problematic doctest is the following one from sage/misc/functional.py:

sage: f = exp(-x) * sinh(sqrt(x))
sage: t = integrate(f, x, 0, Infinity)

With an unpatched Sage 6.2 this does finish, but takes a very long time (2-3 minutes). The same situation occurs for

sage: maxima('integrate(exp(-x)*sinh(sqrt(x)), x, 0, inf);')

This is surprising given that the following takes less than a second in Maxima (both the one that comes with Sage and a standalone copy, and both versions 5.29.1 and 5.33.0):

I am not at all convinced it is the interface problem. It is more likely the Maxima packages loaded by default in Sage library (see the corr. Sage source files) and not loaded in your test run. When I only load(abs_integrate) i Sage's 6.2 Maxima, or don't load anything at all, I get an almost instant answer.
When I load the whole bunch from lines 142-4 of sage/interfaces/maxima_lib.py

init_code = ['display2d : false', 'domain : complex', 'keepfloat : true',
            'load(to_poly_solve)', 'load(simplify_sum)',
            'load(abs_integrate)']

it goes much slower.

comment:34 Changed 6 months ago by pbruin

Thanks! In fact it seems to be the domain : complex setting that is causing the slowness.

comment:35 follow-up: Changed 6 months ago by pbruin

How about the minimal workaround of temporarily setting domain: real in this doctest?

comment:36 in reply to: ↑ 35 Changed 6 months ago by dimpase

Replying to pbruin:

How about the minimal workaround of temporarily setting domain: real in this doctest?

sure, this is not the 1st time something like this is needed.

comment:37 Changed 6 months ago by pbruin

  • Authors changed from Jeroen Demeyer to Peter Bruin
  • Branch set to u/pbruin/13973-maxima_update
  • Commit set to 13d48c244fa349d456ecd74cc426931433266102
  • Description modified (diff)
  • Report Upstream changed from Reported upstream. Developers acknowledge bug. to Completely fixed; Fix reported upstream
  • Status changed from new to needs_review
  • Summary changed from Upgrade Maxima past the release 5.29.1 to Upgrade Maxima to 5.33.0

comment:38 follow-up: Changed 6 months ago by pbruin

  • Description modified (diff)

Note about the patches: maxima_bug_2526.patch, which fixes http://sourceforge.net/p/maxima/bugs/2526/, is still necessary even though the bug is marked "closed" in the Maxima bug tracking system; the commit somehow didn't make it into the Maxima repository.

comment:39 Changed 6 months ago by pbruin

  • Description modified (diff)

comment:40 Changed 5 months ago by pbruin

  • Description modified (diff)

comment:41 Changed 5 months ago by mmezzarobba

  • Status changed from needs_review to needs_work
sage -t --long src/sage/interfaces/maxima_lib.py
**********************************************************************
File "src/sage/interfaces/maxima_lib.py", line 702, in sage.interfaces.maxima_lib.MaximaLib.sr_integral
Failed example:
    integrate(cos(x + abs(x)), x)
Expected:
    -1/4*(2*x - sin(2*x))*realpart(sgn(x)) + 1/2*x + 1/4*sin(2*x)
Got:
    -1/4*(2*x - sin(2*x))*real_part(sgn(x)) + 1/2*x + 1/4*sin(2*x)
**********************************************************************

This is with Volker's current develop branch on github (d409685), which includes a few dozens new tickets compared to 6.3.beta1.

comment:42 Changed 5 months ago by git

  • Commit changed from 13d48c244fa349d456ecd74cc426931433266102 to a130eed39a9e2e86301e94d9e16932fd38be64df

Branch pushed to git repo; I updated commit sha1. New commits:

a130eedTrac 13973: realpart should be real_part

comment:43 follow-up: Changed 5 months ago by pbruin

  • Status changed from needs_work to needs_review

Thanks, that must be a consequence of #16224.

comment:44 in reply to: ↑ 43 Changed 5 months ago by kcrisman

Thanks, that must be a consequence of #16224.

So... did we have an incorrect translation of Maxima's realpart all along? Interesting.

comment:45 follow-up: Changed 5 months ago by mmezzarobba

Is there a reason for no longer using z... in the following test?

@@ -9231,7 +9231,7 @@ cdef class Expression(CommutativeRingElement):
             sage: from sage.calculus.calculus import maxima
             sage: sol = maxima(cos(x)==0).to_poly_solve(x)
             sage: sol.sage()
-            [[x == -1/2*pi + 2*pi*z...], [x == 1/2*pi + 2*pi*z...]]
+            [[x == 1/2*pi + pi*z82]]
 
         If a returned unsolved expression has a denominator, but the
         original one did not, this may also be true::

The same comment may apply to z[0-9]+ in

src/sage/interfaces/maxima_lib.py:            [[x == pi*z54]]
src/sage/symbolic/relation.py:        [[x == 1/4*pi + pi*z80, y == -1/4*pi - pi*z80]]

The bugs listed in the description indeed appear to be fixed with the update, except #13733, for which I get:

sage: integral(log(cot(x)-1),x,0,pi/4)
...
RuntimeError: ECL says: Error executing code in Maxima: PSLOG: internal error.

Otherwise looks good to me.

Last edited 5 months ago by mmezzarobba (previous) (diff)

comment:46 in reply to: ↑ 45 ; follow-up: Changed 5 months ago by pbruin

  • Description modified (diff)

Replying to mmezzarobba:

Is there a reason for no longer using z... in the following test?

Mostly readability (since the tests are also part of the documentation); with the ... it is not clear that it is just a "serial number" and not part of a larger expression that has been left out. Of course it is somewhat random, so it may change once in a while with Maxima upgrades or changes in other doctests in the same file. I personally think displaying a complete answer is worth the trouble of having to change it every now and then, but if you insist I can change all such examples to ....

The bugs listed in the description indeed appear to be fixed with the update, except #13733, for which I get:

sage: integral(log(cot(x)-1),x,0,pi/4)
...
RuntimeError: ECL says: Error executing code in Maxima: PSLOG: internal error.

You're right; I tested this inside Maxima (sage --maxima), but Sage loads additional packages. In this case the failure is caused by loading abs_integrate. Using 5.33.0:

$ sage --maxima
...
(%i1) display2d: false$

(%i2) integrate(log(cot(x)-1),x,0,%pi/4);

(%o2) (%i*li[2]((%i+1)/2)-%i*li[2](-(%i-1)/2))/2
 -(%i*(2*li[2](%i+1)-2*li[2](1-%i))+%pi*log(2))/4
(%i3) load(abs_integrate)$

(%i4) integrate(log(cot(x)-1),x,0,%pi/4);

log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
log: encountered log(0).
PSLOG: internal error.
 -- an error. To debug this try: debugmode(true);

comment:47 in reply to: ↑ 46 Changed 5 months ago by mmezzarobba

  • Report Upstream changed from Completely fixed; Fix reported upstream to N/A
  • Reviewers set to Marc Mezzarobba
  • Status changed from needs_review to positive_review

Replying to pbruin:

Replying to mmezzarobba:

Is there a reason for no longer using z... in the following test?

Mostly readability (since the tests are also part of the documentation); with the ... it is not clear that it is just a "serial number" and not part of a larger expression that has been left out. Of course it is somewhat random, so it may change once in a while with Maxima upgrades or changes in other doctests in the same file. I personally think displaying a complete answer is worth the trouble of having to change it every now and then, but if you insist I can change all such examples to ....

No, that's fine with me, I just wanted to make sure that it was intentional.

comment:48 Changed 5 months ago by vbraun

  • Branch changed from u/pbruin/13973-maxima_update to a130eed39a9e2e86301e94d9e16932fd38be64df
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:49 in reply to: ↑ 38 Changed 5 months ago by kcrisman

  • Commit a130eed39a9e2e86301e94d9e16932fd38be64df deleted

Note about the patches: maxima_bug_2526.patch, which fixes http://sourceforge.net/p/maxima/bugs/2526/, is still necessary even though the bug is marked "closed" in the Maxima bug tracking system; the commit somehow didn't make it into the Maxima repository.

Okay, this is finally in Maxima, so the next Maxima upgrade will get to remove this patch.

comment:50 Changed 5 months ago by kcrisman

#14306 needs review.

Note: See TracTickets for help on using tickets.