Opened 7 years ago
Last modified 6 years ago
#18920 closed defect
upgrade Maxima to version >= 5.39 — at Version 89
Reported by:  Dima Pasechnik  Owned by:  

Priority:  major  Milestone:  sage7.6 
Component:  symbolics  Keywords:  
Cc:  Ralf Stephan, JeanPierre Flori, KarlDieter Crisman, Nils Bruin, Jeroen Demeyer, Volker Braun, Emmanuel Charpentier  Merged in:  
Authors:  Dima Pasechnik  Reviewers:  
Report Upstream:  Fixed upstream, in a later stable release.  Work issues:  fix doctests 
Branch:  public/t18920 (Commits, GitHub, GitLab)  Commit:  af7e5ff2e30dfed8bcb4c597403a6a23eb93b86a 
Dependencies:  #22191  Stopgaps: 
Description (last modified by )
Change History (89)
comment:1 Changed 7 years ago by
Cc:  Ralf Stephan added 

Description:  modified (diff) 
comment:2 Changed 7 years ago by
Description:  modified (diff) 

comment:3 Changed 7 years ago by
Description:  modified (diff) 

comment:4 Changed 7 years ago by
comment:5 Changed 7 years ago by
I tried maxima 5.36.0 and reverted (that was with ecl 15.3.7) , not tried 5.36.1 (5.36.0.1 whatever the version number is actually).
Wed Apr 29 11:14:49 2015 >>> scimathematics/maxima5.35.1r2 merge time: 5 minutes and 6 seconds. Fri May 1 10:33:58 2015 >>> scimathematics/maxima5.36.0 merge time: 4 minutes and 52 seconds. Fri May 1 10:42:03 2015 >>> scimathematics/maxima5.35.1r2 merge time: 5 minutes and 8 seconds.
comment:6 Changed 7 years ago by
Branch:  → u/dimpase/eclupdate 

Cc:  JeanPierre Flori added 
Commit:  → 045d21eeb356b04102696d1aa51776b1330c635a 
Description:  modified (diff) 
The branch is currently not working; to build the new ecl pkg, one has to move patches/implib.patch
out of the way; I tried porting the latter patch to the new ecl, but it produces a makefile error that I don't understand; anyway, it is cygwinspecific.
I cc its author now. J.P., any ideas?
comment:7 Changed 7 years ago by
for the record, I get
... config.status: creating ecl/config.h Configuration complete. To build ECL, issue make in this directory. cd build; make j1 make[1]: Entering directory `/home/dima/software/sage/local/var/tmp/sage/build/ecl15.3.7.p0/src/build' Makefile:192: *** commands commence before first target. Stop. make[1]: Leaving directory `/home/dima/software/sage/local/var/tmp/sage/build/ecl15.3.7.p0/src/build' make: *** [all] Error 2 Error  Failed to build ECL ... exiting
comment:8 Changed 7 years ago by
What I don't understand is whether some patches are in fact results of running autoconf/automake/aclocal on patched configure.ac, Makefile.in, etc.
If yes, then these should be split from the "real" ones to allow for fully automatic creation (and commands needed to run autotools need to be spelled out somewhere in the pkg docs). If no, then I don't see the point of patching configure.ac, Makefile.in, etc in the first place.
comment:9 followup: 13 Changed 7 years ago by
Look at spkg_src. autotools are not run. But may be the impl platch was copied from an autotool run. The gmp patch deals with the removal of gmp sources from the upstream tarball (the configure script of ecl checks for stuff in that subdirectory so those checks have to be diverted). the impl patch deals with cygwin stuff.
Getting real autoconf patch and running autotools wouldn't be a bad idea but that means you will need to have autotools installed to run spkgsrc. spkgsrc will need to be amended accordingly.
comment:10 followup: 11 Changed 7 years ago by
I notice that the gmp patch is against configure.ac
while implib is against configure.in
the last one is probably wrong.
comment:11 Changed 7 years ago by
Replying to fbissey:
I notice that the gmp patch is against
configure.ac
while implib is againstconfigure.in
the last one is probably wrong.
the name has changed in version 15.3.7.
comment:12 followup: 14 Changed 7 years ago by
I am trying to read the stuff from the git commit and of course it is patch of patch, it is not readable as this, you changed it already I see.
comment:13 followups: 15 28 Changed 7 years ago by
Replying to fbissey:
Look at spkg_src. autotools are not run. But may be the impl platch was copied from an autotool run. The gmp patch deals with the removal of gmp sources from the upstream tarball (the configure script of ecl checks for stuff in that subdirectory so those checks have to be diverted). the impl patch deals with cygwin stuff.
Getting real autoconf patch and running autotools wouldn't be a bad idea but that means you will need to have autotools installed to run spkgsrc. spkgsrc will need to be amended accordingly.
spkgsrc is outdated anyway.
comment:14 Changed 7 years ago by
Replying to fbissey:
I am trying to read the stuff from the git commit and of course it is patch of patch, it is not readable as this, you changed it already I see.
here they are in a more readable way: https://github.com/dimpase/ecl/commits/sagepatches
comment:15 Changed 7 years ago by
Replying to dimpase:
Replying to fbissey:
Look at spkg_src. autotools are not run. But may be the impl platch was copied from an autotool run. The gmp patch deals with the removal of gmp sources from the upstream tarball (the configure script of ecl checks for stuff in that subdirectory so those checks have to be diverted). the impl patch deals with cygwin stuff.
No, I mean that it is silly and errorprone to rebase by hand patches that are better obtained directly from diffs to output of autoconf/automake/aclocal.
The latter should not be bundled together with the "real" one.
comment:16 Changed 7 years ago by
I have opened https://gitlab.com/embeddablecommonlisp/ecl/issues/93
to hopefully sort out the ECL autotools mess.
comment:17 Changed 7 years ago by
Sorry I have been too busy today to look further into this. I can reproduce your problem with your branch on my mac. I haven't quite identified yet the guilty makefile. I think the problem is not in the top makefile but in one of the subfolders but I haven't identified which one yet.
comment:18 Changed 7 years ago by
To complete the answer you got from upstream: automake is not used (not a crime) but ecl also ship some of its dependencies in the pristine tarball and those come with their own build systems which can use automake. Apart from ffi
we don't want to care about these I think.
comment:19 Changed 7 years ago by
arrghh, that was me mixing of tabs and spaces in src/Makefile.in
patch...
Now all seems to work. I'll post an update soon.
OK, and for the record, src/configure
is generated by autoreconf, that is, one does not need to rebase the corresponding patch manually.
comment:20 Changed 7 years ago by
Commit:  045d21eeb356b04102696d1aa51776b1330c635a → 9fc27ca7ba877a83443c61d22542e49d65096404 

Branch pushed to git repo; I updated commit sha1. New commits:
9fc27ca  fixed broken Makefile.in patch (spaces and tabs don't mix), reorganised patches.

comment:21 Changed 7 years ago by
Cc:  KarlDieter Crisman added 

comment:22 Changed 7 years ago by
Commit:  9fc27ca7ba877a83443c61d22542e49d65096404 → 82fdf0eaa092b63cc98d97b418721795ea6b5081 

Branch pushed to git repo; I updated commit sha1. New commits:
82fdf0e  ported maxima patches; tarball needs bootstrapping; trivial doctests fix

comment:23 followup: 24 Changed 7 years ago by
Cc:  Nils Bruin added 

New Maxima output sometimes breaks Sage parser, e.g.
sage t src/sage/calculus/calculus.py ********************************************************************** File "src/sage/calculus/calculus.py", line 373, in sage.calculus.calculus Failed example: taylor(gamma(1/3+x),x,0,3) Exception raised: Traceback (most recent call last): File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 496, in _run self.compile_and_execute(example, compiler, test.globs) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 858, in compile_and_execute exec(compiled, globs) File "<doctest sage.calculus.calculus[100]>", line 1, in <module> taylor(gamma(Integer(1)/Integer(3)+x),x,Integer(0),Integer(3)) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/calculus/functional.py", line 378, in taylor return f.taylor(*args) File "sage/symbolic/expression.pyx", line 4038, in sage.symbolic.expression.Expression.taylor (/home/dima/software/sage/src/build/cythonized/sage/symbolic/expression.cpp:23671) return self.parent()(l) File "sage/structure/parent.pyx", line 1097, in sage.structure.parent.Parent.__call__ (/home/dima/software/sage/src/build/cythonized/sage/structure/parent.c:9546) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 237, in sage.structure.coerce_maps.NamedConvertMap._call_ (/home/dima/software/sage/src/build/cythonized/sage/structure/coerce_maps.c:5756) cdef Element e = method(C) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/interfaces/maxima_abstract.py", line 1251, in _symbolic_ return R(self._sage_()) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/interfaces/maxima_abstract.py", line 1226, in _sage_ maxima=self.parent()) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/calculus/calculus.py", line 1901, in symbolic_expression_from_maxima_string raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s) TypeError: unable to make sense of Maxima expression '(1/432)*(72*gamma(1/3)*(psi[2])(1/3)+((72)*euler_gamma^3+((36)*pi*3^(1/2)+(324)*log(3))*euler_gamma^2+((108)*log(3)*pi*3^(1/2)+(18)*pi^2+(486)*log(3)^2+(216)*(psi[1])(1/3))*euler_gamma+((1)*pi^3+((81)*log(3)^2+(36)*(psi[1])(1/3))*pi)*3^(1/2)+(27)*log(3)*pi^2+(243)*log(3)^3+(324)*(psi[1])(1/3)*log(3))*gamma(1/3))*_SAGE_VAR_x^3+(1/24)*((12*euler_gamma^2+(4*pi*3^(1/2)+36*log(3))*euler_gamma+6*log(3)*pi*3^(1/2)+pi^2+27*log(3)^2+12*(psi[1])(1/3))*gamma(1/3))*_SAGE_VAR_x^2+((1)/6)*((6*euler_gamma+pi*3^(1/2)+9*log(3))*gamma(1/3))*_SAGE_VAR_x+gamma(1/3)' in Sage
and
File "src/sage/calculus/calculus.py", line 1746, in sage.calculus.calculus.symbolic_expression_from_maxima_string Failed example: maxima('3*li[2](u)+8*li[33](exp(u))').sage() Exception raised: Traceback (most recent call last): File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 496, in _run self.compile_and_execute(example, compiler, test.globs) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 858, in compile_and_execute exec(compiled, globs) File "<doctest sage.calculus.calculus.symbolic_expression_from_maxima_string[6]>", line 1, in <module> maxima('3*li[2](u)+8*li[33](exp(u))').sage() File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/interfaces/interface.py", line 1016, in sage return self._sage_(*args, **kwds) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/interfaces/maxima_abstract.py", line 1226, in _sage_ maxima=self.parent()) File "/home/dima/software/sage/local/lib/python2.7/sitepackages/sage/calculus/calculus.py", line 1901, in symbolic_expression_from_maxima_string raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s) TypeError: unable to make sense of Maxima expression '8*(li[33])(e^u)+3*(li[2])(u)' in Sage
Is it easy to fix?
comment:24 followup: 34 Changed 7 years ago by
Replying to dimpase:
TypeError: unable to make sense of Maxima expression '(1/432)*(72*gamma(1/3)*(psi[2])(1/3)+((72)*euler_gamma^3+((36)*pi*3^(1/2)+(324)*log(3))*euler_gamma^2+((108)*log(3)*pi*3^(1/2)+(18)*pi^2+(486)*log(3)^2+(216)*(psi[1])(1/3))*euler_gamma+((1)*pi^3+((81)*log(3)^2+(36)*(psi[1])(1/3))*pi)*3^(1/2)+(27)*log(3)*pi^2+(243)*log(3)^3+(324)*(psi[1])(1/3)*log(3))*gamma(1/3))*_SAGE_VAR_x^3+(1/24)*((12*euler_gamma^2+(4*pi*3^(1/2)+36*log(3))*euler_gamma+6*log(3)*pi*3^(1/2)+pi^2+27*log(3)^2+12*(psi[1])(1/3))*gamma(1/3))*_SAGE_VAR_x^2+((1)/6)*((6*euler_gamma+pi*3^(1/2)+9*log(3))*gamma(1/3))*_SAGE_VAR_x+gamma(1/3)' in Sageand
TypeError: unable to make sense of Maxima expression '8*(li[33])(e^u)+3*(li[2])(u)' in Sage
No, it is not very easy to fix. We rewrite li[n](
to polylog(n,
and the same for psi[n](
to psi(n,
. This string matching obviously completely breaks if maxima starts spelling this as (li[n])(x)
. Quite frankly, that's an insane spelling, so perhaps the preferred approach is to clarify with the maxima folks if they really intend this. It might be an unforeseen byproduct of some other change that they prefer to fix themselves, in which case we shouldn't waste time.
If they insist this is a reasonable spelling, I think we can work around it, but it'll be a bit of real work. Note that we can handle calls to parenthesized functions:
sage: from sage.calculus.calculus import symbolic_expression_from_maxima_string as sefms sage: sefms('(sin)(x)') sin(x)
so as long as we ensure that expressions like li[n]
and psi[n]
themselves get parsed to callable functions, we should be OK. Something like this would be OK:
sage: sefms('li[n]') curried_polylog(n) sage: sefms('li[n]')(x) polylog(n,x) sage: sefms('(li[n])(x)') polylog(n,x)
Since we cannot really handle square brackets, we'd need to do the li[n]
>curried_polylog(n)
via a string substitution, but that should be OK. The curried_polylog(n)
would just create a symbolic function that, when called with a single argument, would create the appropriate polylog.
The binary conversion sr_to_max
in maxima_lib.py
shouldn't be affected, unless they have changed their internal representation of polylogarithms.
comment:25 Changed 7 years ago by
I opened https://sourceforge.net/p/maxima/bugs/2998/ to ask about these extra ()...
comment:26 Changed 7 years ago by
Commit:  82fdf0eaa092b63cc98d97b418721795ea6b5081 → 0d0649ad925808a308084b04253d7eb5c3fe2fad 

Branch pushed to git repo; I updated commit sha1. New commits:
0d0649a  some easy doctest fixes

comment:27 followup: 29 Changed 7 years ago by
This is bad:

build/pkgs/maxima/spkginstall
diff git a/build/pkgs/maxima/spkginstall b/build/pkgs/maxima/spkginstall index 3c80e5c..b08d0d7 100755
a b done 47 47 echo 48 48 echo "Now configuring Maxima..." 49 ./bootstrap 49 50 ./configure prefix="$SAGE_LOCAL" libdir="$SAGE_LOCAL/lib" enableecl git_found=false 50 51 check_error "Failed to configure Maxima."
Bootstrapping should ideally be done by upstream. Alternatively, it should be done in spkgsrc
, but certainly not in spkginstall
. Autotools are not a Sage prerequisite.
comment:28 followup: 30 Changed 7 years ago by
comment:29 Changed 7 years ago by
Replying to jdemeyer:
This is bad:
build/pkgs/maxima/spkginstall
diff git a/build/pkgs/maxima/spkginstall b/build/pkgs/maxima/spkginstall index 3c80e5c..b08d0d7 100755
a b done 47 47 echo 48 48 echo "Now configuring Maxima..." 49 ./bootstrap 49 50 ./configure prefix="$SAGE_LOCAL" libdir="$SAGE_LOCAL/lib" enableecl git_found=false 50 51 check_error "Failed to configure Maxima." Bootstrapping should ideally be done by upstream. Alternatively, it should be done in
spkgsrc
, but certainly not inspkginstall
. Autotools are not a Sage prerequisite.
I am acutely aware of this; upstream is not doing bootstrapping in the release in question. Also, note that the ticket status is "new". I'll sort this out after the bigger issues are fixed.
comment:30 Changed 7 years ago by
comment:31 Changed 7 years ago by
with sf.net back online, one finds there version 5.36.1. Not sure yet how much different it is from 5.36.0.1, which is not official...
5.36.1 has the same issue with too many ()...
comment:32 Changed 7 years ago by
Branch:  u/dimpase/eclupdate → u/dimpase/updateecl 

Commit:  0d0649ad925808a308084b04253d7eb5c3fe2fad → b669a434bdf2f91d21254ccf186bc86396dd4909 
comment:33 Changed 7 years ago by
Branch:  u/dimpase/updateecl → u/dimpase/eclupdate 

Commit:  b669a434bdf2f91d21254ccf186bc86396dd4909 → 0d0649ad925808a308084b04253d7eb5c3fe2fad 
Dependencies:  → #18961 
Description:  modified (diff) 
Summary:  upgrade Maxima and ECL to 5.36.0.1 and 15.3.7, respectively → upgrade Maxima to 5.36.1 
I've split the update of ECL to a separate ticket #18961, as it is ready.
comment:34 followup: 35 Changed 7 years ago by
Replying to nbruin:
Replying to dimpase:
The binary conversion
sr_to_max
inmaxima_lib.py
shouldn't be affected, unless they have changed their internal representation of polylogarithms.
They say the change is a byproduct to fix another bug. Obviously it is a low priority for them to make the output backwards compatible. I wonder if we should just fully switch to maxima_lib.py
and do not try to deal with this in some other way?
comment:35 followup: 36 Changed 7 years ago by
Replying to dimpase:
They say the change is a byproduct to fix another bug. Obviously it is a low priority for them to make the output backwards compatible. I wonder if we should just fully switch to
maxima_lib.py
and do not try to deal with this in some other way?
Robert isn't making any statement there about priority. I would not think it's a very low priority, because the current behaviour is tantamount to writing (sin)(x)
, which is silly. I would not be surprised if in the next couple of weeks, when he has had a chance to think about a solution, he will have a fix. I'm sure it's not hard to fix. It just needs some thought to fix it properly and efficiently.
In terms of total amount of work, I would think introducing "curried" poylog and psi will be less work than eliminating any dependence on string parsing from maxima.
For calculus use, we're already moved entirely to maxima_lib. The library instance just allows two interfaces back and forth: a stringsbased one and the binary sr_to_max
and max_to_sr
one.
Moving calculus entirely to sr_to_max
is a nice idea in general, and that's a worthwhile project that should be quite doable (we're halfway there already and it didn't take much work to do that).
It would mean identifying all remaining places where calculus triggers string conversions and replace those sites with mechanisms comparable to sr_integral, sr_limit, sr_sum etc.
I think it's worthwhile if somehow we do maintain the pexpect interface to maxima as well, even if we don't need it for the calculus functionality in sage proper (and since the stringsbased interface in maxima_lib shares nearly all its code with that, we might as well keep that too). Having the pexpect interfaces in sage is a useful feature for communicating with other computer algebra systems.
comment:36 Changed 7 years ago by
Replying to nbruin:
Replying to dimpase:
They say the change is a byproduct to fix another bug. Obviously it is a low priority for them to make the output backwards compatible. I wonder if we should just fully switch to
maxima_lib.py
and do not try to deal with this in some other way?Robert isn't making any statement there about priority. I would not think it's a very low priority, because the current behaviour is tantamount to writing
(sin)(x)
, which is silly. I would not be surprised if in the next couple of weeks, when he has had a chance to think about a solution, he will have a fix. I'm sure it's not hard to fix. It just needs some thought to fix it properly and efficiently.
he replied, saying that he does not see a way to fix this, and that Sage has to put up with this for the time being.
I'd rather see this fixed im maxima, of course, but this is probably a nontrivial job for a Lisp hacker we don't have...
comment:37 Changed 7 years ago by
Quick workaround:
ecl_eval("(defprop $LI 210 rbp)") ecl_eval("(defprop $PSI 210 rbp)")
it certainly avoids the extra parentheses around li
and psi
invocations. I've reported this workaround on the maxima bugtracker too. Hopefully Robert will comment there about any possible adverse effects. Including these statements in the maxima_calculus initialization in maxima_lib.py
might do the trick (and possibly the change is accepted upstream!)
EDIT: A better and more universal change is probably:
ecl_eval("(defprop MFUNCTION 190 lbp)")
This way it applies to all "MQAPPLY" calls. That could be problematic, of course, but it doesn't seem to be, probably because 190 is still high enough. Things go wrong if you go too low:
sage: maxima_calculus("(a+b)(t)") (b+a)(t) sage: ecl_eval("(defprop MFUNCTION 1 lbp)") <ECL: 1> sage: maxima_calculus("(a+b)(t)") b+a(t)
comment:38 Changed 7 years ago by
Of course, the expression type that gets constructed for li
and psi
is more general:
sage: maxima_calculus("a[1](x)") a[1](x) sage: maxima_calculus("a[1](x)").ecl() <ECL: ((MQAPPLY SIMP) (($A SIMP ARRAY) 1) $X)>
but we already fail with that (because we carefully only convert li[
and psi[
):
sage: SR(maxima_calculus("a[1](x)")) TypeError: unable to make sense of Maxima expression 'a[1](x)' in Sage
Since square brackets are really part of Maxima's expression syntax, the proper solution would be to have a parser that can handle them. The following seems valid maxima:
sage: maxima_calculus("(sin(x)+cos(y))[tan(u)[3/(a+b)]+log(w)](t)") (cos(y)+sin(x))[log(w)+tan(u)[3/(b+a)]](t)
and parsing that with anything less than parser support for [
will be. Of course, without []
support on the side of pynac, we'd have to parse a[n]
to something like get_item(a,n)
but that would be fine (and then get_item
can have the right logic to actually resolve in relevant cases, but this would be more an extension of our symbolic expressions before it's a parsing issue for maxima)
comment:39 Changed 7 years ago by
The maxima ticket https://sourceforge.net/p/maxima/bugs/2998/ is now marked as resolved, so with any luck, the extra parentheses issue will not occur with the new maxima release.
The fix implemented is indeed ecl_eval("(defprop MFUNCTION 190 lbp)")
, so if you do not want to wait for the next maxima release, just put this command in the initialization section of maxima_lib
.
comment:40 Changed 7 years ago by
Report Upstream:  N/A → Not yet reported upstream; Will do shortly. 

I tried upgrading to Maxima 5.37.0 (released 17 August). The formatting problem is fixed, although some of the output still looks slightly different (more/fewer parentheses, different ordering of terms in some polynomials).
However, there is a serious bug in the function eigenvectors
causing too few eigenvectors to be returned:
(%i1) M: matrix([1, 1, 0], [0, 1, 0], [0, 0, 2]); [ 1 1 0 ] [ ] (%o1) [ 0 1 0 ] [ ] [ 0 0 2 ]
Maxima 5.35.1 computes the eigenvalues and eigenvectors correctly:
(%i2) eigenvectors(M); (%o2) [[[1, 2], [2, 1]], [[[1, 0, 0]], [[0, 0, 1]]]]
In Maxima 5.37.0, the eigenvector [0, 0, 1]
is missing:
(%i2) eigenvectors(M); (%o2) [[[1, 2], [2, 1]], [[[1, 0, 0]]]]
I found this out through several doctest failures in Sage.
comment:41 Changed 7 years ago by
Report Upstream:  Not yet reported upstream; Will do shortly. → Reported upstream. No feedback yet. 

Upstream bug report: https://sourceforge.net/p/maxima/bugs/3008/
comment:42 followup: 44 Changed 6 years ago by
It seems that version 5.38 was released a few months ago, and we are currently at 5.35.1. I think we can upgrade now since the reported bug was fixed.
comment:43 Changed 6 years ago by
Cc:  Jeroen Demeyer Volker Braun added 

Milestone:  sage6.8 → sage7.3 
Report Upstream:  Reported upstream. No feedback yet. → Fixed upstream, in a later stable release. 
comment:44 Changed 6 years ago by
Replying to tscrim:
It seems that version 5.38 was released a few months ago, and we are currently at 5.35.1. I think we can upgrade now since the reported bug was fixed.
Note that the bug fix is only in the development branch, not in 5.38.1. We can of course add the relevant upstream commit (3e4e107) as a patch.
comment:45 Changed 6 years ago by
Shall we then move forward then with upgrading to 5.38.1 with backporting 3e4e107 as a patch? Or should we ask the Maxima team to see how close they are to cutting a 5.38.2?
comment:46 Changed 6 years ago by
Milestone:  sage7.3 → sage7.4 

Summary:  upgrade Maxima to 5.36.1 → upgrade Maxima to version >= 5.38.1 
it's about time. Should I have a go at it?
comment:47 Changed 6 years ago by
Please do. We have to make sagemath 7.4 work with maxima 5.38.1 in Debian and it would help if we had a patch that we could backport.
comment:50 Changed 6 years ago by
Branch:  u/dimpase/eclupdate → public/t18920 

Commit:  0d0649ad925808a308084b04253d7eb5c3fe2fad → 231002039aeed3f1926ec58804bd538b5ec9385d 
Dependencies:  #18961 
comment:51 Changed 6 years ago by
Work issues:  → rebase the rest of Maxima patches 

So far, only the relevant commits from the old branch are rebased. I still have to fix patches to Maxima sources  they don't apply cleanly.
comment:53 Changed 6 years ago by
Commit:  231002039aeed3f1926ec58804bd538b5ec9385d → a4bf7166892c6dae760cd2cd1c84f5cb9675ef15 

Branch pushed to git repo; I updated commit sha1. New commits:
a4bf716  rebased patches, added eigenvectors patch too

comment:54 Changed 6 years ago by
Authors:  → Dima Pasechnik 

Description:  modified (diff) 
Milestone:  sage7.4 → sage7.5 
Work issues:  rebase the rest of Maxima patches → fix doctests, if any. 
comment:55 Changed 6 years ago by
this seems to be a change:
File "src/sage/interfaces/maxima.py", line 210, in sage.interfaces.maxima Failed example: g.taylor('x', 0, 3) Expected: x+x^3/6+(3*x^5)/40+(5*x^7)/112+(35*x^9)/1152 Got: f/(k^4*x^4)(2*f)/((3*k^2)*x^2)+(11*f)/45((62*k^2*f)*x^2)/945
there f
is defined as f=maxima(`1/x^2`)
, so this no longer works, defining maxima
functions this way.
Anyhow, I'm pushing the other doctest changes for this file, and few other.
comment:56 Changed 6 years ago by
Commit:  a4bf7166892c6dae760cd2cd1c84f5cb9675ef15 → 62b5825544b467e481c7a752604b8d45c64d9752 

Branch pushed to git repo; I updated commit sha1. New commits:
62b5825  some more easy doctest fixes

comment:57 Changed 6 years ago by
the following looks to me like a regression (or perhaps it's not really, as if one picked the right branches of the corresponding complex variable functions, it would be correct) :
File "src/sage/calculus/tests.py", line 107, in sage.calculus.tests Failed example: integrate(log(1x^2)/x, x) Expected: 1/2*log(x^2)*log(x^2 + 1) + 1/2*polylog(2, x^2 + 1) Got: log(x)*log(x + 1) + log(x)*log(x + 1) + polylog(2, x + 1) + polylog(2, x + 1)
New commits:
62b5825  some more easy doctest fixes

comment:58 Changed 6 years ago by
and another regression of sorts: despite besselexpand
set to False, one gets
File "src/sage/interfaces/maxima_lib.py", line 66, in sage.interfaces.maxima_lib Failed example: sum((x)^n/(factorial(n)*factorial(n+3/2)),n,0,oo) Expected: bessel_J(3/2, 2*sqrt(x))/x^(3/4) Got: 1/2*(2*x*cos(2*sqrt(x))  sqrt(x)*sin(2*sqrt(x)))/(sqrt(pi)*x^2)
Perhaps it's a deficiency of our test, which is too simple for besselexpand
to kick in.
comment:59 Changed 6 years ago by
this one needs work:
File "src/sage/interfaces/maxima_lib.py", line 724, in sage.interfaces.maxima_lib.Ma ximaLib.sr_integral Failed example: integrate(1 / (1 + abs(x5)), x, 5, 6) Exception raised: Traceback (most recent call last): ... TypeError: unable to make sense of Maxima expression 'limit(if((_SAGE_VAR_x5)>0,log(6_SAGE_VAR_x),log(abs(_SAGE_VAR_x4))),_SAGE_VAR_x,5,plus)' in Sage
comment:60 followup: 65 Changed 6 years ago by
and this one is puzzling on the Sage side:
File "src/sage/misc/functional.py", line 658, in sage.misc.functional.integral Failed example: _ = integrate(y, x, 1000, 1000) Exception raised: ... FloatingPointError: Floating point exception
and comes from a regression in Maxima/ECL version combo (I did check that with Maxima 5.38.1 running on SBCL 1.3.6 this integral computes just fine).
Maxima 5.38.1 http://maxima.sourceforge.net using Lisp ECL 16.1.2 Distributed under the GNU Public License. See the file COPYING. Dedicated to the memory of William Schelter. The function bug_report() provides bug reporting information. (%i1) integrate((x^2)*exp(x) / (1 + exp(x))^2,x,1000,1000); Maxima encountered a Lisp error: #<a ARITHMETICERROR> Automatically continuing. To enable the Lisp debugger set *debuggerhook* to nil.
comment:61 Changed 6 years ago by
this one needs an expert opinion:
File "src/sage/schemes/projective/projective_morphism.py", line 853, in sage.schemes.projective.projective_morphism.SchemeMorphism_polynomial_projective_space.dynatomic_polynomial Failed example: f.dynatomic_polynomial(2) Expected: 0.666666666666667*x^2 + 0.333333333333333*y^2 Got: 2.00000000000000*x^2 + 0.999999999999999*y^2
The output is clearly different, but perhaps it's OK, I don't knowI suspect it's OK, as we must be interested in the projective points where it vanishes(?)  documentation talks about "roots" though, and these, projectively, remain the same.
comment:62 Changed 6 years ago by
Commit:  62b5825544b467e481c7a752604b8d45c64d9752 → 9ef994d79cfe0af28d17d0951b709576647a4231 

Branch pushed to git repo; I updated commit sha1. New commits:
9ef994d  more easy doctest fixes

comment:63 Changed 6 years ago by
Status:  new → needs_review 

setting this to "needs review", so that the issues I raised in comments can be discussed.
There are still few easy doctest fixes to be done, I'll do them.
comment:64 Changed 6 years ago by
Commit:  9ef994d79cfe0af28d17d0951b709576647a4231 → 3c280228256c02dcba6e6a8df2bcb98625031613 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
3c28022  more easy doctest fixes

comment:65 followup: 66 Changed 6 years ago by
Replying to dimpase:
regression in Maxima/ECL version combo (I did check that with Maxima 5.38.1 running on SBCL 1.3.6 this integral computes just fine).
comment:66 followup: 68 Changed 6 years ago by
Replying to dimpase:
Replying to dimpase:
regression in Maxima/ECL version combo (I did check that with Maxima 5.38.1 running on SBCL 1.3.6 this integral computes just fine).
Robert Dodier has traced this down to a problem with ECL, see https://gitlab.com/embeddablecommonlisp/ecl/issues/299
ECL has a problem comparing their own (there is no Common Lisp standard for this) infinity constants with 1/1000000...
comment:67 followup: 71 Changed 6 years ago by
Commit:  3c280228256c02dcba6e6a8df2bcb98625031613 → 100d9d6516e4bcfbb93ee979f086f52072ace786 

comment:68 Changed 6 years ago by
Replying to dimpase:
Replying to dimpase:
Replying to dimpase:
regression in Maxima/ECL version combo (I did check that with Maxima 5.38.1 running on SBCL 1.3.6 this integral computes just fine).
Robert Dodier has traced this down to a problem with ECL, see https://gitlab.com/embeddablecommonlisp/ecl/issues/299
ECL has a problem comparing their own (there is no Common Lisp standard for this) infinity constants with 1/1000000...
this is fixed by Maxima people, and in the latest commit 100d9d6
comment:69 Changed 6 years ago by
Commit:  100d9d6516e4bcfbb93ee979f086f52072ace786 → 814e8edc2eb51ba9706c122a39bf75d3c77e141c 

comment:70 Changed 6 years ago by
Commit:  814e8edc2eb51ba9706c122a39bf75d3c77e141c → af7e5ff2e30dfed8bcb4c597403a6a23eb93b86a 

Branch pushed to git repo; I updated commit sha1. New commits:
af7e5ff  new scaling of dynatomic polynomial

comment:71 Changed 6 years ago by
Replying to git:
Branch pushed to git repo; I updated commit sha1. New commits:
5d86413 Merge branch 'public/t18920' of trac.sagemath.org:sage into max5381
100d9d6 workaround for #<a ARITHMETICERROR> (18920#comment:60
this workaround turned out to be buggy, and anyway ECL has released a fix for this problem, making the workaround not necessary. (But it would need an update of our ECL package, as it does depend upon a number of changes, see https://gitlab.com/embeddablecommonlisp/ecl/issues/299; they promise a new release by the end of the year).
Should we wait, or should we proceed?
comment:72 Changed 6 years ago by
I think we should either backport the fix as a patch to ECL or perhaps take a development cut.
comment:73 followup: 75 Changed 6 years ago by
I would go for the first option which looks easier: just put the ECL patch into build/pkgs/ecl/patches
.
comment:74 Changed 6 years ago by
There is the issue that you have to force an ECL rebuild. Add a p(n+1)
suffix in packageversion.txt
?
comment:75 Changed 6 years ago by
Replying to jpflori:
I would go for the first option which looks easier: just put the ECL patch into
build/pkgs/ecl/patches
.
a single patch is not enough. In fact, it hangs the ECL build if I do this. ECL people say the have worked on a number of related things, that is support of IEEE floats, since the release, so the latest fix relies on these. A proper ECL update is the only way I think.
comment:76 Changed 6 years ago by
We may have to wait. I could try a git snapshot in sageongentoo to see how things are going but I am overloaded as it is. I have to draw the line somewhere.
comment:77 followup: 78 Changed 6 years ago by
Hi, we're following this ticket very closely in Debian. I notice that the following two patches are still applicable to 5.38.1:
 https://git.sagemath.org/sage.git/tree/build/pkgs/maxima/patches/0001taylor2Avoidblowingthestackwhendiffexpandisn.patch
 https://git.sagemath.org/sage.git/tree/build/pkgs/maxima/patches/matrixexp.patch
Have these patches been forwarded upstream yet? Would they be suitable for other users of Maxima, or are they only useful for Sage?
comment:78 Changed 6 years ago by
Replying to infinity0:
Hi, we're following this ticket very closely in Debian. I notice that the following two patches are still applicable to 5.38.1:
this one comes from upstream, from here: https://sourceforge.net/p/maxima/bugs/2520/#e3a5 (so it's sort of unfinished digging on the maxima side; it's definitely useful outside of Sage)
this is also from upstream (updated somewhat): https://sourceforge.net/p/maxima/bugs/2596/
again, upstream still has it open.
comment:79 followups: 81 83 Changed 6 years ago by
Status:  needs_review → needs_work 

Work issues:  fix doctests, if any. → fix doctests 
sage t long src/sage/interfaces/maxima.py ********************************************************************** File "src/sage/interfaces/maxima.py", line 210, in sage.interfaces.maxima Failed example: g.taylor('x', 0, 3) Expected: x+x^3/6+(3*x^5)/40+(5*x^7)/112+(35*x^9)/1152 Got: f/(k^4*x^4)(2*f)/((3*k^2)*x^2)+(11*f)/45((62*k^2*f)*x^2)/945 ********************************************************************** 1 item had failures: 1 of 96 in sage.interfaces.maxima [182 tests, 1 failure, 10.65 s] sage t long src/sage/symbolic/expression_conversions.py ********************************************************************** File "src/sage/symbolic/expression_conversions.py", line 535, in sage.symbolic.expression_conversions.InterfaceInit.derivative Failed example: b = maxima(a); b Expected: %at('diff('f(_SAGE_VAR_t0),_SAGE_VAR_t0,1),[_SAGE_VAR_t0=%e^_SAGE_VAR_x]) Got: %at('diff('f(_SAGE_VAR_t0),_SAGE_VAR_t0,1),_SAGE_VAR_t0=%e^_SAGE_VAR_x) ********************************************************************** File "src/sage/symbolic/expression_conversions.py", line 544, in sage.symbolic.expression_conversions.InterfaceInit.derivative Failed example: b = maxima(a); b Expected: %at('diff('f(_SAGE_VAR_t0),_SAGE_VAR_t0,1),[_SAGE_VAR_t0=4]) Got: %at('diff('f(_SAGE_VAR_t0),_SAGE_VAR_t0,1),_SAGE_VAR_t0=4) ********************************************************************** File "src/sage/symbolic/expression_conversions.py", line 564, in sage.symbolic.expression_conversions.InterfaceInit.derivative Failed example: b = maxima(a); b Expected: %at('diff('f(_SAGE_VAR_t0,_SAGE_VAR_t1),_SAGE_VAR_t0,1),[_SAGE_VAR_t0=4,_SAGE_VAR_t1=_SAGE_VAR_y]) Got: %at('diff('f(_SAGE_VAR_t0,_SAGE_VAR_y),_SAGE_VAR_t0,1),_SAGE_VAR_t0=4) ********************************************************************** File "src/sage/symbolic/expression_conversions.py", line 573, in sage.symbolic.expression_conversions.InterfaceInit.derivative Failed example: b = maxima(a); b Expected: %at('diff('f(_SAGE_VAR_t0,_SAGE_VAR_t1),_SAGE_VAR_t0,1),[_SAGE_VAR_t0=4,_SAGE_VAR_t1=8]) Got: %at('diff('f(_SAGE_VAR_t0,8),_SAGE_VAR_t0,1),_SAGE_VAR_t0=4) ********************************************************************** 1 item had failures: 4 of 37 in sage.symbolic.expression_conversions.InterfaceInit.derivative [462 tests, 4 failures, 5.14 s] sage t long src/sage/calculus/tests.py ********************************************************************** File "src/sage/calculus/tests.py", line 107, in sage.calculus.tests Failed example: integrate(log(1x^2)/x, x) Expected: 1/2*log(x^2)*log(x^2 + 1) + 1/2*polylog(2, x^2 + 1) Got: log(x)*log(x + 1) + log(x)*log(x + 1) + polylog(2, x + 1) + polylog(2, x + 1) ********************************************************************** 1 item had failures: 1 of 83 in sage.calculus.tests [82 tests, 1 failure, 8.56 s] sage t long src/sage/interfaces/maxima_lib.py ********************************************************************** File "src/sage/interfaces/maxima_lib.py", line 66, in sage.interfaces.maxima_lib Failed example: sum((x)^n/(factorial(n)*factorial(n+3/2)),n,0,oo) Expected: bessel_J(3/2, 2*sqrt(x))/x^(3/4) Got: 1/2*(2*x*cos(2*sqrt(x))  sqrt(x)*sin(2*sqrt(x)))/(sqrt(pi)*x^2) ********************************************************************** File "src/sage/interfaces/maxima_lib.py", line 724, in sage.interfaces.maxima_lib.MaximaLib.sr_integral Failed example: integrate(1 / (1 + abs(x5)), x, 5, 6) Exception raised: Traceback (most recent call last): File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 498, in _run self.compile_and_execute(example, compiler, test.globs) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 861, in compile_and_execute exec(compiled, globs) File "<doctest sage.interfaces.maxima_lib.MaximaLib.sr_integral[12]>", line 1, in <module> integrate(Integer(1) / (Integer(1) + abs(xInteger(5))), x, Integer(5), Integer(6)) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/misc/functional.py", line 662, in integral return x.integral(*args, **kwds) File "sage/symbolic/expression.pyx", line 11624, in sage.symbolic.expression.Expression.integral (/usr/local/src/sageconfig/src/build/cythonized/sage/symbolic/expression.cpp:64315) return integral(self, *args, **kwds) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/symbolic/integration/integral.py", line 777, in integrate return definite_integral(expression, v, a, b, hold=hold) File "sage/symbolic/function.pyx", line 995, in sage.symbolic.function.BuiltinFunction.__call__ (/usr/local/src/sageconfig/src/build/cythonized/sage/symbolic/function.cpp:11329) res = super(BuiltinFunction, self).__call__( File "sage/symbolic/function.pyx", line 485, in sage.symbolic.function.Function.__call__ (/usr/local/src/sageconfig/src/build/cythonized/sage/symbolic/function.cpp:6372) res = g_function_evalv(self._serial, vec, hold) File "sage/symbolic/function.pyx", line 1084, in sage.symbolic.function.BuiltinFunction._evalf_or_eval_ (/usr/local/src/sageconfig/src/build/cythonized/sage/symbolic/function.cpp:12689) return self._eval0_(*args) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/symbolic/integration/integral.py", line 178, in _eval_ return integrator(*args) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/symbolic/integration/external.py", line 24, in maxima_integrator result = maxima.sr_integral(expression, v, a, b) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/interfaces/maxima_lib.py", line 799, in sr_integral return max_to_sr(maxima_eval(([max_integrate],[sr_to_max(SR(a)) for a in args]))) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/interfaces/maxima_lib.py", line 1672, in max_to_sr args=[max_to_sr(a) for a in max_args] File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/interfaces/maxima_lib.py", line 1672, in max_to_sr args=[max_to_sr(a) for a in max_args] File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/interfaces/maxima_lib.py", line 1663, in max_to_sr sage_expr=SR(maxima(expr)) File "sage/structure/parent.pyx", line 953, in sage.structure.parent.Parent.__call__ (/usr/local/src/sageconfig/src/build/cythonized/sage/structure/parent.c:9849) return mor._call_(x) File "sage/structure/coerce_maps.pyx", line 238, in sage.structure.coerce_maps.NamedConvertMap._call_ (/usr/local/src/sageconfig/src/build/cythonized/sage/structure/coerce_maps.c:6388) cdef Element e = method(C) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/interfaces/maxima_abstract.py", line 1255, in _symbolic_ return R(self._sage_()) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/interfaces/maxima_abstract.py", line 1230, in _sage_ maxima=self.parent()) File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/calculus/calculus.py", line 1897, in symbolic_expression_from_maxima_string raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s) TypeError: unable to make sense of Maxima expression 'limit(if((_SAGE_VAR_x5)>0,log(6_SAGE_VAR_x),log(abs(_SAGE_VAR_x4))),_SAGE_VAR_x,5,plus)' in Sage **********************************************************************
comment:80 Changed 6 years ago by
Some of these doctests are solely Maxima's regressions, one is fixable via a ECL update. I can just mark them appropriately and proceed, if this is acceptable.
comment:81 followup: 82 Changed 6 years ago by
Replying to jdemeyer:
File "src/sage/interfaces/maxima_lib.py", line 724, in sage.interfaces.maxima_lib.MaximaLib.sr_integral Failed example: integrate(1 / (1 + abs(x5)), x, 5, 6) Exception raised: Traceback (most recent call last): ... File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/calculus/calculus.py", line 1897, in symbolic_expression_from_maxima_string raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s) TypeError: unable to make sense of Maxima expression 'limit(if((_SAGE_VAR_x5)>0,log(6_SAGE_VAR_x),log(abs(_SAGE_VAR_x4))),_SAGE_VAR_x,5,plus)' in Sage **********************************************************************
This is both #17892 and #20191, because the result changed, presumably to something more correct that needs these symbolic functions we don't have atm.
comment:82 Changed 6 years ago by
Replying to rws:
Replying to jdemeyer:
File "src/sage/interfaces/maxima_lib.py", line 724, in sage.interfaces.maxima_lib.MaximaLib.sr_integral Failed example: integrate(1 / (1 + abs(x5)), x, 5, 6) Exception raised: Traceback (most recent call last): ... File "/usr/local/src/sageconfig/local/lib/python2.7/sitepackages/sage/calculus/calculus.py", line 1897, in symbolic_expression_from_maxima_string raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s) TypeError: unable to make sense of Maxima expression 'limit(if((_SAGE_VAR_x5)>0,log(6_SAGE_VAR_x),log(abs(_SAGE_VAR_x4))),_SAGE_VAR_x,5,plus)' in Sage **********************************************************************This is both #17892 and #20191, because the result changed, presumably to something more correct that needs these symbolic functions we don't have atm.
I think this one is a new Maxima bug: https://sourceforge.net/p/maxima/bugs/3237/  still under investigation by Maxima people.
comment:83 Changed 6 years ago by
Replying to jdemeyer:
sage t long src/sage/interfaces/maxima.py ********************************************************************** File "src/sage/interfaces/maxima.py", line 210, in sage.interfaces.maxima Failed example: g.taylor('x', 0, 3) Expected: x+x^3/6+(3*x^5)/40+(5*x^7)/112+(35*x^9)/1152 Got: f/(k^4*x^4)(2*f)/((3*k^2)*x^2)+(11*f)/45((62*k^2*f)*x^2)/945
That's odd. The doctest skipped to the next result line which is of course different.
I think this one is a new Maxima bug: https://sourceforge.net/p/maxima/bugs/3237/  still under investigation by Maxima people.
Looking more closely, I agree.
comment:84 Changed 6 years ago by
Cc:  Paul Masson added 

comment:85 Changed 6 years ago by
ecl
16.1.3 is out. I haven't checked yet if all we want is in there but I will look int it.
comment:87 Changed 6 years ago by
Considering the havoc it wrecks in the building of the documentation it deserves its own ticket. Surprisingly, I haven't spotted libecl
in the numerous backtraces produced before the build petered out. But that's the only thing I changed from one successful build to this one.
comment:88 Changed 6 years ago by
Milestone:  sage7.5 → sage7.6 

comment:89 Changed 6 years ago by
Dependencies:  → #22191 

Description:  modified (diff) 
Summary:  upgrade Maxima to version >= 5.38.1 → upgrade Maxima to version >= 5.39 
maxima 5.39.0 has been released. ECL upgrade is happening on #22191.
For info, I have been using ecl 15.3.7 for a while in sageongentoo. That part is a straight upgrade as far as I can see. I do not remember having tried a newer maxima but it usually breaks a lest a couple of doctest.