Opened 7 years ago

Last modified 6 years ago

#18920 closed defect

upgrade Maxima to 5.36.1 — at Version 33

Reported by: dimpase Owned by:
Priority: major Milestone: sage-7.6
Component: symbolics Keywords:
Cc: rws, jpflori, kcrisman, nbruin, jdemeyer, vbraun, charpent Merged in:
Authors: Reviewers:
Report Upstream: Fixed upstream, in a later stable release. Work issues:
Branch: u/dimpase/eclupdate (Commits, GitHub, GitLab) Commit: 0d0649ad925808a308084b04253d7eb5c3fe2fad
Dependencies: #18961 Stopgaps:

Status badges

Description (last modified by dimpase)

this would fix some bugs, e.g. as reported here.

  • get Maxima here (then git checkout 5.36.0.1); the tarball; note that sf.net with tarballs is down...

ECL is updated on #18961

Change History (33)

comment:1 Changed 7 years ago by dimpase

  • Cc rws added
  • Description modified (diff)

comment:2 Changed 7 years ago by dimpase

  • Description modified (diff)

comment:3 Changed 7 years ago by dimpase

  • Description modified (diff)

comment:4 Changed 7 years ago by fbissey

For info, I have been using ecl 15.3.7 for a while in sage-on-gentoo. That part is a straight upgrade as far as I can see. I do not remember having tried a newer maxima but it usually breaks a lest a couple of doctest.

comment:5 Changed 7 years ago by fbissey

I tried maxima 5.36.0 and reverted (that was with ecl 15.3.7) , not tried 5.36.1 (5.36.0.1 whatever the version number is actually).

     Wed Apr 29 11:14:49 2015 >>> sci-mathematics/maxima-5.35.1-r2
       merge time: 5 minutes and 6 seconds.

     Fri May  1 10:33:58 2015 >>> sci-mathematics/maxima-5.36.0
       merge time: 4 minutes and 52 seconds.

     Fri May  1 10:42:03 2015 >>> sci-mathematics/maxima-5.35.1-r2
       merge time: 5 minutes and 8 seconds.

comment:6 Changed 7 years ago by dimpase

  • Branch set to u/dimpase/eclupdate
  • Cc jpflori added
  • Commit set to 045d21eeb356b04102696d1aa51776b1330c635a
  • Description modified (diff)

The branch is currently not working; to build the new ecl pkg, one has to move patches/implib.patch out of the way; I tried porting the latter patch to the new ecl, but it produces a makefile error that I don't understand; anyway, it is cygwin-specific.

I cc its author now. J.-P., any ideas?

comment:7 Changed 7 years ago by dimpase

for the record, I get

...
config.status: creating ecl/config.h
Configuration complete. To build ECL, issue make in this directory.
cd build; make -j1
make[1]: Entering directory `/home/dima/software/sage/local/var/tmp/sage/build/ecl-15.3.7.p0/src/build'
Makefile:192: *** commands commence before first target.  Stop.
make[1]: Leaving directory `/home/dima/software/sage/local/var/tmp/sage/build/ecl-15.3.7.p0/src/build'
make: *** [all] Error 2
Error - Failed to build ECL ... exiting

comment:8 Changed 7 years ago by dimpase

What I don't understand is whether some patches are in fact results of running autoconf/automake/aclocal on patched configure.ac, Makefile.in, etc.

If yes, then these should be split from the "real" ones to allow for fully automatic creation (and commands needed to run autotools need to be spelled out somewhere in the pkg docs). If no, then I don't see the point of patching configure.ac, Makefile.in, etc in the first place.

comment:9 follow-up: Changed 7 years ago by fbissey

Look at spkg_src. autotools are not run. But may be the impl platch was copied from an autotool run. The gmp patch deals with the removal of gmp sources from the upstream tarball (the configure script of ecl checks for stuff in that subdirectory so those checks have to be diverted). the impl patch deals with cygwin stuff.

Getting real autoconf patch and running autotools wouldn't be a bad idea but that means you will need to have autotools installed to run spkg-src. spkg-src will need to be amended accordingly.

comment:10 follow-up: Changed 7 years ago by fbissey

I notice that the gmp patch is against configure.ac while implib is against configure.in the last one is probably wrong.

comment:11 in reply to: ↑ 10 Changed 7 years ago by dimpase

Replying to fbissey:

I notice that the gmp patch is against configure.ac while implib is against configure.in the last one is probably wrong.

the name has changed in version 15.3.7.

comment:12 follow-up: Changed 7 years ago by fbissey

I am trying to read the stuff from the git commit and of course it is patch of patch, it is not readable as this, you changed it already I see.

comment:13 in reply to: ↑ 9 ; follow-ups: Changed 7 years ago by dimpase

Replying to fbissey:

Look at spkg_src. autotools are not run. But may be the impl platch was copied from an autotool run. The gmp patch deals with the removal of gmp sources from the upstream tarball (the configure script of ecl checks for stuff in that subdirectory so those checks have to be diverted). the impl patch deals with cygwin stuff.

Getting real autoconf patch and running autotools wouldn't be a bad idea but that means you will need to have autotools installed to run spkg-src. spkg-src will need to be amended accordingly.

spkg-src is outdated anyway.

comment:14 in reply to: ↑ 12 Changed 7 years ago by dimpase

Replying to fbissey:

I am trying to read the stuff from the git commit and of course it is patch of patch, it is not readable as this, you changed it already I see.

here they are in a more readable way: https://github.com/dimpase/ecl/commits/sagepatches

comment:15 in reply to: ↑ 13 Changed 7 years ago by dimpase

Replying to dimpase:

Replying to fbissey:

Look at spkg_src. autotools are not run. But may be the impl platch was copied from an autotool run. The gmp patch deals with the removal of gmp sources from the upstream tarball (the configure script of ecl checks for stuff in that subdirectory so those checks have to be diverted). the impl patch deals with cygwin stuff.

No, I mean that it is silly and error-prone to rebase by hand patches that are better obtained directly from diffs to output of autoconf/automake/aclocal.

The latter should not be bundled together with the "real" one.

comment:16 Changed 7 years ago by dimpase

I have opened https://gitlab.com/embeddable-common-lisp/ecl/issues/93

to hopefully sort out the ECL autotools mess.

comment:17 Changed 7 years ago by fbissey

Sorry I have been too busy today to look further into this. I can reproduce your problem with your branch on my mac. I haven't quite identified yet the guilty makefile. I think the problem is not in the top makefile but in one of the subfolders but I haven't identified which one yet.

comment:18 Changed 7 years ago by fbissey

To complete the answer you got from upstream: automake is not used (not a crime) but ecl also ship some of its dependencies in the pristine tarball and those come with their own build systems which can use automake. Apart from ffi we don't want to care about these I think.

comment:19 Changed 7 years ago by dimpase

arrghh, that was me mixing of tabs and spaces in src/Makefile.in patch... Now all seems to work. I'll post an update soon.

OK, and for the record, src/configure is generated by autoreconf, that is, one does not need to rebase the corresponding patch manually.

comment:20 Changed 7 years ago by git

  • Commit changed from 045d21eeb356b04102696d1aa51776b1330c635a to 9fc27ca7ba877a83443c61d22542e49d65096404

Branch pushed to git repo; I updated commit sha1. New commits:

9fc27cafixed broken Makefile.in patch (spaces and tabs don't mix), reorganised patches.

comment:21 Changed 7 years ago by kcrisman

  • Cc kcrisman added

comment:22 Changed 7 years ago by git

  • Commit changed from 9fc27ca7ba877a83443c61d22542e49d65096404 to 82fdf0eaa092b63cc98d97b418721795ea6b5081

Branch pushed to git repo; I updated commit sha1. New commits:

82fdf0eported maxima patches; tarball needs bootstrapping; trivial doctests fix

comment:23 follow-up: Changed 7 years ago by dimpase

  • Cc nbruin added

New Maxima output sometimes breaks Sage parser, e.g.

sage -t src/sage/calculus/calculus.py
**********************************************************************
File "src/sage/calculus/calculus.py", line 373, in sage.calculus.calculus
Failed example:
    taylor(gamma(1/3+x),x,0,3)
Exception raised:
    Traceback (most recent call last):
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.calculus.calculus[100]>", line 1, in <module>
        taylor(gamma(Integer(1)/Integer(3)+x),x,Integer(0),Integer(3))
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/calculus/functional.py", line 378, in taylor
        return f.taylor(*args)
      File "sage/symbolic/expression.pyx", line 4038, in sage.symbolic.expression.Expression.taylor (/home/dima/software/sage/src/build/cythonized/sage/symbolic/expression.cpp:23671)
        return self.parent()(l)
      File "sage/structure/parent.pyx", line 1097, in sage.structure.parent.Parent.__call__ (/home/dima/software/sage/src/build/cythonized/sage/structure/parent.c:9546)
        return mor._call_(x)
      File "sage/structure/coerce_maps.pyx", line 237, in sage.structure.coerce_maps.NamedConvertMap._call_ (/home/dima/software/sage/src/build/cythonized/sage/structure/coerce_maps.c:5756)
        cdef Element e = method(C)
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/interfaces/maxima_abstract.py", line 1251, in _symbolic_
        return R(self._sage_())
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/interfaces/maxima_abstract.py", line 1226, in _sage_
        maxima=self.parent())
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/calculus/calculus.py", line 1901, in symbolic_expression_from_maxima_string
        raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s)
    TypeError: unable to make sense of Maxima expression '(1/432)*(72*gamma(1/3)*(psi[2])(1/3)+((-72)*euler_gamma^3+((-36)*pi*3^(1/2)+(-324)*log(3))*euler_gamma^2+((-108)*log(3)*pi*3^(1/2)+(-18)*pi^2+(-486)*log(3)^2+(-216)*(psi[1])(1/3))*euler_gamma+((-1)*pi^3+((-81)*log(3)^2+(-36)*(psi[1])(1/3))*pi)*3^(1/2)+(-27)*log(3)*pi^2+(-243)*log(3)^3+(-324)*(psi[1])(1/3)*log(3))*gamma(1/3))*_SAGE_VAR_x^3+(1/24)*((12*euler_gamma^2+(4*pi*3^(1/2)+36*log(3))*euler_gamma+6*log(3)*pi*3^(1/2)+pi^2+27*log(3)^2+12*(psi[1])(1/3))*gamma(1/3))*_SAGE_VAR_x^2+((-1)/6)*((6*euler_gamma+pi*3^(1/2)+9*log(3))*gamma(1/3))*_SAGE_VAR_x+gamma(1/3)' in Sage

and

File "src/sage/calculus/calculus.py", line 1746, in sage.calculus.calculus.symbolic_expression_from_maxima_string
Failed example:
    maxima('3*li[2](u)+8*li[33](exp(u))').sage()
Exception raised:
    Traceback (most recent call last):
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 496, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 858, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.calculus.calculus.symbolic_expression_from_maxima_string[6]>", line 1, in <module>
        maxima('3*li[2](u)+8*li[33](exp(u))').sage()
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/interfaces/interface.py", line 1016, in sage
        return self._sage_(*args, **kwds)
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/interfaces/maxima_abstract.py", line 1226, in _sage_
        maxima=self.parent())
      File "/home/dima/software/sage/local/lib/python2.7/site-packages/sage/calculus/calculus.py", line 1901, in symbolic_expression_from_maxima_string
        raise TypeError("unable to make sense of Maxima expression '%s' in Sage"%s)
    TypeError: unable to make sense of Maxima expression '8*(li[33])(e^u)+3*(li[2])(u)' in Sage

Is it easy to fix?

comment:24 in reply to: ↑ 23 Changed 7 years ago by nbruin

Replying to dimpase:

    TypeError: unable to make sense of Maxima expression '(1/432)*(72*gamma(1/3)*(psi[2])(1/3)+((-72)*euler_gamma^3+((-36)*pi*3^(1/2)+(-324)*log(3))*euler_gamma^2+((-108)*log(3)*pi*3^(1/2)+(-18)*pi^2+(-486)*log(3)^2+(-216)*(psi[1])(1/3))*euler_gamma+((-1)*pi^3+((-81)*log(3)^2+(-36)*(psi[1])(1/3))*pi)*3^(1/2)+(-27)*log(3)*pi^2+(-243)*log(3)^3+(-324)*(psi[1])(1/3)*log(3))*gamma(1/3))*_SAGE_VAR_x^3+(1/24)*((12*euler_gamma^2+(4*pi*3^(1/2)+36*log(3))*euler_gamma+6*log(3)*pi*3^(1/2)+pi^2+27*log(3)^2+12*(psi[1])(1/3))*gamma(1/3))*_SAGE_VAR_x^2+((-1)/6)*((6*euler_gamma+pi*3^(1/2)+9*log(3))*gamma(1/3))*_SAGE_VAR_x+gamma(1/3)' in Sage

and

    TypeError: unable to make sense of Maxima expression '8*(li[33])(e^u)+3*(li[2])(u)' in Sage

No, it is not very easy to fix. We rewrite li[n]( to polylog(n, and the same for psi[n]( to psi(n,. This string matching obviously completely breaks if maxima starts spelling this as (li[n])(x). Quite frankly, that's an insane spelling, so perhaps the preferred approach is to clarify with the maxima folks if they really intend this. It might be an unforeseen byproduct of some other change that they prefer to fix themselves, in which case we shouldn't waste time.

If they insist this is a reasonable spelling, I think we can work around it, but it'll be a bit of real work. Note that we can handle calls to parenthesized functions:

sage: from sage.calculus.calculus import symbolic_expression_from_maxima_string as sefms
sage: sefms('(sin)(x)')
sin(x)

so as long as we ensure that expressions like li[n] and psi[n] themselves get parsed to callable functions, we should be OK. Something like this would be OK:

sage: sefms('li[n]')
curried_polylog(n)
sage: sefms('li[n]')(x)
polylog(n,x)
sage: sefms('(li[n])(x)')
polylog(n,x)

Since we cannot really handle square brackets, we'd need to do the li[n]->curried_polylog(n) via a string substitution, but that should be OK. The curried_polylog(n) would just create a symbolic function that, when called with a single argument, would create the appropriate polylog.

The binary conversion sr_to_max in maxima_lib.py shouldn't be affected, unless they have changed their internal representation of polylogarithms.

comment:25 Changed 7 years ago by dimpase

I opened https://sourceforge.net/p/maxima/bugs/2998/ to ask about these extra ()...

comment:26 Changed 7 years ago by git

  • Commit changed from 82fdf0eaa092b63cc98d97b418721795ea6b5081 to 0d0649ad925808a308084b04253d7eb5c3fe2fad

Branch pushed to git repo; I updated commit sha1. New commits:

0d0649asome easy doctest fixes

comment:27 follow-up: Changed 7 years ago by jdemeyer

This is bad:

  • build/pkgs/maxima/spkg-install

    diff --git a/build/pkgs/maxima/spkg-install b/build/pkgs/maxima/spkg-install
    index 3c80e5c..b08d0d7 100755
    a b done 
    4747echo
    4848echo "Now configuring Maxima..."
     49./bootstrap
    4950./configure --prefix="$SAGE_LOCAL" --libdir="$SAGE_LOCAL/lib" --enable-ecl git_found=false
    5051check_error "Failed to configure Maxima."

Bootstrapping should ideally be done by upstream. Alternatively, it should be done in spkg-src, but certainly not in spkg-install. Autotools are not a Sage prerequisite.

comment:28 in reply to: ↑ 13 ; follow-up: Changed 7 years ago by jdemeyer

Replying to dimpase:

spkg-src is outdated anyway.

[citation needed]

comment:29 in reply to: ↑ 27 Changed 7 years ago by dimpase

Replying to jdemeyer:

This is bad:

  • build/pkgs/maxima/spkg-install

    diff --git a/build/pkgs/maxima/spkg-install b/build/pkgs/maxima/spkg-install
    index 3c80e5c..b08d0d7 100755
    a b done 
    4747echo
    4848echo "Now configuring Maxima..."
     49./bootstrap
    4950./configure --prefix="$SAGE_LOCAL" --libdir="$SAGE_LOCAL/lib" --enable-ecl git_found=false
    5051check_error "Failed to configure Maxima."

Bootstrapping should ideally be done by upstream. Alternatively, it should be done in spkg-src, but certainly not in spkg-install. Autotools are not a Sage prerequisite.

I am acutely aware of this; upstream is not doing bootstrapping in the release in question. Also, note that the ticket status is "new". I'll sort this out after the bigger issues are fixed.

comment:30 in reply to: ↑ 28 Changed 7 years ago by dimpase

Replying to jdemeyer:

Replying to dimpase:

spkg-src is outdated anyway.

[citation needed]

I meant to say that parts of this spkg-src need an update.

comment:31 Changed 7 years ago by dimpase

with sf.net back online, one finds there version 5.36.1. Not sure yet how much different it is from 5.36.0.1, which is not official...

5.36.1 has the same issue with too many ()...

Last edited 7 years ago by dimpase (previous) (diff)

comment:32 Changed 7 years ago by dimpase

  • Branch changed from u/dimpase/eclupdate to u/dimpase/updateecl
  • Commit changed from 0d0649ad925808a308084b04253d7eb5c3fe2fad to b669a434bdf2f91d21254ccf186bc86396dd4909

New commits:

1a0e5c9initial update; implib.patch is problematic
b669a43fixed broken Makefile.in patch (spaces and tabs don't mix), reorganised patches.

comment:33 Changed 7 years ago by dimpase

  • Branch changed from u/dimpase/updateecl to u/dimpase/eclupdate
  • Commit changed from b669a434bdf2f91d21254ccf186bc86396dd4909 to 0d0649ad925808a308084b04253d7eb5c3fe2fad
  • Dependencies set to #18961
  • Description modified (diff)
  • Summary changed from upgrade Maxima and ECL to 5.36.0.1 and 15.3.7, respectively to upgrade Maxima to 5.36.1

I've split the update of ECL to a separate ticket #18961, as it is ready.

Note: See TracTickets for help on using tickets.