#30063 closed enhancement (fixed)
Upgrade: Maxima 5.44.0
Reported by:  pbruin  Owned by:  

Priority:  major  Milestone:  sage9.2 
Component:  packages: standard  Keywords:  upgrade, maxima 
Cc:  dimpase, fbissey, ghtimokau, saraedum, slelievre, arojas, kcrisman, nbruin, paulmasson  Merged in:  
Authors:  Matthias Koeppe, Thierry Monteil  Reviewers:  Emmanuel Charpentier, Matthias Koeppe 
Report Upstream:  N/A  Work issues:  
Branch:  21e2fea (Commits, GitHub, GitLab)  Commit:  
Dependencies:  #22191  Stopgaps: 
Description (last modified by )
Change History (59)
comment:1 Changed 2 years ago by
 Cc dimpase added
 Dependencies set to #22191
comment:2 Changed 2 years ago by
 Branch set to u/mkoeppe/upgrade_maxima_to_5_44_0
comment:3 Changed 2 years ago by
 Commit set to abb763e2ac72741ea86d0eee5b758c026fc9d1d4
 Description modified (diff)
comment:4 Changed 2 years ago by
 Cc fbissey ghtimokau saraedum slelievre added
 Description modified (diff)
comment:5 Changed 2 years ago by
 Description modified (diff)
comment:6 Changed 2 years ago by
 Cc arojas added
comment:7 Changed 2 years ago by
 Commit changed from abb763e2ac72741ea86d0eee5b758c026fc9d1d4 to 3db87ad1fd15fb50b6e44267cc5c37759376bba0
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
3db87ad  build/pkgs/maxima: Update to 5.44.0

comment:8 Changed 2 years ago by
Removing bugfix3629.patch
, which is in 5.44.0
0001taylor2Avoidblowingthestackwhendiffexpandisn.patch
does not apply. 5.44.0 has a different (simpler) change in this code.
comment:9 Changed 2 years ago by
 Commit changed from 3db87ad1fd15fb50b6e44267cc5c37759376bba0 to bcf3c98839dfdfa37a9863e6f1c70f20997bb4ea
Branch pushed to git repo; I updated commit sha1. New commits:
bcf3c98  Remove old patches

comment:10 Changed 2 years ago by
On macOS,
[maxima5.44.0] /bin/bash /Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/local/var/tmp/sage/build/maxima5.44.0/src/missing makeinfo html c TEXI2HTML=1 split=chapter documentlang=en output=. cssinclude=../../../doc/info/manual.css initfile ../../../doc/info/texi2html.init xmaxima.texi [maxima5.44.0] makeinfo: invalid option  c [maxima5.44.0] Try `makeinfo help' for more information.
comment:11 Changed 2 years ago by
This is /usr/bin/makeinfo
$ makeinfo version makeinfo (GNU texinfo) 4.8 Copyright (C) 2004 Free Software Foundation, Inc.
comment:12 followup: ↓ 15 Changed 2 years ago by
 Commit changed from bcf3c98839dfdfa37a9863e6f1c70f20997bb4ea to 07aa2f93345d2cecee75e074dc9bbcb22074129c
Branch pushed to git repo; I updated commit sha1. New commits:
07aa2f9  build/pkgs/maxima/spkginstall.in: New workaround for makeinfo trouble

comment:13 Changed 2 years ago by
Builds OK now on macOS. Haven't tested anything else
comment:14 Changed 2 years ago by
comment:15 in reply to: ↑ 12 Changed 2 years ago by
comment:16 Changed 2 years ago by
Unfortunately they don't.
comment:17 followups: ↓ 19 ↓ 20 ↓ 22 Changed 2 years ago by
These are the test failures I get downstream (using system maxima):
********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/interfaces/maxima_abstract.py", line 225, in sage.interfaces.maxima_abstract.MaximaAbstract.example Failed example: maxima.example('arrays') Expected: a[n]:=n*a[n1] a := n a n n  1 a[0]:1 a[5] 120 a[n]:=n a[6] 6 a[4] 24 done Got: n n  1 a[0]:1 a[5] 120 a[n]:=n a[6] 6 a[4] 24 done <BLANKLINE> ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/interfaces/maxima_abstract.py", line 283, in sage.interfaces.maxima_abstract.MaximaAbstract.completions Failed example: sorted(maxima.completions('gc', verbose=False)) Expected: ['gcd', 'gcdex', 'gcfactor', 'gctime'] Got: ['gcd', 'gcd\\impl', 'gcdex', 'gcfactor', 'gcfactor\\impl', 'gctime'] ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/symbolic/expression.pyx", line 10031, in sage.symbolic.expression.Expression.simplify_hypergeometric Failed example: hypergeometric_M(1, 3, x).simplify_hypergeometric() Expected: 2*(x  e^x + 1)/x^2 Got: 2*e^x*gamma_incomplete_lower(2, x)/x^2 ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/symbolic/integration/integral.py", line 864, in sage.symbolic.integration.integral.integrate Failed example: a = integral(log(cot(x)  1), x, 0, pi/4); a # long time (about 6 s) Expected: 1/4*pi*log(2)  1/2*I*dilog(I + 1) + 1/2*I*dilog(I + 1) + 1/2*I*dilog(1/2*I + 1/2)  1/2*I*dilog(1/2*I + 1/2) Got: 1/4*pi*(I*pi + log(2))  1/2*pi*log(2)  1/2*I*dilog(I + 1) + 1/2*I*dilog(I + 1) + 1/2*I*dilog(1/2*I + 1/2)  1/2*I*dilog(1/2*I + 1/2) ********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/symbolic/integration/integral.py", line 866, in sage.symbolic.integration.integral.integrate Failed example: abs(N(a  pi*log(2)/8)) < 1e15 # long time Expected: True Got: False **********************************************************************
The first one needs a readjustment of the number of removed lines in https://git.sagemath.org/sage.git/tree/src/sage/interfaces/maxima_abstract.py#n183
The ones in integration/integral.py
are a known upstream issue since 5.43. The corresponding tests in maxima are disabled as known to fail, so maybe they can be tagged as known bug here.
The other two seem harmless.
comment:18 Changed 2 years ago by
 Commit changed from 07aa2f93345d2cecee75e074dc9bbcb22074129c to 046634d7429f9c82a47b011dd986bbb665759a13
Branch pushed to git repo; I updated commit sha1. New commits:
046634d  sage.interfaces.maxima_abstract.MaximaAbstract._command_runner: Skip all comment lines, rather than counting lines to skip

comment:19 in reply to: ↑ 17 Changed 2 years ago by
Replying to arojas:
The first one needs a readjustment of the number of removed lines in https://git.sagemath.org/sage.git/tree/src/sage/interfaces/maxima_abstract.py#n183
I have fixed the first one by a method that I hope to be more robust.
comment:20 in reply to: ↑ 17 Changed 2 years ago by
Replying to arojas:
These are the test failures I get downstream (using system maxima):
********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/interfaces/maxima_abstract.py", line 283, in sage.interfaces.maxima_abstract.MaximaAbstract.completions Failed example: sorted(maxima.completions('gc', verbose=False)) Expected: ['gcd', 'gcdex', 'gcfactor', 'gctime'] Got: ['gcd', 'gcd\\impl', 'gcdex', 'gcfactor', 'gcfactor\\impl', 'gctime'] **********************************************************************
This is now https://sourceforge.net/p/maxima/bugs/3643/
comment:21 Changed 2 years ago by
 Commit changed from 046634d7429f9c82a47b011dd986bbb665759a13 to d1ad8d7a9482bbbb3b7c0a6b919744e954f396cf
Branch pushed to git repo; I updated commit sha1. New commits:
d1ad8d7  sage.interfaces.maxima_abstract.MaximaAbstract.completions: Remove 'impl' functions

comment:22 in reply to: ↑ 17 ; followup: ↓ 23 Changed 2 years ago by
 Cc kcrisman nbruin paulmasson added
Replying to arojas:
These are the test failures I get downstream (using system maxima):
********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/symbolic/expression.pyx", line 10031, in sage.symbolic.expression.Expression.simplify_hypergeometric Failed example: hypergeometric_M(1, 3, x).simplify_hypergeometric() Expected: 2*(x  e^x + 1)/x^2 Got: 2*e^x*gamma_incomplete_lower(2, x)/x^2
For this one I hope we can enroll the help of some symbolics experts. A related older ticket: #16697
comment:23 in reply to: ↑ 22 Changed 2 years ago by
Replying to mkoeppe:
Replying to arojas:
These are the test failures I get downstream (using system maxima):
********************************************************************** File "/usr/lib/python3.8/sitepackages/sage/symbolic/expression.pyx", line 10031, in sage.symbolic.expression.Expression.simplify_hypergeometric Failed example: hypergeometric_M(1, 3, x).simplify_hypergeometric() Expected: 2*(x  e^x + 1)/x^2 Got: 2*e^x*gamma_incomplete_lower(2, x)/x^2For this one I hope we can enroll the help of some symbolics experts. A related older ticket: #16697
M(1,3,x) = 1F1(1,3,x)  a.k.a Kummer function, whereas lower incomplete gamma is gamma(a,x)=x^{a}/a 1F1(a,a+1,x), a=2, i.e. the claim is that 1F1(1,3,x)=exp(x) 1F1(2,3,x). So this seems to be Kummer transform from https://en.wikipedia.org/wiki/Confluent_hypergeometric_function#Kummer's_transformation
but with a wrong sign (modulo typos...)
see also https://dlmf.nist.gov/13.6#E5
comment:24 Changed 2 years ago by
From the maxima 5.43 change log:
 series expansion for expintegral_si and gamma_incomplete; breaks some existing tests, see commit 47a6afd.
From the maxima 5.44 change log:
 gamma_incomplete_lower is now returned in the noun form in many places
 Taylor expansion for gamma_incomplete_lower(1/2,z) now works
 gamma_incomplete_lower now respects gamma_expand
I note the following:
sage: hypergeometric_M(1, 3, 1).n() 1.43656365691809 sage: hypergeometric_M(1, 3, 1).simplify_hypergeometric() 2*e*gamma_incomplete_lower(2, 1) sage: hypergeometric_M(1, 3, 1).simplify_hypergeometric().n() TypeError: cannot evaluate symbolic expression numerically
comment:25 Changed 2 years ago by
 Status changed from new to needs_info
comment:26 Changed 2 years ago by
For the problem in comment:22 I recommend doing as Nils mentions in the recent sagedevel discussion around a Maxima infinite recursion error, and isolate which Maxima commands do this. I believe simplify_hypergeometric
is a Maxima command wrapper, so it should be completely replicated in Maxima. By the way, in diagnosing this, a a few things to note.
sage: hypergeometric_M(1, 3, x) hypergeometric_M(1, 3, x) sage: maxima_calculus(hypergeometric_M(1,3,x)) kummer_m(1,3,_SAGE_VAR_x)
but
sage: B = hypergeometric_M(1, 3, x) sage: B.generalized() hypergeometric((1,), (3,), x) sage: maxima_calculus(B.generalized()) hypergeometric([1],[3],_SAGE_VAR_x)
and also don't forget
sage: B.simplify_hypergeometric(algorithm='sage') 2*(x  e^x + 1)/x^2
Unfortunately I don't have a chance to check out this update in order to diagnose the problem further. I do note that the hgfred
uses maxima
and not maxima_calculus
, though perhaps it is imported as such earlier in the file.
comment:27 Changed 2 years ago by
 Milestone changed from sage9.2 to sage9.3
comment:28 Changed 2 years ago by
 Branch changed from u/mkoeppe/upgrade_maxima_to_5_44_0 to u/tmonteil/upgrade_maxima_to_5_44_0
comment:29 Changed 2 years ago by
 Commit changed from d1ad8d7a9482bbbb3b7c0a6b919744e954f396cf to 21e2feaa1188735241e988836a7d21bec258c741
Branch pushed to git repo; I updated commit sha1. New commits:
21e2fea  #30063 : mark Maxima's integration regression as known bug.

comment:30 followups: ↓ 35 ↓ 49 Changed 2 years ago by
 Milestone changed from sage9.3 to sage9.2
 Priority changed from minor to major
 Status changed from needs_info to needs_review
 Type changed from task to enhancement
I was working on upgrading Maxima too (as i did not see that ticket), so while i am not a symbolics expert, i fixed the gamma_incomplete_lower
issue: that function was called gamma_greek
by Maxima before.
I also marked the intergration regression as known bug.
I think it is ready for review now.
comment:31 followup: ↓ 32 Changed 2 years ago by
I'm trying to test this one. make
started to getgcc 9.2.0
(whereas the system's gcc
is at 10.1.0
).
WTF ?
comment:32 in reply to: ↑ 31 ; followup: ↓ 33 Changed 2 years ago by
comment:33 in reply to: ↑ 32 ; followup: ↓ 34 Changed 2 years ago by
Replying to tmonteil:
Replying to charpent:
I'm trying to test this one.
make
started to getgcc 9.2.0
(whereas the system'sgcc
is at10.1.0
).WTF ?
This is probably because #29674 has been merged. Is it not related to the current ticket.
That was merged three months ago. I have rebuild andor ipgraded my sage
installation a few times since, and this never happened.
Furthermore, I tried reconfiguring with ./configure CC=gcc9 CXX=g++9 FC=gfortran9 enabledownloadfromupstreamurl
; configure
now wants to install a bunch of system packages that were accepted before. Its answers :
configure: notice: the following SPKGs did not find equivalent system packages: cbc coxeter3 fflas_ffpack gfortran gsl iml openblas r suitesparse checking for the package system in use... debian configure: hint: installing the following system packages is recommended and may avoid building some of the above SPKGs from source: configure: $ sudo aptget update $ sudo aptget install coinorcbc coinorlibcbcdev fflasffpack gfortran libgsldev libimldev libopenblasdev rbasedev rcranlattice libsuitesparsedev configure: After installation, rerun configure using: configure: $ ./config.status recheck && ./config.status
Attempting a (simulated) installation tells me that all those packages are already at their latest versions... and the check fails again.
I'm afraid to be stuck...
comment:34 in reply to: ↑ 33 Changed 2 years ago by
Replying to charpent:
[ Snip... ]
I'm afraid to be stuck...
Rereading my terminal I may have typed a small but devastating typo. I cleaned up and retried, and things seem to be OK for now.
Deeply sorry for the noise...
comment:35 in reply to: ↑ 30 ; followup: ↓ 36 Changed 2 years ago by
Replying to tmonteil:
I was working on upgrading Maxima too (as i did not see that ticket), so while i am not a symbolics expert, i fixed the
gamma_incomplete_lower
issue: that function was calledgamma_greek
by Maxima before.I also marked the intergration regression as known bug.
I think it is ready for review now.
Thanks for working on this!
Do we actually know whether this new Maxima version brings any improvements that are relevant for us  given that there is this regression?
comment:36 in reply to: ↑ 35 Changed 2 years ago by
comment:37 Changed 2 years ago by
Dima had a concern about a possible sign error in comment 23 above  could someone double check?
comment:38 Changed 2 years ago by
Tickets with this ticket as a dependency:
comment:39 followup: ↓ 42 Changed 2 years ago by
 Status changed from needs_review to positive_review
Okay. ptestlong
gives me the same (unrelated) failures as what I have already reported since 9.2.beta3
(IIRC).
Concerning comment 23 :
sage: hypergeometric_M(1, 3, x).simplify_hypergeometric() 2*((x + 1)*e^(x)  1)*e^x/x^2 sage: hypergeometric_M(1, 3, x).simplify_hypergeometric().factor() 2*(x  e^x + 1)/x^2
which seems kosher :
sage: mathematica.FullSimplify(hypergeometric_M(1, 3, x)).sage().factor() 2*(x  e^x + 1)/x^2
==> (tentative) positive_review
.
As always, crosschecks on various platforms are more than useful.
[ Only incidentally related... ] : I sternuously object to have to retain obsolete versions of the compilers in order to be able to compile without being told so. This problem, which appears only in some platforms, should be documented in the README.md
; furthermore, ./configure
should :
 suggest the installation of the relevant packages for older versions (recompiling the
gcc
smalah takes a geological age or two...), and  suggest reconfiguring with the relevant arguments.
Where should this rant be usefully posted ? I am aware of too much tickets related to system configuration and options, and am a bit lost...
comment:40 followup: ↓ 46 Changed 2 years ago by
 Status changed from positive_review to needs_work
Reviewer name is missing, I am changing to needs work before the release manager needs to do it.
comment:41 followup: ↓ 45 Changed 2 years ago by
Also, this ticket should undergo portability testing before it should be set to "positive review".
comment:42 in reply to: ↑ 39 ; followup: ↓ 50 Changed 2 years ago by
Replying to charpent:
./configure
should :
 suggest the installation of the relevant packages for older versions (recompiling the
gcc
smalah takes a geological age or two...), and suggest reconfiguring with the relevant arguments.
Where should this ... be usefully posted ?
Try #29586 (Improve configure
's recommendations). It would be helpful to include the messages that configure
prints on your system, and to explain what is missing.
comment:43 Changed 2 years ago by
 Status changed from needs_work to needs_review
comment:44 Changed 2 years ago by
 Reviewers set to Emmanuel Charpentier
Adding my name (as I should have initially...), but keeping it at needs_review
.
comment:45 in reply to: ↑ 41 Changed 2 years ago by
Replying to mkoeppe:
Also, this ticket should undergo portability testing before it should be set to "positive review".
I agree. But trac
lacks a state of tentative_positive_review
, which should be useful in similar situations (i. e. code changes with (knownpotential) platform issues...).
comment:46 in reply to: ↑ 40 Changed 2 years ago by
Replying to slabbe:
Reviewer name is missing, I am changing to needs work before the release manager needs to do it.
Damn ! I keep forgetting this. Old age ? Incipient Alzheimer's ?
Done, anyway.
comment:47 Changed 2 years ago by
 Reviewers changed from Emmanuel Charpentier to Emmanuel Charpentier, github.com/mkoeppe/sage/actions/runs/208788708, github.com/mkoeppe/sage/actions/runs/208788707
comment:48 followup: ↓ 55 Changed 2 years ago by
All clean on debian and ubuntu, waiting for more results
comment:49 in reply to: ↑ 30 ; followup: ↓ 51 Changed 2 years ago by
Replying to tmonteil:
I was working on upgrading Maxima too (as i did not see that ticket), so while i am not a symbolics expert, i fixed the
gamma_incomplete_lower
issue: that function was calledgamma_greek
by Maxima before.
Nice solution!
I also marked the intergration regression as known bug.
Would it be worth finding out when this regression was introduced in Maxima and try to add a patch to fix it?
comment:50 in reply to: ↑ 42 Changed 2 years ago by
Replying to mkoeppe:
Replying to charpent:
./configure
should :
 suggest the installation of the relevant packages for older versions (recompiling the
gcc
smalah takes a geological age or two...), and suggest reconfiguring with the relevant arguments.
Where should this ... be usefully posted ?
Try #29586 (Improve
configure
's recommendations). It would be helpful to include the messages thatconfigure
prints on your system, and to explain what is missing.
I would change the default ./configure outcomes of not finding gcc, gfortran, gmp, etc to Error. (which can be overridden by specifying explicit options)
comment:51 in reply to: ↑ 49 ; followup: ↓ 52 Changed 2 years ago by
Replying to pbruin:
Replying to tmonteil:
I was working on upgrading Maxima too (as i did not see that ticket), so while i am not a symbolics expert, i fixed the
gamma_incomplete_lower
issue: that function was calledgamma_greek
by Maxima before.Nice solution!
I also marked the intergration regression as known bug.
Would it be worth finding out when this regression was introduced in Maxima and try to add a patch to fix it?
It goes far beyond my skills, as i never wrote a single line of lisp.
The best i can do is to report the regression upstream and follow its evolution to untag the corresponding doctest once it is fixed. I am not sure this regression should prevent us to update Maxima.
comment:52 in reply to: ↑ 51 ; followups: ↓ 53 ↓ 54 Changed 2 years ago by
The best i can do is to report the regression upstream and follow its evolution to untag the corresponding doctest once it is fixed. I am not sure this regression should prevent us to update Maxima.
You can also open a new ticket for tracking that upstream report so we don't lose it (as opposed to only noticing the doctest change if Maxima fixes this, if anyone even tests the "known bug" doctests much). I agree that if there is significant improvement introduced, then in practice we have often upgraded on a utilitarian view of things.
comment:53 in reply to: ↑ 52 Changed 2 years ago by
Replying to kcrisman:
The best i can do is to report the regression upstream and follow its evolution to untag the corresponding doctest once it is fixed. I am not sure this regression should prevent us to update Maxima.
You can also open a new ticket for tracking that upstream report so we don't lose it (as opposed to only noticing the doctest change if Maxima fixes this, if anyone even tests the "known bug" doctests much). I agree that if there is significant improvement introduced, then in practice we have often upgraded on a utilitarian view of things.
Followup for tracking this issue: #30389
comment:54 in reply to: ↑ 52 Changed 2 years ago by
Replying to kcrisman:
The best i can do is to report the regression upstream and follow its evolution to untag the corresponding doctest once it is fixed. I am not sure this regression should prevent us to update Maxima.
You can also open a new ticket for tracking that upstream report so we don't lose it (as opposed to only noticing the doctest change if Maxima fixes this, if anyone even tests the "known bug" doctests much). I agree that if there is significant improvement introduced, then in practice we have often upgraded on a utilitarian view of things.
Possibly related : #30379 and the fossil tickets mentioned therein...
comment:55 in reply to: ↑ 48 Changed 2 years ago by
Replying to mkoeppe:
All clean on debian and ubuntu, waiting for more results
FWIW, worked again in upgrading to 9.2.beta9.
comment:56 Changed 2 years ago by
 Reviewers changed from Emmanuel Charpentier, github.com/mkoeppe/sage/actions/runs/208788708, github.com/mkoeppe/sage/actions/runs/208788707 to Emmanuel Charpentier, Matthias Koeppe
 Status changed from needs_review to positive_review
comment:57 Changed 2 years ago by
 Description modified (diff)
 Keywords upgrade maxima added
 Summary changed from Upgrade Maxima to 5.44.0 to Upgrade: Maxima 5.44.0
comment:58 Changed 2 years ago by
 Branch changed from u/tmonteil/upgrade_maxima_to_5_44_0 to 21e2feaa1188735241e988836a7d21bec258c741
 Resolution set to fixed
 Status changed from positive_review to closed
comment:59 Changed 2 years ago by
 Commit 21e2feaa1188735241e988836a7d21bec258c741 deleted
 Description modified (diff)
Patches need to be removed or updated.
Last 10 new commits:
doctest fixes
backport ECL PR #210
reject old makeinfo
add upstream fix from MR 215
add the patch from upstream MR 214
add upstream MR 216 (to fix cygwin fork)
Merge tag '9.2.beta2' into t/22191/public/packages/ecl20
Commit 75877dd8 from upstream
Merge branch 'public/packages/ecl20' of git://trac.sagemath.org/sage into t/30063/upgrade_maxima_to_5_44_0
build/pkgs/maxima: Update to 5.44.0