Opened 6 months ago

Last modified 3 weeks ago

#32867 needs_work enhancement

spkg-configure.m4 for maxima

Reported by: mjo Owned by:
Priority: major Milestone: sage-9.7
Component: build: configure Keywords: spkg-configure, maxima
Cc: slelievre, arojas Merged in:
Authors: Michael Orlitzky Reviewers: https://github.com/sagemath/sage/actions/runs/1992568379
Report Upstream: N/A Work issues:
Branch: u/mjo/ticket/32867 (Commits, GitHub, GitLab) Commit: 544b6014e4b88aba1d5c9b569878db95de184466
Dependencies: 33718 Stopgaps:

Status badges

Description

We handled ECL by itself in #29617, so now maxima needs its own ticket.

This would actually be "easy" at this point if not for our custom matrixexp.patch (https://sourceforge.net/p/maxima/bugs/2596/).

Change History (58)

comment:1 Changed 6 months ago by mjo

  • Authors set to Michael Orlitzky
  • Branch set to u/mjo/ticket/32867
  • Commit set to 88992bfae6d2aba0819b8cd67ae36621a5873b38

Here's an otherwise-working branch that fails only matrixexp.patch tests.


New commits:

ab54000Trac #32867: don't export MAXIMA_PREFIX.
0e29761Trac #32867: remove the MAXIMA sage_conf variable.
88992bfTrac #32867: new spkg-configure.m4 for maxima.

comment:2 Changed 6 months ago by slelievre

I opened #32898 so that removing matrixexp.patch has its own ticket.

comment:3 Changed 6 months ago by tornaria

This works for me on void linux.

Note that debian, arch, and I think also gentoo all include matrixexp.patch; I'll include it in void as well, so this ticket should not depend on #32898.

comment:4 Changed 6 months ago by slelievre

  • Cc slelievre added
  • Keywords spkg-configure maxima added

Maxima is packaged for Cygwin as "maxima", perhaps add a distros/cygwin.txt file?

Or that can be a separate ticket.

comment:5 Changed 6 months ago by mjo

We don't have the patch in Gentoo...

comment:6 Changed 6 months ago by git

  • Commit changed from 88992bfae6d2aba0819b8cd67ae36621a5873b38 to 3306df2b06195f55db92923a679fd3a15f08d0d0

Branch pushed to git repo; I updated commit sha1. New commits:

3306df2Trac #32867: maxima package information for Cygwin.

comment:7 follow-ups: Changed 6 months ago by tornaria

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

For the command line interface, it seems to me setting MAXIMA to "maxima" instead of "maxima -l ecl" would work (btw: why eliminate the config option MAXIMA and hardcode "maxima -l ecl" everywhere instead of just changing MAXIMA in a single place?)

For the library interface I don't know if it is possible to compile maxima.fas for sbcl; system packages for maxima that are usually compiled for sbcl don't include the library.

comment:8 in reply to: ↑ 7 Changed 6 months ago by mjo

Replying to tornaria:

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

Sadly it's not possible. Sage interfaces with maxima at a very low level through ECL; seesage/libs/ecl.pyx and sage/interfaces/maxima_lib.py.

For the command line interface, it seems to me setting MAXIMA to "maxima" instead of "maxima -l ecl" would work (btw: why eliminate the config option MAXIMA and hardcode "maxima -l ecl" everywhere instead of just changing MAXIMA in a single place?)

Given that we need ECL maxima for Sage, there aren't any other good values for that variable. The -l ecl is necessary to support system installations of maxima with (for example) both SBCL and ECL, where running just "maxima" would run it with SBCL. And the -l ecl flag doesn't do any harm with the maxima SPKG, because we know that the SPKG is built with ECL. The executable name "maxima" is standard and PATH manipulations in the sage build system ensure that the correct one gets used, so you're almost forced to have MAXIMA="maxima -l ecl".

With that in mind, removing the variable eliminates a "configurable" option that isn't actually configurable in practice, and also gets rid of a tiny bit of coupling between modules.

For the library interface I don't know if it is possible to compile maxima.fas for sbcl; system packages for maxima that are usually compiled for sbcl don't include the library.

No, until very recently even the ability to create an ECL library required a custom patch. That patch has finally been accepted upstream, but I don't think anything similar is possible with the other lisp implementations. (Sage is married to ECL for the moment either way.)

comment:9 Changed 5 months ago by mkoeppe

  • Milestone changed from sage-9.5 to sage-9.6

comment:10 Changed 5 months ago by git

  • Commit changed from 3306df2b06195f55db92923a679fd3a15f08d0d0 to 49197f57339401090d3d07cbba54e1c98ffdf946

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

a647806Trac #32867: don't export MAXIMA_PREFIX.
8cf10beTrac #32867: remove the MAXIMA sage_conf variable.
5009767Trac #32867: new spkg-configure.m4 for maxima.
d0f1651Trac #32867: maxima package information for Cygwin.
853ff79Trac #32867: disable maxima test that requires matrixexp.patch.
49197f5Trac #32867: add maxima package information for Gentoo.

comment:11 Changed 5 months ago by mjo

  • Status changed from new to needs_review

I've disabled (for now) the one remaining test that needs matrixexp.patch, with a pointer to #32898. Otherwise we could wait another decade for this.

comment:12 in reply to: ↑ 7 ; follow-up: Changed 5 months ago by nbruin

Replying to tornaria:

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

Adding to mjo's answer: for the expect-based interface, it may be doable to use SBCL. There are peculiarities in the way the stdin/stdout character streams are driven in different LISPs that may interfere with a simple drop-in replacement.

The main use in sage now, however, is through maxima_lib which is really using that the E in ECL means "embeddable": we link in the ecl, start it through its binary interface, and then instruct it to load maxima.fas. Our interfacing is using an ABI, not I/O streams, and we directly access the memory objects that ECL constructs to translate them to python (or leave them wrapped). I'm pretty sure this is all fundamentally impossible with SBCL. We are getting huge latency and efficiency gains through the ABI interface compared to I/O, so in many cases I suspect we can afford slower execution in ECL, and I think we don't want to go back to expect-based interfaces. That's why we're tied to ECL.

comment:13 in reply to: ↑ 12 Changed 5 months ago by tornaria

Replying to nbruin:

Replying to tornaria:

Would it be hard to support system maxima compiled with sbcl if available? It seems to be ~ 5-6 times faster than maxima on ecl (at least the time it takes to run the testsuite) and it is also more available in general.

Adding to mjo's answer: for the expect-based interface, it may be doable to use SBCL. There are peculiarities in the way the stdin/stdout character streams are driven in different LISPs that may interfere with a simple drop-in replacement.

I tried it (using sbcl maxima for the binary but maxima-ecl for maxima_lib) and I think it worked ok (as in: doctests passed).

The main use in sage now, however, is through maxima_lib which is really using that the E in ECL means "embeddable": we link in the ecl, start it through its binary interface, and then instruct it to load maxima.fas. Our interfacing is using an ABI, not I/O streams, and we directly access the memory objects that ECL constructs to translate them to python (or leave them wrapped). I'm pretty sure this is all fundamentally impossible with SBCL. We are getting huge latency and efficiency gains through the ABI interface compared to I/O, so in many cases I suspect we can afford slower execution in ECL, and I think we don't want to go back to expect-based interfaces. That's why we're tied to ECL.

In theory it may be possible (see http://www.sbcl.org/manual/#Calling-Lisp-From-C) although it might be a lot of work. The speed gains could be considerable for that part of sage, I don't know if it's overall worth the trouble.

comment:14 follow-up: Changed 5 months ago by mkoeppe

Isn't a version check for maxima necessary? On the oldest supported platform, system maxima is 5.32.1. https://repology.org/project/maxima/versions

comment:15 in reply to: ↑ 14 Changed 5 months ago by mjo

Replying to mkoeppe:

Isn't a version check for maxima necessary? On the oldest supported platform, system maxima is 5.32.1. https://repology.org/project/maxima/versions

In spirit yes, but in practice... maybe. The spkg-configure.m4 checks for a maxima with support for building maxima.fas as an ECL library, and that commit hasn't even made it into a released version of maxima yet. The only reason any system maxima would be carrying it as a custom patch is to be able to use maxima with sage, but that means that the system maxima probably works with sage. Up to some version (mis)matching anyway.

comment:16 Changed 5 months ago by git

  • Commit changed from 49197f57339401090d3d07cbba54e1c98ffdf946 to b615d8e1011a88032a310d1ad06907b198b5747d

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

8a56d85Trac #32867: don't export MAXIMA_PREFIX.
5ab8dacTrac #32867: remove the MAXIMA sage_conf variable.
0f9d66aTrac #32867: new spkg-configure.m4 for maxima.
38e916eTrac #32867: maxima package information for Cygwin.
c2aa14aTrac #32867: disable maxima tests that require matrixexp.patch.
b615d8eTrac #32867: add maxima package information for Gentoo.

comment:17 Changed 2 months ago by git

  • Commit changed from b615d8e1011a88032a310d1ad06907b198b5747d to bf826bb0acfe8f1d3ab81d0ca78a6a705756281d

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

6b5dc22Trac #32867: don't export MAXIMA_PREFIX.
25489cfTrac #32867: remove the MAXIMA sage_conf variable.
4655570Trac #32867: new spkg-configure.m4 for maxima.
88f5bb7Trac #32867: maxima package information for Cygwin.
511c067Trac #32867: disable maxima tests that require matrixexp.patch.
bf826bbTrac #32867: add maxima package information for Gentoo.

comment:18 Changed 2 months ago by mkoeppe

Removing the MAXIMA variable from sage_conf and just relying on PATH to find maxima is a regression on #29038/#30563 for #30818.

comment:19 Changed 2 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:20 Changed 2 months ago by git

  • Commit changed from bf826bb0acfe8f1d3ab81d0ca78a6a705756281d to 23b2d684ecb9746ae633a4db0230cf5bf92b1b46

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3890d03Trac #32867: new spkg-configure.m4 for maxima.
30f8b90Trac #32867: maxima package information for Cygwin.
308c81cTrac #32867: disable maxima tests that require matrixexp.patch.
59170f9Trac #32867: add maxima package information for Gentoo.
23b2d68Trac #32867: add "-l ecl" to all maxima invocations.

comment:21 follow-up: Changed 2 months ago by mjo

  • Status changed from needs_work to needs_review

Ok, I've put sage_conf.MAXIMA back, and have added -l ecl wherever it is used instead.

comment:22 in reply to: ↑ 21 ; follow-up: Changed 2 months ago by tornaria

Replying to mjo:

Ok, I've put sage_conf.MAXIMA back, and have added -l ecl wherever it is used instead.

What is wrong with just running maxima and use whatever is the system default?

In void linux we use system maxima which defaults to sbcl and there's no issue at all (it only needs a tiny patch due to numerical unstability on 32 bit sbcl: https://github.com/void-linux/void-packages/blob/master/srcpkgs/sagemath/patches/09-doctest_numerical_fix.patch).

In principle maxima is 5-6 times faster on sbcl than on ecl, although I don't know how much this affects sage.

Anyway, even if there are good reasons to default to maxima -l ecl, I think it's better that sage_conf.MAXIMA contains the whole command that runs maxima so it's easy to change to just maxima if one whishes so.

comment:23 in reply to: ↑ 22 Changed 2 months ago by mjo

Replying to tornaria:

What is wrong with just running maxima and use whatever is the system default?

In void linux we use system maxima which defaults to sbcl and there's no issue at all (it only needs a tiny patch due to numerical unstability on 32 bit sbcl: https://github.com/void-linux/void-packages/blob/master/srcpkgs/sagemath/patches/09-doctest_numerical_fix.patch).

I think that's the main problem with it: sage has only been tested with an ECL-only maxima until now. While SBCL might be fine(ish), what about gcl or clozurecl? If for example maxima is built supporting both gcl and ecl, and if using gcl causes the test suite to fail, it'd be nice if sage used the known-good ecl on that machine.

I don't feel strongly about it: on Gentoo we could force the problematic lisps to be disabled in maxima when sage is installed. Binary distros could theoretically run into trouble, but they're probably all using SBCL or ECL anyways, and one patch isn't too bad if they want the test suite to pass.

Does anyone else care if I drop the -l ecl?

In principle maxima is 5-6 times faster on sbcl than on ecl, although I don't know how much this affects sage.

We already use ECL for the maxima library interface, so there's probably some efficiency to not loading both SBCL and ECL. But really we should be trying to get rid of the pexpect interface entirely if efficiency is the goal (#30097).

comment:24 Changed 2 months ago by git

  • Commit changed from 23b2d684ecb9746ae633a4db0230cf5bf92b1b46 to 370275e82a6de148b320d71869689ecea7d2896d

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b170346Trac #32867: new spkg-configure.m4 for maxima.
b8632b1Trac #32867: maxima package information for Cygwin.
5ad9c20Trac #32867: disable maxima tests that require matrixexp.patch.
370275eTrac #32867: add maxima package information for Gentoo.

comment:25 follow-up: Changed 2 months ago by tornaria

Replying to mjo:

Replying to tornaria:

What is wrong with just running maxima and use whatever is the system default?

In void linux we use system maxima which defaults to sbcl and there's no issue at all (it only needs a tiny patch due to numerical unstability on 32 bit sbcl: https://github.com/void-linux/void-packages/blob/master/srcpkgs/sagemath/patches/09-doctest_numerical_fix.patch).

I think that's the main problem with it: sage has only been tested with an ECL-only maxima until now. While SBCL might be fine(ish), what about gcl or clozurecl? If for example maxima is built supporting both gcl and ecl, and if using gcl causes the test suite to fail, it'd be nice if sage used the known-good ecl on that machine.

I'm systematically testing sage with maxima-sbcl and it works ok save that tiny patch (it's a really tiny unstability in the fifth digit of the error bound for a numerical integral).

I don't feel strongly about it: on Gentoo we could force the problematic lisps to be disabled in maxima when sage is installed. Binary distros could theoretically run into trouble, but they're probably all using SBCL or ECL anyways, and one patch isn't too bad if they want the test suite to pass.

Does anyone else care if I drop the -l ecl?

Why not have the configure script tests for maxima -l ecl working (as before) and if so set MAXIMA="maxima -l ecl" in sage_conf.py. This way nothing really changes when using system maxima via configure but distro packages can use whatever works better for MAXIMA.

I should point out that maxima-sbcl, by default, will NOT use gmp for arithmetic, but some slow quadratic implementation. That will make the test in src/sage/interfaces/maxima.py:630 to take a very long time. To test this:

$ maxima -q --batch-string='showtime : true $ a : 10^(10^5) $ b : a^600 $'
(%i1) showtime:true
Evaluation took 0.0000 seconds (0.0000 elapsed) using 0 bytes.
(%i2) a:10^10^5
Evaluation took 0.0004 seconds (0.0000 elapsed) using 7.797 KB.
(%i3) b:a^600
Evaluation took 0.8037 seconds (0.8060 elapsed) using 23.760 MB.

This will take a very long time if arithmetic is slow. See: https://github.com/void-linux/void-packages/issues/34849.

This is the main reason why I think keeping the default to maxima -l ecl might be easier. Unless you want to run the command above with a timeout in spkg-configure and complain if the timeout triggers.

BTW, could you add my tiny patch from https://github.com/void-linux/void-packages/blob/master/srcpkgs/sagemath/patches/09-doctest_numerical_fix.patch to your branch? It's a numerical integration, the answer is exactly the same, but the error bound is sligthly different in the e-15 digit.

comment:26 follow-up: Changed 2 months ago by tornaria

In debian, there's a package maxima which ships a binary maxima (using gcl) and a package maxima-sage which ships a binary maxima-sage (using ecl) together with maxima.fas, so just running maxima -l ecl fails but maxima-sage -l ecl works. I didn't compile sage in debian so I don't know if maxima-gcl would work ok with sage.

Also, the location of maxima.fas in maxima-sage is not correct so ecl --eval "(require 'maxima)" --eval "(quit)" fails, but ecl --eval "(require 'maxima \"/usr/lib/ecl/maxima.fas\")" --eval "(quit)" works.

This is from a very quick test using docker run --security-opt seccomp=unconfined -it debian:latest (beware maxima doesn't run in docker without the seccomp option).

comment:27 Changed 2 months ago by git

  • Commit changed from 370275e82a6de148b320d71869689ecea7d2896d to 0beed918a704060f763b4a504dec86ecf7f5ba0f

Branch pushed to git repo; I updated commit sha1. New commits:

0beed91Trac #32867: fuzz some maxima test output to support system installs.

comment:28 in reply to: ↑ 25 ; follow-up: Changed 2 months ago by mjo

Replying to tornaria:

This will take a very long time if arithmetic is slow. See: https://github.com/void-linux/void-packages/issues/34849.

This is the main reason why I think keeping the default to maxima -l ecl might be easier. Unless you want to run the command above with a timeout in spkg-configure and complain if the timeout triggers.

A priori I consider this a packaging issue and not one that we need to worry about in sage. If a distro supplies a slow maxima... maxima will be slow. It still works though, and I see from the void bug that workarounds are available.

However: the point of that test was actually to test a memory limit, in #6772. The corresponding code is,

# Remove limit on the max heapsize (since otherwise it defaults                                                                                                      
# to 256MB with ECL).                                                                                                                                                
self._sendline(":lisp (ext:set-limit 'ext:heap-size 0)")

At least on my machine, the corresponding (default) limit is 4G, and that line of (ECL specific) code is not needed any longer. Can we just remove it along with the test? 256M is not a serious concern in 2022.

BTW, could you add my tiny patch from https://github.com/void-linux/void-packages/blob/master/srcpkgs/sagemath/patches/09-doctest_numerical_fix.patch to your branch? It's a numerical integration, the answer is exactly the same, but the error bound is sligthly different in the e-15 digit.

Done, thanks.

comment:29 in reply to: ↑ 26 Changed 2 months ago by mjo

Replying to tornaria:

In debian, there's a package maxima which ships a binary maxima (using gcl) and a package maxima-sage which ships a binary maxima-sage (using ecl) together with maxima.fas, so just running maxima -l ecl fails but maxima-sage -l ecl works. I didn't compile sage in debian so I don't know if maxima-gcl would work ok with sage.

Also, the location of maxima.fas in maxima-sage is not correct so ecl --eval "(require 'maxima)" --eval "(quit)" fails, but ecl --eval "(require 'maxima \"/usr/lib/ecl/maxima.fas\")" --eval "(quit)" works.

distros/debian.txt suggests installing both maxima and maxima-sage, but the library location issue should cause the SPKG to be built on debian. At least for now I am OK with that because it's not a regression and it makes this ticket easier to review. If however someone wants to try to fix those issues in a follow-up ticket I'd be happy to help.

comment:30 Changed 2 months ago by git

  • Commit changed from 0beed918a704060f763b4a504dec86ecf7f5ba0f to 9d38bd9a7b4bdabe35af9c81ace2046ccb715205

Branch pushed to git repo; I updated commit sha1. New commits:

9d38bd9Trac #32867: don't set a heap limit for the pexpect maxima.

comment:31 follow-up: Changed 2 months ago by mkoeppe

I think it would be good to figure out MAXIMA vs. SAGE_MAXIMA_COMMAND (see #33405)

comment:32 in reply to: ↑ 31 Changed 2 months ago by mjo

Replying to mkoeppe:

I think it would be good to figure out MAXIMA vs. SAGE_MAXIMA_COMMAND (see #33405)

There is no such environment variable. Or are you suggesting we standardize the names?

comment:34 in reply to: ↑ 33 Changed 2 months ago by mjo

Replying to mkoeppe:

https://github.com/sagemath/sage/blob/develop/src/sage/interfaces/expect.py#L137

That only gets executed if command is None. The Maxima subclass passes in command = '{0} -p {1}'.format(MAXIMA, shlex.quote(STARTUP)).

comment:35 follow-up: Changed 2 months ago by mkoeppe

Some of the interfaces pass command = os.getenv('SAGE_MATHEMATICA_COMMAND') or 'math -rawterm'. Maxima doesn't, so a questions is whether it should

comment:36 in reply to: ↑ 35 Changed 2 months ago by mjo

Replying to mkoeppe:

Some of the interfaces pass command = os.getenv('SAGE_MATHEMATICA_COMMAND') or 'math -rawterm'. Maxima doesn't, so a questions is whether it should

I don't see much value in it, but MAXIMA already comes from sage.env, so we could accomplish the same thing by renaming that variable to SAGE_MAXIMA_COMMAND.

More generally I think we're better off keeping the sage_conf values closer to what autotools detects, so that e.g. MAXIMA is the path of the "maxima" executable that was found. A path is more useful when you want to call something like subprocess.run, or any of the path-specific methods. I also think the environment variable overrides are mostly a pointless layer of indirection -- why have two ways to specify the same value? So ultimately I'd rather see SAGE_*_COMMAND disappear, but I don't want to hold this ticket up any more than necessary, so I'll go along with the consensus.

comment:37 in reply to: ↑ 28 ; follow-up: Changed 2 months ago by tornaria

Replying to mjo:

Replying to tornaria:

This will take a very long time if arithmetic is slow. See: https://github.com/void-linux/void-packages/issues/34849.

A priori I consider this a packaging issue and not one that we need to worry about in sage. If a distro supplies a slow maxima... maxima will be slow. It still works though, and I see from the void bug that workarounds are available.

Fair enough. The workaround is simple (and probably I should submit it upstream).

# Remove limit on the max heapsize (since otherwise it defaults                                                                                                      
# to 256MB with ECL).                                                                                                                                                
self._sendline(":lisp (ext:set-limit 'ext:heap-size 0)")

At least on my machine, the corresponding (default) limit is 4G, and that line of (ECL specific) code is not needed any longer. Can we just remove it along with the test? 256M is not a serious concern in 2022.

Did you try in 32 bits? The limit seems to be 1G. Anyway, the point of setting heap-size to 0 is to forget about the limit, since my computer has 32G > 4G. Indeed (64 bit void linux where ecl default heap limit is 4G):

$ time maxima -l ecl -q --batch-string='showtime : true $ a : 10^(10^5) $ b : a^20000 $'
;;; Loading #P"/usr/lib64/ecl-21.2.1/sb-bsd-sockets.fas"
;;; Loading #P"/usr/lib64/ecl-21.2.1/sockets.fas"
(%i1) showtime:true
Evaluation took 0.0000 seconds (0.0000 elapsed)
(%i2) a:10^10^5
Evaluation took 0.0010 seconds (0.0010 elapsed)
(%i3) b:a^20000
Maxima encountered a Lisp error:

 Memory limit reached. Please jump to an outer pointer, quit program and enlarge the
memory limits before executing the program again.

Automatically continuing.
To enable the Lisp debugger set *debugger-hook* to nil.

real	0m51.739s
user	0m50.231s
sys	0m1.031s

However setting heap-size to 0:

$ time maxima -l ecl -q --batch-string=":lisp (ext:set-limit 'ext:heap-size 0)
showtime : true $ a : 10^(10^5) $ b : a^20000 $"
;;; Loading #P"/usr/lib64/ecl-21.2.1/sb-bsd-sockets.fas"
;;; Loading #P"/usr/lib64/ecl-21.2.1/sockets.fas"
0
(%i1) showtime:true
Evaluation took 0.0000 seconds (0.0000 elapsed)
(%i2) a:10^10^5
Evaluation took 0.0000 seconds (0.0010 elapsed)
(%i3) b:a^20000
Evaluation took 101.7190 seconds (104.6690 elapsed)

real	1m45.805s
user	1m42.147s
sys	0m2.826s

For the sake of comparing with maxima/sbcl:

$ time maxima -q --batch-string='showtime : true $ a : 10^(10^5) $ b : a^20000 $'
(%i1) showtime:true
Evaluation took 0.0000 seconds (0.0010 elapsed) using 0 bytes.
(%i2) a:10^10^5
Evaluation took 0.0004 seconds (0.0000 elapsed) using 8.906 KB.
(%i3) b:a^20000
Evaluation took 41.2510 seconds (41.5055 elapsed) using 792.009 MB.

real	0m41.704s
user	0m39.644s
sys	0m1.725s

In the same spirit as your comment about slow maxima, maybe this should be patched in maxima itself, i.e. add (ext:set-limit 'ext:heap-size 0) to its initialization code for ECL, rather than have sage poke at this. It seems nicer if the lisp differences are abstracted out by maxima itself as much as possible.

comment:38 in reply to: ↑ 37 Changed 2 months ago by mjo

Replying to tornaria:

Did you try in 32 bits? The limit seems to be 1G. Anyway, the point of setting heap-size to 0 is to forget about the limit, since my computer has 32G > 4G. Indeed (64 bit void linux where ecl default heap limit is 4G):

I think even 1G is reasonable. This code predates the maxima library (introduced in #7287). When there was only a pexpect interface, it needed to be able to perform any computation asked of it. But today you have to go out of your way to both use the pexpect maxima and then to exceed the default limit.

In the same spirit as your comment about slow maxima, maybe this should be patched in maxima itself, i.e. add (ext:set-limit 'ext:heap-size 0) to its initialization code for ECL, rather than have sage poke at this. It seems nicer if the lisp differences are abstracted out by maxima itself as much as possible.

Yeah, that's an even better idea since it will affect maxima_console() and the system's "maxima" command as well. Looking through some old tickets I see that clisp also had memory limits in the past, so if they still exist, it would be nice for maxima to make them consistent.

comment:39 Changed 8 weeks ago by mjo

  • Reviewers set to https://github.com/sagemath/sage/actions/runs/1992568379

The GH action results for this are now visible and I don't think any of the failures are due to this ticket.

comment:40 Changed 8 weeks ago by mkoeppe

On archlinux-standard (https://github.com/sagemath/sage/runs/5569512407):

sage -t --warn-long 10.0 --random-seed=323017505472922216154275490909301858905 src/sage/interfaces/maxima_abstract.py
**********************************************************************
File "src/sage/interfaces/maxima_abstract.py", line 163, in sage.interfaces.maxima_abstract.MaximaAbstract._command_runner
Failed example:
    maxima._command_runner('describe', 'gcd')
Expected:
    -- Function: gcd (<p_1>, <p_2>, <x_1>, ...)
    ...
Got:
    ;;; Warning: Maxima is unable to set up the help system.
    (Details: CL-INFO::LOAD-PRIMARY-INDEX: Filesystem error with pathname #P"/usr/share/info/maxima-index.lisp".
    Either
     1) the file does not exist, or
     2) we are not allowed to access the file, or
     3) the pathname points to a broken symbolic link.)
    <BLANKLINE>
      No exact match found for topic `gcd'.
      Try `?? gcd' (inexact match) instead.
    <BLANKLINE>
                                         false
    <BLANKLINE>

comment:41 follow-up: Changed 8 weeks ago by mjo

  • Cc arojas added

Antonio, do you know what's going wrong in comment:40? I could add a check for maxima --very-quiet --batch-string="describe(gcd);" into spkg-configure but I think that it would be better if it just worked on arch.

comment:42 in reply to: ↑ 41 ; follow-ups: Changed 8 weeks ago by arojas

Replying to mjo:

Antonio, do you know what's going wrong in comment:40? I could add a check for maxima --very-quiet --batch-string="describe(gcd);" into spkg-configure but I think that it would be better if it just worked on arch.

I have no idea. This works just fine in my distro packages and when building this branch from source locally (which picks up system maxima). I suppose it's some docker weirdness, and adding an additional check to spkg-configure won't fix anything since the check will pass.

I noticed there's another failure

**********************************************************************
File "src/sage/interfaces/expect.py", line 608, in sage.interfaces.expect.Expect.quit
Failed example:
    maxima.quit(verbose=True)
Expected:
    Exiting Maxima with PID ... running .../bin/maxima...
Got:
    Exiting Maxima with PID 24915 running /usr/sbin/maxima -p /sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/sage/interfaces/sage-maxima.lisp
**********************************************************************

which is caused by using a wrong PATH:

2022-03-18T09:34:23.7683929Z PATH=/sage/build/bin:/sage/src/bin:/sage/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

/usr/sbin and /sbin should not be in PATH on Arch, those are symlinks to /usr/bin and having them in PATH before /usr/bin itself is known to cause issues with cmake (and from what I can see with autotools too). I don't know whether this is also causing the other failure (I don't see how it could, since the path printed in the error message is correct).

comment:43 in reply to: ↑ 42 Changed 8 weeks ago by mjo

Replying to arojas:

Replying to mjo:

I have no idea. This works just fine in my distro packages and when building this branch from source locally (which picks up system maxima). I suppose it's some docker weirdness, and adding an additional check to spkg-configure won't fix anything since the check will pass.

Ok, thanks. Matthias, how do you think we should proceed? Ignore the docker-only errors and try to fix them in another ticket? Try the spkg-configure check? I'd rather not break a CI target if we don't have to, but I'm also not keen on trying to debug a docker container for issues that don't appear in the base distro.

comment:44 Changed 8 weeks ago by arojas

Maybe try a re-run first in case this was a transient I/O error during installation of dependencies? It's the only reasonable explanation I can think of. (The failure caused by the wrong PATH will still happen, though)

comment:45 in reply to: ↑ 42 ; follow-up: Changed 8 weeks ago by mkoeppe

Replying to arojas:

/usr/sbin and /sbin should not be in PATH on Arch, those are symlinks to /usr/bin and having them in PATH before /usr/bin itself is known to cause issues with cmake (and from what I can see with autotools too).

This is just the standard PATH when running as root.

$ docker run -it archlinux:latest
Unable to find image 'archlinux:latest' locally
latest: Pulling from library/archlinux
4e498e04970c: Pull complete 
c5582fa67946: Pull complete 
Digest: sha256:45a6ee70ee27af0d343a678bc4382c7034d2184576e46c0e5207d5ab7f6dfd27
Status: Downloaded newer image for archlinux:latest
[root@d124a299445f /]# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

comment:46 in reply to: ↑ 42 Changed 8 weeks ago by mkoeppe

Replying to arojas:

**********************************************************************
File "src/sage/interfaces/expect.py", line 608, in sage.interfaces.expect.Expect.quit
Failed example:
    maxima.quit(verbose=True)
Expected:
    Exiting Maxima with PID ... running .../bin/maxima...
Got:
    Exiting Maxima with PID 24915 running /usr/sbin/maxima -p /sage/local/var/lib/sage/venv-python3.10/lib/python3.10/site-packages/sage/interfaces/sage-maxima.lisp
**********************************************************************

This can be easily fixed by just relaxing the doctest to look for ...bin/maxima... instead.

comment:47 Changed 8 weeks ago by git

  • Commit changed from 9d38bd9a7b4bdabe35af9c81ace2046ccb715205 to 77124a570d5aa846dc71f6b3cba2eba03bb7dbf3

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

53bb4a4Trac #32867: don't export MAXIMA_PREFIX.
11e83f1Trac #32867: new spkg-configure.m4 for maxima.
6243301Trac #32867: maxima package information for Cygwin.
50b4ccbTrac #32867: disable maxima tests that require matrixexp.patch.
1777260Trac #32867: add maxima package information for Gentoo.
6224bfcTrac #32867: fuzz some maxima test output to support system installs.
27cc783Trac #32867: don't set a heap limit for the pexpect maxima.
77124a5Trac #32867: don't require pexpect maxima to live in a "/bin" directory.

comment:48 Changed 8 weeks ago by mjo

I've updated that test to look for ...maxima..., since in theory, the user could put it anywhere so long as it's in his PATH.

comment:49 in reply to: ↑ 45 Changed 8 weeks ago by arojas

Replying to mkoeppe:

This is just the standard PATH when running as root.

$ docker run -it archlinux:latest
Unable to find image 'archlinux:latest' locally
latest: Pulling from library/archlinux
4e498e04970c: Pull complete 
c5582fa67946: Pull complete 
Digest: sha256:45a6ee70ee27af0d343a678bc4382c7034d2184576e46c0e5207d5ab7f6dfd27
Status: Downloaded newer image for archlinux:latest
[root@d124a299445f /]# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Ah, I see. I reported it to the docker image maintainers.

comment:50 follow-up: Changed 7 weeks ago by jhpalmieri

For what it's worth, after brew install maxima on OS X, config.log says

## ------------------------------------------------------- ##
## Checking whether SageMath should install SPKG maxima... ##
## ------------------------------------------------------- ##
configure:34270: checking whether any of ecl is installed as or will be installed as SPKG
configure:34280: result: no
configure:34285: checking for maxima
configure:34308: found /usr/local/bin/maxima
configure:34320: result: /usr/local/bin/maxima
configure:34335: checking if ECL can "require" the maxima module
An error occurred during initialization:
Module error: Don't know how to REQUIRE MAXIMA..
configure:34346: result: no
configure:34377: no suitable system package found for SPKG maxima

Is this the issue discussed in comment:8 and later?

comment:51 in reply to: ↑ 50 Changed 7 weeks ago by mjo

Replying to jhpalmieri:

For what it's worth, after brew install maxima on OS X, config.log says

configure:34335: checking if ECL can "require" the maxima module
An error occurred during initialization:
Module error: Don't know how to REQUIRE MAXIMA..

Is this the issue discussed in comment:8 and later?

Yes, that's going to be typical for a while. The homebrew maxima doesn't install the corresponding ECL library, because the upstream release doesn't install it yet. For now, only distros that have patched maxima specifically to work with sage will have have any success with this test. But until then, the fallback of building the SPKG should continue to work normally.

comment:52 follow-up: Changed 7 weeks ago by arojas

Please add distro info for Arch (package name is maxima-fas)

comment:53 in reply to: ↑ 52 ; follow-up: Changed 7 weeks ago by mjo

Replying to arojas:

Please add distro info for Arch (package name is maxima-fas)

distros/arch.txt already contains "maxima-ecl". Should "maxima-fas" replace that, or be appended to it?

comment:54 in reply to: ↑ 53 Changed 7 weeks ago by arojas

Replying to mjo:

Replying to arojas:

Please add distro info for Arch (package name is maxima-fas)

distros/arch.txt already contains "maxima-ecl". Should "maxima-fas" replace that, or be appended to it?

oh, sorry, I didn't expect such a file to exist before an spkg-configure was introduced. I have reworked the maxima packaging recently, the fas module is now in a separate maxima-fas package (which will in turn pull maxima as a dependency). So maxima-ecl sould be replaced.

comment:55 Changed 7 weeks ago by git

  • Commit changed from 77124a570d5aa846dc71f6b3cba2eba03bb7dbf3 to 544b6014e4b88aba1d5c9b569878db95de184466

Branch pushed to git repo; I updated commit sha1. New commits:

544b601Trac #32867: update package information in distros/arch.txt.

comment:56 Changed 6 weeks ago by tornaria

See #33718 for doctest failures using maxima 5.46.0 (just released).

comment:57 Changed 3 weeks ago by mjo

  • Dependencies set to 33718
  • Status changed from needs_review to needs_work

We may need a lower version bound if the doctest fixes in #33718 are backwards-incompatible. But either way we have to wait because this branch would accept maxima-5.46.0 and begin to fail tests.

comment:58 Changed 3 weeks ago by mkoeppe

  • Milestone changed from sage-9.6 to sage-9.7
Note: See TracTickets for help on using tickets.